Differences

This shows you the differences between two versions of the page.

--- wiki:birdfeeder_app [2021/10/18 15:30]
vizycam
+++ wiki:birdfeeder_app [2023/05/09 16:25] (current)
vizycam
@@ Line 1: / Line 1: @@
-====== Setting up Google services ======
+====== Birdfeeder Application ======
-Vizy uses several Google services such as Google Photos, Google Sheets, and Gmail to communicate to the outside world and share its findings.  The main advantage is that these are all free and robust services that many are familiar with using.
+{{https://vizycam.com/wp-content/uploads/2023/03/birdfeeder720h.mp4||loop,autoplay}}
-===== Create a Google account =====
+===== Overview =====
-It's strongly recommended that you create a new Google account exclusively for your Vizy camera instead of using a personal Google account.  If you have multiple Vizys, they can easily share a single Google account.  Using a separate account for your Vizy(s) will prevent any thorny security issues if the credentials are compromised for some reason (theft, etc.)  Particularly, you should keep all personal information (passwords, credit card numbers, etc.) free and clear of this account.  You can create a new Google account [[https://accounts.google.com/SignUp|here]].
+With this app, Vizy sits in your backyard and patiently watches your birdfeeder, taking pictures of birds that visit your feeder, identifying their species, uploading the pictures to the cloud, and sending you important text updates.  And it will identify unwanted visitors (e.g. squirrels) and administer a harmless squirt of water, if you wish. :-)
-===== Authenticating =====
+It uses custom convolutional neural networks (CNNs) trained to classify 47 different North American bird species or 40 different European bird species.
-Once you have a Google account for your Vizy, you need to give Vizy access credentials so it can access this account.  Begin by turning on your Vizy and pointing your browser to it.  (Please refer to the [[wiki:vizy_getting_started|getting started guide]] if you need help with connecting to your Vizy, etc.)  Click on the ☰ icon in the upper right corner and select ''%%Google Cloud%%''.
-{{wiki:image_471.jpg}}
+  * **North American species**: American Bushtit, American Crow, American Goldfinch, American Robin, Baltimore Oriole, Black-capped Chickadee, Blue Jay, Brown Creeper, Brown-headed Cowbird, California Scrub Jay, California Towhee, Carolina Chickadee, Carolina Wren, Cedar Waxwing, Chestnut-backed Chickadee, Common Grackle, Common Pigeon, Common Raven, Common Starling, Curve-billed Thrasher, Dark-eyed Junco, Downy Woodpecker, Eastern Bluebird, Eurasian Collared Dove, House Finch, Inca Dove, Indigo Bunting, Lesser Goldfinch, Mourning Dove, Northern Cardinal, Northern Mockingbird, Painted Bunting, Pileated Woodpecker, Pine Siskin, Red-Bellied Woodpecker, Red-winged Blackbird, Rose-breasted Grosbeak, Ruby-Crowned Kinglet, Song Sparrow, Spotted Towhee, Steller's Jay, Tufted Titmouse, Western Meadowlark, White-breasted Nuthatch, White-crowned Sparrow, White-throated Sparrow, and White-winged Dove.
-then click on ''%%Authenticate%%'':
+  * **European species**: Abyssian Roller, Black Woodpecker, Blue Tit, Carrion Crow, Chaffinch, Coal Tit, Common Blackbird, Common Cuckoo, Common Firecrest, Common Raven, Common Wood Pigeon, Dunnock, Eurasian Blackcap, Eurasian Collard Dove, Eurasian Hoopoe, Eurasian Jay, Eurasian Nuthatch, Eurasian Turtle Dove, Eurasian Wren, European Goldfinch, European Green Woodpecker, European Greenfinch, European Robin, European Roller, European Starling, Goldcrest, Golden Oriole, Great Tit, Grey-headed Woodpecker, Hawfinch, House Sparrow, Long-tailed Tit, Magpie, Marsh Tit, Song Thrush, Spotted Woodpecker, Stock Dove, Tree Sparrow, White Wagtail, and Willow Tit
+===== Getting started =====
-{{wiki:image_472.jpg}}
+Begin by turning on your Vizy and pointing your browser to it.  (Please refer to the [[wiki:vizy_getting_started|getting started guide]] if you need help with connecting to your Vizy, etc.)  Run the Birdfeeder application by clicking on the ☰ icon in the upper right corner and selecting **Apps/examples**.  Then scroll over to Birdfeeder in **Apps**, then click on **Run**.
-This will create a new browser tab, so if you have popups disabled, you may need to provide Vizy permission to bring up the new tab.  Within the new browser tab, you will be asked which account you want to use -- choose the Google account that you created for your Vizy.  It will likely require that you to type in the username and password.  You will then get a somewhat troubling message:
+{{wiki:image_1204.jpg?350}}
-{{wiki:image_473.jpg?500}}
+The Birdfeeder application takes several seconds to start up.  You'll be presented with a screen similar to below.  The image at the top of the screen is the live video feed of what Vizy sees (your birdfeeder and its visitors).  Various controls (buttons, sliders) are shown below the video feed, which we'll describe in more detail.
-What this is saying is that Google can't vouch for whatever you are giving access to.  Go ahead and click on ''%%Advanced%%''
+{{wiki:image_1196.jpg?600}}
-{{wiki:image_474.jpg?500}}
+To test, you can hold pictures of birds in front of Vizy as shown:
-and ''%%Go to Vizy (unsafe)%%''  Since it's a separate Google account, you can be confident that you're not doing anything too unsafe such as compromising any personal information, etc.
+{{wiki:image_1198.jpg?600}}
-Next, give Vizy access permissions to whatever it's asking for:
+You can also test with pictures of squirrels:
-{{wiki:image_476.jpg}}
+{{wiki:image_1199.jpg?600}}
-and press ''%%Continue%%''.
+==== Media Queue ====
-{{wiki:image_477.jpg}}
+You'll notice that when detecting birds, Vizy will display the picture of the bird it detected and a timestamp in the media queue as shown.
-You will then be given an access code that you must copy and paste into Vizy.  Go ahead and click on the copy icon and then paste the code into your Vizy browser tab as shown and click on ''%%Submit%%''.
+{{wiki:image_1200.jpg?700}}
-{{wiki:image_478.jpg}}
+Vizy keeps track of each bird between frames and does its best to determine when birds first enter the scene and when they have left the scene.  When a bird leaves the scene, Vizy will pick a "good" picture and add it to the media queue.  This way you can get a quick sense of recent birdfeeder activity by scrolling through the media queue's pictures.  In general, each picture represents a different visitor.
-If all went well, you will be presented with an ''%%Upload test image%%'' button:
-{{wiki:image_479.jpg}}
+===== Controls and Settings =====
-And you can go to [[https://photos.google.com|https://photos.google.com]] to see if the test image made it.
+At any time you can manually take a picture by clicking on **Take picture**, or take a video by clicking on **Take video**.  This will take a picture or video and add it to the media queue.  And at any time you can manually defend your birdfeeder by clicking on **Defend**.  See the section on [[wiki:birdfeeder_app#Defense|Defense]] below.  The **Brightness** slider gives you control over the brightness of the pictures and live video feed.
-{{wiki:image_480.jpg}}
+==== Settings dialog ====
-If so, nice work!  Your Vizy now has access to its very own Google account!  You can remove Vizy's access at any time by clicking on the ''%%Remove authentication%%'' button in the ''%%Google Cloud%%'' dialog.  (So Vizy better behave...)
+Clicking on **Settings** will bring up the Settings dialog.
+{{wiki:image_1438.jpg?400}}
+  * **Species of interest**: Check the checkbox of species that you're interested in and clear the checkbox of species that you're not interested in.  The species of interest will be logged in the media queue, including non-birds (squirrels, raccoons, etc.) if you wish.
+  * **Pest species**: Check the checkbox of species that you wish to be defended against.  Non-birds (squirrels, raccoons, etc.) are considered pest species by default, but you can add other species here if you wish.  Note, a given species can be both a species of interest //and// a pest species.
+  * **Trigger species**: Check the checkbox of the species that you want to trigger events -- in particular, texting a picture of the detected species -- and clear the checkbox of the species that you don't want to trigger events.  See the section on [[#Texting|Texting]] and [[#Customized handlers|Customized handlers]].
+  * **Detection sensitivity**: Increasing the sensitivity will result in more detections, but possibly more false positive detections.  Decreasing the sensitivity will result in fewer detections, but possibly more false negative detections.
+  * **Defense duration**: this control adjusts how long you want to "defend" your birdfeeder each time Vizy detects an unwanted visitor.  It controls how long the squirt lasts, for example.  See the section on [[wiki:birdfeeder_app#Defense|Defense]] below.
+  * **Record defense**: Setting this will post the videos of Vizy defending your birdfeeder from unwanted visitors -- you know, so you can make sure that the birdseed you've so generously provided makes it to the intended recipients.
+  * **Upload to Google Photos**: Check this if you want the media items in the media queue to be uploaded to Google Photos.  Google services need to be configured, however.  See the section on [[wiki:birdfeeder_app#Configuring Google services|Configuring Google services]].
+  * **Share photos to help improve accuracy**: you can share the Birdfeeder's Google Photos photo album with us (the Vizy team) so we can use the pictures to improve the accuracy of the Birdfeeder's detection and classification networks.  We bootstrapped the Birdfeeder's networks with pictures from iNaturalist, but most of these pictures are not in birdfeeder contexts.  By collecting Vizy's detections in the field we can use them (mostly the misclassifications) to help improve the Birdfeeder App's accuracy.
+  * **Text new species**: Vizy will keep track of all species it's seen and text you pictures of new species that visit your birdfeeder.  See [[wiki:texting|texting]].
+  * **Smooth video**: Enabling smooth video will make the streaming video smoother, but increase the time it takes to detect and classify birds (latency).
+==== Configuring Google services ====
+Your Vizy can upload pictures and videos to Google Photos. In order for Vizy to upload photos to the cloud, you'll want to [[wiki:google_cloud_setup3|set up Google services]].
+Once you've set this up, other Vizy applications will have access to Google cloud services such as Photos, Gmail, Sheets, and Google Drive.
+==== Texting ====
+Vizy's texting service allows Vizy to send you updates (typically to your phone).  See [[wiki:texting|texting]] to learn how to configure the texting service.  For example, Vizy can text pictures of new bird species that visit your birdfeeder (see [[wiki:birdfeeder_app#Settings dialog|settings]] and the picture below, from the Telegram smartphone app).
+{{wiki:img_4224.jpg?300}}
+Or you can ask it to show you pictures of your birdfeeder's most recent visitors (see [[wiki:birdfeeder_app#Text commands|Text commands]] below).  One of the advantages of texting is that you can interact with your Vizy from practically anywhere as long as your Vizy has a network connection.  It's also quick!
+=== Text commands ===
+Currently, the only text command that the Birdfeeder application supports is **mrm** (most recent media). For example, to get information (description and picture) of the most recent bird visitor:
+{{wiki:img_4222.jpg?300}}
+Or you can get the N most recent visitors by adding a number:
+{{wiki:img_4223.jpg?300}}
+Videos that you take or "defense videos" are supported by **mrm** as well.
+===== Customizations, tweaks, etc. =====
+==== Deploying Vizy ====
+You can either have Vizy sitting outside, next to your birdfeeder, all comfy and cozy in an [[wiki:outdoor_enclosure_getting_started|outdoor enclosure]], or you can point Vizy at your birdfeeder through a window, possibly using a [[wiki:zoom_getting_started|zoom lens]].  In general, you want Vizy to be able to see a good amount of detail of the birds -- the birds should occupy about 20% of the image.  If they occupy much less than 10% of the image, much of the detail will be lost, and Vizy may not be able to reliably identify them.
+This [[wiki:simple_outdoor_enclosure_mounting|guide ]] describes how to mount your Vizy outdoor enclosure.
+==== European bird species ====
+Currently, the Birdfeeder app can identify bird species from two different geographic regions: North America and Europe.  It's configured for North America by default.  To switch to European birds, bring up the file ''/home/pi/vizy/etc/birdfeeder_consts.py'' in a text editor.  Alternatively, you can use [[wiki:vizy_editor#Text editor|Vizy's built-in text editor]] by clicking on the ☰ button in the text editor and selecting ''birdfeeder_consts.py''.
+{{wiki:image_1193.jpg}}
+Find where the ''CLASSIFIER'' variable is set and comment-out the North American line and uncomment the European line as shown.
+{{wiki:image_1194.jpg}}
+Once you change ''birdfeeder_consts.py'' you can simply click reload/refresh on your browser and Vizy will automatically restart the application for the changes to take effect.
+==== Defense ====
+When Vizy detects a pest, it can deliver a squirt of water to discourage said pest from returning.  That is, Vizy can control a sprinkler valve (the type used for automatic irrigation systems), and in turn, the sprinkler valve can control water to a sprayer nozzle that's aimed at the birdfeeder.
+Here are [[wiki:setup_birdfeeder_defense|instructions]] about how to set something like this up.
+==== Customized handlers ====
+For more advanced users who want to add their own custom features, the Birdfeeder application has handler code for various events and for text messages.  The handler code is in ''/home/pi/vizy/apps/birdfeeder/handlers.py''.  Note, you can bring up the handler code easily from [[wiki:vizy_editor#Text editor|Vizy's built-in text editor]] by clicking on the ☰ button in the text editor and selecting ''handlers.py''.  Note also, once you change ''handlers.py'' you can simply click reload/refresh on your browser and Vizy will automatically restart the application and your code changes will take effect.
+{{wiki:image_1192.jpg}}
+=== Event handler ===
+The event handler function is **handle_event**, which is called when an event occurs:
+<sxh python>
+def handle_event(self, event):
+    print(f"handle_event: {event}")
+</sxh>
+Here, the argument **self** is the Birdfeeder class object and **event** is a dictionary with various values depending on the event.
+ In particular, the **event_type** value specifies the type of the event.  The different types are listed below:
+  * **species_of_interest**: This event indicates that a species of interest has been identified.  The **image** and **timestamp** are included.
+  * **pest_species**: This event indicates that a pest species has been identified.  The **image** and **timestamp** are included.
+  * **defend**: This event indicates that it has very recently identified a pest species and has invoked its defense.
+  * **register**: This event indicates when a bird or non-bird has entered the scene.  The birds and non-birds are listed in the **dets** field.
+  * **deregister**: This event indicates when a bird or non-bird has left the scene.  The birds and non-birds are listed in the **dets** field.
+  * **daytime**: This event indicates when it has entered the "daytime" state and has enough light to reliably identify species.
+  * **nighttime**:  This event indicates when it has entered the "nighttime" state and is inactive.
+=== Text handler ===
+The text handler function is **handle_text**, which is called when Vizy receives a [[wiki:texting|text message via Telegram]].  **handle_text** is called when none of Vizy's text handlers know how to handle the text message.
+<sxh python>
+def handle_text(self, words, sender, context):
+    print(f"handle_text from {sender}: {words}, context: {context}")
+</sxh>
+Here, the argument **self** is the Birdfeeder class object, and **words** is the list of words in the text message.  **sender** is the person that sent the text and **context** is a list of contextual strings.
+==== Promote it! ====
+Create a share link for your Birdfeeder photostream by selecting the album in Google photos and sharing.
+{{wiki:image_491.jpg}}
+Then generate a QR code of the URL and put up a little sign.  :-)
+{{wiki:image_490.jpg}}

Documentation

User Tools

Site Tools

Differences

Page Tools