<< return to Vizycam.com

User Tools

Site Tools


wiki:birdfeeder_app

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
wiki:birdfeeder_app [2021/10/18 15:34]
vizycam
wiki:birdfeeder_app [2023/05/09 16:25] (current)
vizycam
Line 1: Line 1:
 ====== Birdfeeder Application ====== ====== Birdfeeder Application ======
 +
 +{{https://​vizycam.com/​wp-content/​uploads/​2023/​03/​birdfeeder720h.mp4||loop,​autoplay}}
  
 ===== Overview ===== ===== Overview =====
  
-Vizy will sit in your backyard and patiently ​watch your birdfeeder, taking pictures of birds that visit your feeder, identifying their species, ​and uploading the pictures to the cloud. ​ And it will identify unwanted visitors (e.g. squirrels) and administer a harmless squirt of water (if you like) to teach them a lesson. :-)+With this app, Vizy sits in your backyard and patiently ​watches ​your birdfeeder, taking pictures of birds that visit your feeder, identifying their species, uploading the pictures to the cloud, and sending you important text updates.  And it will identify unwanted visitors (e.g. squirrels) and administer a harmless squirt of waterif you wish. :-)
  
-It uses custom ​TensorFlow network ​trained ​on 20 North American bird species ​(Bluebird, Cardinal, Sparrow, etc.) and 10 potential pest species ​(squirrels, cats, opossums, etc. +It uses custom ​convolutional neural networks (CNNs) ​trained ​to classify 47 different ​North American bird species ​or 40 different European bird species.  ​
  
-===== Getting started ===== 
  
-You can run the Birdfeeder application at any time.  Begin by turning on your Vizy and pointing your browser to it.  (Please refer to the [[wiki:vizy_getting_started|getting started guide]] if you need help with connecting to your Vizyetc.)  Click on the ☰ icon in the upper right corner and select ''​%%Apps/​examples%%'​'.+  * **North American species**American BushtitAmerican Crow, American Goldfinch, American Robin, Baltimore Oriole, Black-capped Chickadee, Blue Jay, Brown Creeper, Brown-headed Cowbird, California Scrub Jay, California Towhee, Carolina Chickadee, Carolina Wren, Cedar Waxwing, Chestnut-backed Chickadee, Common Grackle, Common Pigeon, Common Raven, Common Starling, Curve-billed Thrasher, Dark-eyed Junco, Downy Woodpecker, Eastern Bluebird, Eurasian Collared Dove, House Finch, Inca Dove, Indigo Bunting, Lesser Goldfinch, Mourning Dove, Northern Cardinal, Northern Mockingbird,​ Painted Bunting, Pileated Woodpecker, Pine Siskin, Red-Bellied Woodpecker, Red-winged Blackbird, Rose-breasted Grosbeak, Ruby-Crowned Kinglet, Song Sparrow, Spotted Towhee, Steller's Jay, Tufted Titmouse, Western Meadowlark, White-breasted Nuthatch, White-crowned Sparrow, White-throated Sparrow, and White-winged Dove
  
-{{wiki:image_482.jpg}}+  * **European species**Abyssian Roller, Black Woodpecker, Blue Tit, Carrion Crow, Chaffinch, Coal Tit, Common Blackbird, Common Cuckoo, Common Firecrest, Common Raven, Common Wood Pigeon, Dunnock, Eurasian Blackcap, Eurasian Collard Dove, Eurasian Hoopoe, Eurasian Jay, Eurasian Nuthatch, Eurasian Turtle Dove, Eurasian Wren, European Goldfinch, European Green Woodpecker, European Greenfinch, European Robin, European Roller, European Starling, Goldcrest, Golden Oriole, Great Tit, Grey-headed Woodpecker, Hawfinch, House Sparrow, Long-tailed Tit, Magpie, Marsh Tit, Song Thrush, Spotted Woodpecker, Stock Dove, Tree Sparrow, White Wagtail, and Willow Tit  
 +===== Getting started =====
  
-And then selecting ''​%%birdfeeder%%''​ from the apps list and clicking on ''​%%Run%%'':​+Begin by turning on your Vizy and pointing your browser to it.  (Please refer to the [[wiki:​vizy_getting_started|getting started guide]] if you need help with connecting to your Vizy, etc.)  Run the Birdfeeder application by clicking on the ☰ icon in the upper right corner and selecting **Apps/​examples**. ​ Then scroll over to Birdfeeder in **Apps**, then click on **Run**.  ​
  
-{{wiki:image_481.jpg}}+{{wiki:image_1204.jpg?350}}
  
-The birdfeeder ​application takes several seconds ​(~20 seconds) ​to load.  ​When it finishes loading, you'll be presented with a screen similar to below.  ​To test, you can hold pictures ​of birds in front of Vizy as shown:+The Birdfeeder ​application takes several seconds to start up.  ​You'll be presented with a screen similar to below.  ​The image at the top of the screen is the live video feed of what Vizy sees (your birdfeeder and its visitors). ​ Various controls (buttons, sliders) are shown below the video feed, which we'll describe in more detail.  ​
  
-{{wiki:image_483.jpg?500}}+{{wiki:image_1196.jpg?600}}
  
-Or you can hold pictures of squirrels:+To test, you can hold pictures of birds in front of Vizy as shown:
  
-{{wiki:image_486.jpg?500}}+{{wiki:image_1198.jpg?600}}
  
 +You can also test with pictures of squirrels:
  
-===== Deploying =====+{{wiki:​image_1199.jpg?​600}}
  
-You can either have Vizy sitting outside, next to your birdfeeder, all comfy and cozy in an [[wiki:​outdoor_enclosure_getting_started|outdoor enclosure]],​ or you can point Vizy at your birdfeeder through a window, possibly using a [[wiki:​zoom_getting_started|zoom lens]]. ​ In general, you want Vizy to be able to see a good amount of detail of the birds -- the birds should occupy about 20% of the image. ​ If they occupy 10% of the image or less, much of the detail will be lost, and Vizy won't be able to identify them.    ​+==== Media Queue ====
  
-===== Configuring Google services =====+You'll notice that when detecting birds, Vizy will display the picture of the bird it detected and a timestamp in the media queue as shown. ​
  
-In order for Vizy to upload photos to the cloud, you'll want [[wiki:google_cloud_setup|to give Vizy access to its own Google account]]+{{wiki:image_1200.jpg?700}}
  
-Once you've set this up, you can test by holding up a picture ​of bird in front of Vizy and see if the picture of the bird, once detected, is uploaded ​to the Google Photos page of the account.+Vizy keeps track of each bird between frames ​and does its best to determine when birds first enter the scene and when they have left the scene. ​ When a bird leaves the sceneVizy will pick a "​good"​ picture and add it to the media queue. ​ This way you can get a quick sense of recent birdfeeder activity by scrolling through ​the media queue'​s pictures. ​ In general, each picture represents a different visitor 
  
  
 ===== Controls and Settings ===== ===== Controls and Settings =====
  
-At any time you can manually take a picture by clicking on ''​%%Take picture%%''​.  This will take a picture and upload ​it to Google Photos.  And at any time you can manually defend your birdfeeder by clicking on ''​%%Defend%%''​.  See the section on Defense below. ​ The ''​%%Brightness%%'' ​slider gives you control over the brightness of the pictures ​-- the brightness of the live images is the same as the pictures that are uploaded to Google Photos.+At any time you can manually take a picture by clicking on **Take picture**, or take a video by clicking on **Take video**.  This will take a picture ​or video and add it to the media queue.  And at any time you can manually defend your birdfeeder by clicking on **Defend**.  See the section on [[wiki:​birdfeeder_app#​Defense|Defense]] ​below. ​ The **Brightness** slider gives you control over the brightness of the pictures ​and live video feed.
  
-{{wiki:​image_488.jpg}}+==== Settings dialog ====
  
 +Clicking on **Settings** will bring up the Settings dialog.  ​
 +
 +{{wiki:​image_1438.jpg?​400}}
 +
 +  * **Species of interest**: Check the checkbox of species that you're interested in and clear the checkbox of species that you're not interested in.  The species of interest will be logged in the media queue, including non-birds (squirrels, raccoons, etc.) if you wish.
 +  * **Pest species**: Check the checkbox of species that you wish to be defended against. ​ Non-birds (squirrels, raccoons, etc.) are considered pest species by default, but you can add other species here if you wish.  Note, a given species can be both a species of interest //and// a pest species.  ​
 +  * **Trigger species**: Check the checkbox of the species that you want to trigger events -- in particular, texting a picture of the detected species -- and clear the checkbox of the species that you don't want to trigger events. ​ See the section on [[#​Texting|Texting]] and [[#​Customized handlers|Customized handlers]].
   * **Detection sensitivity**:​ Increasing the sensitivity will result in more detections, but possibly more false positive detections. ​ Decreasing the sensitivity will result in fewer detections, but possibly more false negative detections.  ​   * **Detection sensitivity**:​ Increasing the sensitivity will result in more detections, but possibly more false positive detections. ​ Decreasing the sensitivity will result in fewer detections, but possibly more false negative detections.  ​
-  * **Defense duration**: this control adjusts how long you want to "​defend"​ your birdfeeder each time Vizy detects an unwanted visitor. ​ It controls how long the squirt lasts, for example. ​  +  * **Defense duration**: this control adjusts how long you want to "​defend"​ your birdfeeder each time Vizy detects an unwanted visitor. ​ It controls how long the squirt lasts, for example.  ​See the section on [[wiki:​birdfeeder_app#​Defense|Defense]] below. 
-  * **Seconds between pics**: If Vizy detects a bird at your birdfeeder, Vizy will continuously take pictures and upload ​the pictures to Google Photos.  ​The seconds between pics value controls ​how many pictures are uploaded over given amount of time.  ​For exampleif this value is set to 5 seconds12 pictures per minute will be the maximum possible picture rate  ​ +  * **Record defense**: Setting this will post the videos of Vizy defending ​your birdfeeder ​from unwanted visitors -- you knowso you can make sure that the birdseed you've so generously provided makes it to the intended recipients. ​  
-  * **Post pics with labels**: Setting this will post images with the detection boxes and labels.  +  * **Upload to Google Photos**: Check this if you want the media items in the media queue to be uploaded to Google Photos. ​ Google services need to be configured, however. ​ See the section on [[wiki:​birdfeeder_app#​Configuring Google services|Configuring Google services]]. ​    
-  * **Post pics of pests**:  ​Setting this will post images ​of unwanted visitors (pests).  ​+  * **Share photos to help improve accuracy**: you can share the Birdfeeder'​s Google Photos photo album with us (the Vizy team) so we can use the pictures to improve the accuracy of the Birdfeeder'​s detection and classification networks. ​ We bootstrapped the Birdfeeder'​s networks with pictures from iNaturalist,​ but most of these pictures are not in birdfeeder contexts. ​ By collecting Vizy's detections in the field we can use them (mostly the misclassifications) to help improve the Birdfeeder App's accuracy. ​   
 +  * **Text new species**: ​Vizy will keep track of all species it's seen and text you pictures ​of new species that visit your birdfeeder. ​ See [[wiki:​texting|texting]]. 
 +  * **Smooth video**: Enabling smooth video will make the streaming video smoother, but increase the time it takes to detect ​and classify birds (latency). 
 + 
 +==== Configuring Google services ==== 
 + 
 +Your Vizy can upload pictures ​and videos ​to Google Photos. In order for Vizy to upload photos to the cloud, you'll want to [[wiki:​google_cloud_setup3|set up Google services]].  
 + 
 +Once you've set this up, other Vizy applications will have access to Google cloud services such as Photos, Gmail, Sheets, and Google Drive. 
 + 
 + 
 +==== Texting ==== 
 + 
 +Vizy's texting service allows Vizy to send you updates (typically to your phone).  ​See [[wiki:​texting|texting]] to learn how to configure the texting service. ​ For example, Vizy can text pictures ​of new bird species that visit your birdfeeder (see [[wiki:​birdfeeder_app#​Settings dialog|settings]] and the picture below, from the Telegram smartphone app). 
 + 
 +{{wiki:​img_4224.jpg?​300}} 
 + 
 +Or you can ask it to show you pictures of your birdfeeder'​s most recent visitors (see [[wiki:​birdfeeder_app#​Text commands|Text commands]] below). ​ One of the advantages of texting is that you can interact with your Vizy from practically anywhere as long as your Vizy has a network connection. ​ It's also quick! ​    
 + 
 +=== Text commands === 
 + 
 +Currently, the only text command that the Birdfeeder application supports is **mrm** (most recent media). For example, to get information (description and picture) of the most recent bird visitor: 
 + 
 +{{wiki:​img_4222.jpg?​300}} 
 + 
 +Or you can get the N most recent visitors by adding a number: 
 + 
 +{{wiki:​img_4223.jpg?​300}} 
 + 
 +Videos that you take or "​defense videos" ​are supported by **mrm** as well.  
 + 
 +===== Customizations,​ tweaks, etc. ===== 
 + 
 +==== Deploying Vizy ==== 
 + 
 +You can either have Vizy sitting outside, next to your birdfeeder, all comfy and cozy in an [[wiki:​outdoor_enclosure_getting_started|outdoor enclosure]],​ or you can point Vizy at your birdfeeder through a window, possibly using a [[wiki:​zoom_getting_started|zoom lens]]. ​ In general, you want Vizy to be able to see good amount of detail of the birds -- the birds should occupy about 20% of the image.  ​If they occupy much less than 10% of the imagemuch of the detail will be lost, and Vizy may not be able to reliably identify them.     
 + 
 +This [[wiki:​simple_outdoor_enclosure_mounting|guide ]] describes how to mount your Vizy outdoor enclosure. ​  
 + 
 + 
 +==== European bird species ==== 
 + 
 +Currently, the Birdfeeder app can identify bird species from two different geographic regions: North America and Europe. ​ It's configured for North America by default. ​ To switch to European birds, bring up the file ''/​home/​pi/​vizy/​etc/​birdfeeder_consts.py''​ in a text editor. ​ Alternatively,​ you can use [[wiki:​vizy_editor#​Text editor|Vizy'​s built-in text editor]] by clicking on the ☰ button in the text editor and selecting ''​birdfeeder_consts.py''​. ​  
 + 
 +{{wiki:​image_1193.jpg}} 
 + 
 +Find where the ''​CLASSIFIER''​ variable ​is set and comment-out the North American line and uncomment the European line as shown. 
 + 
 +{{wiki:​image_1194.jpg}} 
 + 
 +Once you change ''​birdfeeder_consts.py''​ you can simply click reload/​refresh on your browser and Vizy will automatically restart the application for the changes ​to take effect. 
 + 
 +==== Defense ==== 
 + 
 +When Vizy detects a pestit can deliver a squirt of water to discourage said pest from returning. ​ That is, Vizy can control a sprinkler valve (the type used for automatic irrigation systems), and in turn, the sprinkler valve can control water to a sprayer nozzle that's aimed at the birdfeeder  
 + 
 +Here are [[wiki:​setup_birdfeeder_defense|instructions]] about how to set something like this up. 
 + 
 + 
 +==== Customized handlers ==== 
 + 
 +For more advanced users who want to add their own custom features, the Birdfeeder application has handler code for various events and for text messages. ​ The handler code is in ''/​home/​pi/​vizy/​apps/​birdfeeder/​handlers.py''​. ​ Note, you can bring up the handler code easily from [[wiki:​vizy_editor#​Text editor|Vizy'​s built-in text editor]] by clicking on the ☰ button in the text editor and selecting ''​handlers.py''​. ​ Note also, once you change ''​handlers.py''​ you can simply click reload/​refresh on your browser and Vizy will automatically restart the application and your code changes will take effect. 
 + 
 +{{wiki:​image_1192.jpg}} 
 + 
 +=== Event handler === 
 + 
 +The event handler function is **handle_event**, which is called when an event occurs: 
 + 
 +<sxh python>​ 
 +def handle_event(self,​ event): 
 +    print(f"​handle_event:​ {event}"​) 
 +</​sxh>​ 
 + 
 +Here, the argument **self** is the Birdfeeder class object and **event** is a dictionary ​with various values depending on the event.  
 + In particular, the **event_type** value specifies the type of the event. ​ The different types are listed below: 
 + 
 +  * **species_of_interest**:​ This event indicates that a species of interest has been identified. ​ The **image** and **timestamp** are included.  
 +  * **pest_species**:​ This event indicates that a pest species has been identified. ​ The **image** and **timestamp** are included.  
 +  * **defend**: This event indicates that it has very recently identified a pest species and has invoked its defense. 
 +  * **register**:​ This event indicates when a bird or non-bird has entered ​the scene. ​ The birds and non-birds are listed in the **dets** field  
 +  * **deregister**: This event indicates when a bird or non-bird has left the scene. ​ The birds and non-birds are listed in the **dets** field.  
 +  * **daytime**:​ This event indicates when it has entered the "​daytime"​ state and has enough light to reliably identify species. 
 +  * **nighttime**: ​ This event indicates when it has entered the "​nighttime"​ state and is inactive. 
 + 
 + 
 +=== Text handler === 
 + 
 +The text handler function is **handle_text**,​ which is called when Vizy receives a [[wiki:​texting|text message via Telegram]]. ​ **handle_text** is called when none of Vizy's text handlers know how to handle the text message.  ​
  
-==== Editing constants ==== +<sxh python>​ 
 +def handle_text(self,​ words, sender, context): 
 +    print(f"​handle_text from {sender}: {words}, context: {context}"​) 
 +</​sxh>​
  
-For more advanced and fine-grained configuration optionsyou can click on ''​%%Edit constants%%''​at ​the bottom of the ''​%%Settings%%''​ dialog. ​ This will bring up an editor ​in a separate browser tab so you can make edits to the consts file.  ​(If you have popups blocked, you may need to give Vizy permission to create ​the new tab. Within this file are configuration values for specific bird/animal species as well as ways to configure which species are considered pests. ​+Here, the argument **self** is the Birdfeeder class object, and **words** is the list of words in the text message.  ​**sender** is the person that sent the text and **context** is a list of contextual strings.  ​
  
-After editing and saving, you will need to restart the birdfeeder application for the changes to take effect. ​ (You can restart the application by selecting ​it again in the ''​%%Apps/​examples%%''​ dialog and clicking on ''​%%Run%%''​.)+==== Promote ​it! ====
  
-===== Defense =====+Create a share link for your Birdfeeder photostream by selecting the album in Google photos and sharing. ​
  
-When Vizy detects a pest, it can deliver a harmless squirt of water to discourage said pest from returning That is, Vizy can control a sprinkler valve (the type used for automatic irrigation systems), and in turn, the sprinkler valve can control water to a sprayer nozzle that's aimed at the birdfeeder.  ​+{{wiki:​image_491.jpg}}
  
-We'll post [[wiki:​birdfeeder_defense:​instructions]] about how to set something like this up. It's not too difficult...+Then generate a QR code of the URL and put up a little sign :-)
  
 +{{wiki:​image_490.jpg}}
  
wiki/birdfeeder_app.1634589251.txt.gz · Last modified: 2021/10/18 15:34 by vizycam