Birdfeeder Application

With this app, Vizy sits in your backyard and patiently watches your birdfeeder, taking pictures of birds that visit your feeder, identifying their species, uploading the pictures to the cloud, and sending you important text updates. And it will identify unwanted visitors (e.g. squirrels) and administer a harmless squirt of water, if you wish.

It uses custom convolutional neural networks (CNNs) trained to classify 47 different North American bird species or 40 different European bird species.

• North American species: American Bushtit, American Crow, American Goldfinch, American Robin, Baltimore Oriole, Black-capped Chickadee, Blue Jay, Brown Creeper, Brown-headed Cowbird, California Scrub Jay, California Towhee, Carolina Chickadee, Carolina Wren, Cedar Waxwing, Chestnut-backed Chickadee, Common Grackle, Common Pigeon, Common Raven, Common Starling, Curve-billed Thrasher, Dark-eyed Junco, Downy Woodpecker, Eastern Bluebird, Eurasian Collared Dove, House Finch, Inca Dove, Indigo Bunting, Lesser Goldfinch, Mourning Dove, Northern Cardinal, Northern Mockingbird, Painted Bunting, Pileated Woodpecker, Pine Siskin, Red-Bellied Woodpecker, Red-winged Blackbird, Rose-breasted Grosbeak, Ruby-Crowned Kinglet, Song Sparrow, Spotted Towhee, Steller's Jay, Tufted Titmouse, Western Meadowlark, White-breasted Nuthatch, White-crowned Sparrow, White-throated Sparrow, and White-winged Dove.

• European species: Abyssian Roller, Black Woodpecker, Blue Tit, Carrion Crow, Chaffinch, Coal Tit, Common Blackbird, Common Cuckoo, Common Firecrest, Common Raven, Common Wood Pigeon, Dunnock, Eurasian Blackcap, Eurasian Collard Dove, Eurasian Hoopoe, Eurasian Jay, Eurasian Nuthatch, Eurasian Turtle Dove, Eurasian Wren, European Goldfinch, European Green Woodpecker, European Greenfinch, European Robin, European Roller, European Starling, Goldcrest, Golden Oriole, Great Tit, Grey-headed Woodpecker, Hawfinch, House Sparrow, Long-tailed Tit, Magpie, Marsh Tit, Song Thrush, Spotted Woodpecker, Stock Dove, Tree Sparrow, White Wagtail, and Willow Tit

Begin by turning on your Vizy and pointing your browser to it. (Please refer to the getting started guide if you need help with connecting to your Vizy, etc.) Run the Birdfeeder application by clicking on the ☰ icon in the upper right corner and selecting Apps/examples. Then scroll over to Birdfeeder in Apps, then click on Run.

The Birdfeeder application takes several seconds to start up. You'll be presented with a screen similar to below. The image at the top of the screen is the live video feed of what Vizy sees (your birdfeeder and its visitors). Various controls (buttons, sliders) are shown below the video feed, which we'll describe in more detail.

To test, you can hold pictures of birds in front of Vizy as shown:

You can also test with pictures of squirrels:

You'll notice that when detecting birds, Vizy will display the picture of the bird it detected and a timestamp in the media queue as shown.

Vizy keeps track of each bird between frames and does its best to determine when birds first enter the scene and when they have left the scene. When a bird leaves the scene, Vizy will pick a “good” picture and add it to the media queue. This way you can get a quick sense of recent birdfeeder activity by scrolling through the media queue's pictures. In general, each picture represents a different visitor.

At any time you can manually take a picture by clicking on Take picture, or take a video by clicking on Take video. This will take a picture or video and add it to the media queue. And at any time you can manually defend your birdfeeder by clicking on Defend. See the section on Defense below. The Brightness slider gives you control over the brightness of the pictures and live video feed.

Clicking on Settings will bring up the Settings dialog.

• Species of interest: Check the checkbox of species that you're interested in and clear the checkbox of species that you're not interested in. The species of interest will be logged in the media queue, including non-birds (squirrels, raccoons, etc.) if you wish.
• Pest species: Check the checkbox of species that you wish to be defended against. Non-birds (squirrels, raccoons, etc.) are considered pest species by default, but you can add other species here if you wish. Note, a given species can be both a species of interest and a pest species.
• Trigger species: Check the checkbox of the species that you want to trigger events – in particular, texting a picture of the detected species – and clear the checkbox of the species that you don't want to trigger events. See the section on Texting and Customized handlers.
• Detection sensitivity: Increasing the sensitivity will result in more detections, but possibly more false positive detections. Decreasing the sensitivity will result in fewer detections, but possibly more false negative detections.
• Defense duration: this control adjusts how long you want to “defend” your birdfeeder each time Vizy detects an unwanted visitor. It controls how long the squirt lasts, for example. See the section on Defense below.
• Record defense: Setting this will post the videos of Vizy defending your birdfeeder from unwanted visitors – you know, so you can make sure that the birdseed you've so generously provided makes it to the intended recipients.
• Upload to Google Photos: Check this if you want the media items in the media queue to be uploaded to Google Photos. Google services need to be configured, however. See the section on Configuring Google services.
• Share photos to help improve accuracy: you can share the Birdfeeder's Google Photos photo album with us (the Vizy team) so we can use the pictures to improve the accuracy of the Birdfeeder's detection and classification networks. We bootstrapped the Birdfeeder's networks with pictures from iNaturalist, but most of these pictures are not in birdfeeder contexts. By collecting Vizy's detections in the field we can use them (mostly the misclassifications) to help improve the Birdfeeder App's accuracy.
• Text new species: Vizy will keep track of all species it's seen and text you pictures of new species that visit your birdfeeder. See texting.
• Smooth video: Enabling smooth video will make the streaming video smoother, but increase the time it takes to detect and classify birds (latency).

Your Vizy can upload pictures and videos to Google Photos. In order for Vizy to upload photos to the cloud, you'll want to set up Google services.

Once you've set this up, other Vizy applications will have access to Google cloud services such as Photos, Gmail, Sheets, and Google Drive.

Vizy's texting service allows Vizy to send you updates (typically to your phone). See texting to learn how to configure the texting service. For example, Vizy can text pictures of new bird species that visit your birdfeeder (see settings and the picture below, from the Telegram smartphone app).

Or you can ask it to show you pictures of your birdfeeder's most recent visitors (see Text commands below). One of the advantages of texting is that you can interact with your Vizy from practically anywhere as long as your Vizy has a network connection. It's also quick!

Currently, the only text command that the Birdfeeder application supports is mrm (most recent media). For example, to get information (description and picture) of the most recent bird visitor:

Or you can get the N most recent visitors by adding a number:

Videos that you take or “defense videos” are supported by mrm as well.

You can either have Vizy sitting outside, next to your birdfeeder, all comfy and cozy in an outdoor enclosure, or you can point Vizy at your birdfeeder through a window, possibly using a zoom lens. In general, you want Vizy to be able to see a good amount of detail of the birds – the birds should occupy about 20% of the image. If they occupy much less than 10% of the image, much of the detail will be lost, and Vizy may not be able to reliably identify them.

This guide describes how to mount your Vizy outdoor enclosure.

Currently, the Birdfeeder app can identify bird species from two different geographic regions: North America and Europe. It's configured for North America by default. To switch to European birds, bring up the file /home/pi/vizy/etc/birdfeeder_consts.py in a text editor. Alternatively, you can use Vizy's built-in text editor by clicking on the ☰ button in the text editor and selecting birdfeeder_consts.py.

Find where the CLASSIFIER variable is set and comment-out the North American line and uncomment the European line as shown.

Once you change birdfeeder_consts.py you can simply click reload/refresh on your browser and Vizy will automatically restart the application for the changes to take effect.

When Vizy detects a pest, it can deliver a squirt of water to discourage said pest from returning. That is, Vizy can control a sprinkler valve (the type used for automatic irrigation systems), and in turn, the sprinkler valve can control water to a sprayer nozzle that's aimed at the birdfeeder.

Here are instructions about how to set something like this up.

For more advanced users who want to add their own custom features, the Birdfeeder application has handler code for various events and for text messages. The handler code is in /home/pi/vizy/apps/birdfeeder/handlers.py. Note, you can bring up the handler code easily from Vizy's built-in text editor by clicking on the ☰ button in the text editor and selecting handlers.py. Note also, once you change handlers.py you can simply click reload/refresh on your browser and Vizy will automatically restart the application and your code changes will take effect.

The event handler function is handle_event, which is called when an event occurs:

def handle_event(self, event):
    print(f"handle_event: {event}")

Here, the argument self is the Birdfeeder class object and event is a dictionary with various values depending on the event. In particular, the event_type value specifies the type of the event. The different types are listed below:

• species_of_interest: This event indicates that a species of interest has been identified. The image and timestamp are included.
• pest_species: This event indicates that a pest species has been identified. The image and timestamp are included.
• defend: This event indicates that it has very recently identified a pest species and has invoked its defense.
• register: This event indicates when a bird or non-bird has entered the scene. The birds and non-birds are listed in the dets field.
• deregister: This event indicates when a bird or non-bird has left the scene. The birds and non-birds are listed in the dets field.
• daytime: This event indicates when it has entered the “daytime” state and has enough light to reliably identify species.
• nighttime: This event indicates when it has entered the “nighttime” state and is inactive.

The text handler function is handle_text, which is called when Vizy receives a text message via Telegram. handle_text is called when none of Vizy's text handlers know how to handle the text message.

def handle_text(self, words, sender, context):
    print(f"handle_text from {sender}: {words}, context: {context}")

Here, the argument self is the Birdfeeder class object, and words is the list of words in the text message. sender is the person that sent the text and context is a list of contextual strings.

Create a share link for your Birdfeeder photostream by selecting the album in Google photos and sharing.

Then generate a QR code of the URL and put up a little sign.