wiki:object_detector_app_2
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
wiki:object_detector_app_2 [2023/01/07 10:52] vizycam [Importing a project] |
wiki:object_detector_app_2 [2023/01/19 13:32] (current) vizycam [Importing, Exporting, and sharing Object Detector projects] |
||
|---|---|---|---|
| Line 54: | Line 54: | ||
| ==== Configuring Google services ==== | ==== Configuring Google services ==== | ||
| - | Your Vizy can upload pictures to Google Photos and interact with Google's Colab servers so you can train your own custom CNN's. In order for these things to happen, you'll want to [[wiki:google_cloud_setup3|set up Google services]]. | + | Your Vizy can upload pictures to Google Photo and interact with Google's Colab servers so you can train your own custom CNN's. You can also export projects to share with other Vizy users. You won't need to configure Google Services to import projects, however. In order for these things to happen, you'll want to [[wiki:google_cloud_setup3|set up Google services]]. |
| Once you've set this up, other Vizy applications will have access to Google cloud services such as Photos, Gmail, Sheets, Colab, and Google Drive. | Once you've set this up, other Vizy applications will have access to Google cloud services such as Photos, Gmail, Sheets, Colab, and Google Drive. | ||
| Line 105: | Line 105: | ||
| From the **File** menu select **New...** to bring up the **New project** dialog. | From the **File** menu select **New...** to bring up the **New project** dialog. | ||
| - | {{wiki:image_1378.jpg}} | + | {{wiki:image_1378.jpg?400}} |
| Type the name of the project into the text box and click on **Save**. For demonstration purposes, we'll create a network to detect rock, paper, and scissors hand gestures. | Type the name of the project into the text box and click on **Save**. For demonstration purposes, we'll create a network to detect rock, paper, and scissors hand gestures. | ||
| Line 118: | Line 118: | ||
| After creating a new project, you'll find yourself in the **Capture** tab. From here you can take various pictures of what you want to detect by clicking on the **Take picture** button. | After creating a new project, you'll find yourself in the **Capture** tab. From here you can take various pictures of what you want to detect by clicking on the **Take picture** button. | ||
| - | [{{wiki:image_1381.jpg}}] | + | [{{wiki:image_1381.jpg?600}}] |
| In our example, there are three "classes": rock, paper, and scissors hand gestures. We want our detector to work at various scales and orientations, so we vary the scale and orientation of the captured pictures. You'll want to take about 25 or more pictures of each detection class to provide a reasonably complete representation of what each class looks like to the CNN. Bear in mind that it may be obvious to you (as a human) that the two pictures below are scissors, but to the CNN, they look quite different. | In our example, there are three "classes": rock, paper, and scissors hand gestures. We want our detector to work at various scales and orientations, so we vary the scale and orientation of the captured pictures. You'll want to take about 25 or more pictures of each detection class to provide a reasonably complete representation of what each class looks like to the CNN. Bear in mind that it may be obvious to you (as a human) that the two pictures below are scissors, but to the CNN, they look quite different. | ||
| Line 134: | Line 134: | ||
| Switching to the **Training set** tab, you'll see all of the pictures you just captured. | Switching to the **Training set** tab, you'll see all of the pictures you just captured. | ||
| - | {{wiki:image_1384.jpg}} | + | {{wiki:image_1384.jpg?700}} |
| Click on one of the pictures in the grid to bring up the picture dialog. | Click on one of the pictures in the grid to bring up the picture dialog. | ||
| - | {{wiki:image_1329.jpg}} | + | {{wiki:image_1329.jpg?500}} |
| From here you can click and drag a rectangle around the object. The rectangle should include all parts of the object, but not much more. The rectangle doesn't need to be an exact fit, but it should be reasonably accurate -- refer to the picture above for a general idea. After selecting the rectangle, the label dialog will appear. Here, you can type in the name of the class. | From here you can click and drag a rectangle around the object. The rectangle should include all parts of the object, but not much more. The rectangle doesn't need to be an exact fit, but it should be reasonably accurate -- refer to the picture above for a general idea. After selecting the rectangle, the label dialog will appear. Here, you can type in the name of the class. | ||
| - | {{wiki:image_1331.jpg}} | + | {{wiki:image_1331.jpg?400}} |
| After typing in the name of the class, click on the **Add** button. Then click on the **Save** button to commit the label(s) for that picture. | After typing in the name of the class, click on the **Add** button. Then click on the **Save** button to commit the label(s) for that picture. | ||
| - | {{wiki:image_1333.jpg}} | + | {{wiki:image_1333.jpg?500}} |
| Once you've typed in the name of a class, you can choose the class from the dropdown menu for subsequent labels. | Once you've typed in the name of a class, you can choose the class from the dropdown menu for subsequent labels. | ||
| - | {{wiki:image_1385.jpg}} | + | {{wiki:image_1385.jpg?400}} |
| Continue to label all of the remaining pictures in this way. You can use the navigation buttons at the top of the **Training set** tab to navigate between pages (see below). | Continue to label all of the remaining pictures in this way. You can use the navigation buttons at the top of the **Training set** tab to navigate between pages (see below). | ||
| Line 160: | Line 160: | ||
| After you have labeled all images in the training set, you are ready to train your CNN and create a //model// for testing. From the **File** menu select **Train...** to bring up the **Train** dialog. | After you have labeled all images in the training set, you are ready to train your CNN and create a //model// for testing. From the **File** menu select **Train...** to bring up the **Train** dialog. | ||
| - | {{wiki:image_1339.jpg}} | + | {{wiki:image_1339.jpg?400}} |
| Click on **Upload training data**. Vizy will then get busy zipping-up all of the images and copying them into Google Drive. This will take some time, depending on how many images you have in the training set. After it has finished, click on **Train**, which will bring up a Google Colab browser tab. Why do we need Google Colab and what is Google Colab? [[https://research.google.com/colaboratory/faq.html|Google Colab]] is a Python notebook, similar to [[https://jupyter.org|Jupyter]]. It includes access to GPU resources for increased processing speed, especially for training CNN's. It's free to use, and ideal for training our CNN. Training CNNs is essentially a million-variable optimization problem. This requires a huge amount of computation! It's only recently that you can bring this level of computing to bear for free. Previously, you would need to purchase a high-end GPU card or compute time on a GPU-equipped AWS server, both of which are pricey. | Click on **Upload training data**. Vizy will then get busy zipping-up all of the images and copying them into Google Drive. This will take some time, depending on how many images you have in the training set. After it has finished, click on **Train**, which will bring up a Google Colab browser tab. Why do we need Google Colab and what is Google Colab? [[https://research.google.com/colaboratory/faq.html|Google Colab]] is a Python notebook, similar to [[https://jupyter.org|Jupyter]]. It includes access to GPU resources for increased processing speed, especially for training CNN's. It's free to use, and ideal for training our CNN. Training CNNs is essentially a million-variable optimization problem. This requires a huge amount of computation! It's only recently that you can bring this level of computing to bear for free. Previously, you would need to purchase a high-end GPU card or compute time on a GPU-equipped AWS server, both of which are pricey. | ||
| Line 178: | Line 178: | ||
| Almost immediately after it starts running, you'll get some extra messages, as shown below (click **OK**). | Almost immediately after it starts running, you'll get some extra messages, as shown below (click **OK**). | ||
| - | {{wiki:image_1342.jpg}} | + | {{wiki:image_1342.jpg?600}} |
| You will also be asked to give Google Colab access to Google Drive, as shown below (click on **Connect to Google Drive**). | You will also be asked to give Google Colab access to Google Drive, as shown below (click on **Connect to Google Drive**). | ||
| - | {{wiki:image_1343.jpg}} | + | {{wiki:image_1343.jpg?600}} |
| This will lead to a familiar Google authorization below. | This will lead to a familiar Google authorization below. | ||
| - | [{{wiki:image_1344.jpg}}] | + | [{{wiki:image_1344.jpg?400}}] |
| Choose the Google account associated with your Vizy, followed by clicking on **Allow**. | Choose the Google account associated with your Vizy, followed by clicking on **Allow**. | ||
| - | [{{wiki:image_1346.jpg}}] | + | [{{wiki:image_1346.jpg?400}}] |
| After navigating through these, Google Colab will get busy training your CNN. It will take several minutes depending on whether a server with GPU resources is available. The actual training takes place with the call to ''object_detector.create'' about halfway through the script. You can watch the detection loss (''det_loss'') decrease with each training epoch as shown below. It's learning! | After navigating through these, Google Colab will get busy training your CNN. It will take several minutes depending on whether a server with GPU resources is available. The actual training takes place with the call to ''object_detector.create'' about halfway through the script. You can watch the detection loss (''det_loss'') decrease with each training epoch as shown below. It's learning! | ||
| - | [{{wiki:image_1361.jpg}}] | + | [{{wiki:image_1361.jpg?850}}] |
| | | ||
| After it's done, it will copy the CNN model it just created back to Google Drive. This happens in the last script command (see below). The green check to the left of the command indicates that it was able to successfully create the CNN model and copy it. | After it's done, it will copy the CNN model it just created back to Google Drive. This happens in the last script command (see below). The green check to the left of the command indicates that it was able to successfully create the CNN model and copy it. | ||
| - | [{{wiki:image_1350.jpg}}] | + | [{{wiki:image_1350.jpg?850}}] |
| Congratulations! You're ready to test the CNN model you just created. | Congratulations! You're ready to test the CNN model you just created. | ||
| Line 206: | Line 206: | ||
| After the Google Colab script is finished, you can go back to the Vizy browser tab and from the **Train** dialog, click on **Download model**. | After the Google Colab script is finished, you can go back to the Vizy browser tab and from the **Train** dialog, click on **Download model**. | ||
| - | {{wiki:image_1389.jpg}} | + | {{wiki:image_1389.jpg?400}} |
| This will download the model file that the Colab script just created. After it's downloaded, it will run the model in the **Detect** tab so you can see how it performs. | This will download the model file that the Colab script just created. After it's downloaded, it will run the model in the **Detect** tab so you can see how it performs. | ||
| Line 219: | Line 219: | ||
| From the **Detect** tab, all detections are logged, including the incorrect detections. You can bring up all past detections in the **Detections** tab to get a better look. | From the **Detect** tab, all detections are logged, including the incorrect detections. You can bring up all past detections in the **Detections** tab to get a better look. | ||
| - | [{{wiki:image_1354.jpg}}] | + | [{{wiki:image_1354.jpg?850}}] |
| As you can see, there are several incorrect detections. Clicking on any of the pictures brings up the picture dialog for that picture. | As you can see, there are several incorrect detections. Clicking on any of the pictures brings up the picture dialog for that picture. | ||
| - | {{wiki:image_1355.jpg}} | + | {{wiki:image_1355.jpg?500}} |
| From here you can click on **Copy image to the training set**. It's recommended to go through all of the detections and copy all incorrect detections to the training set in this way. Next, switch to the **Training set** tab, and locate the copied images, which will appear on the last page as unlabeled images. | From here you can click on **Copy image to the training set**. It's recommended to go through all of the detections and copy all incorrect detections to the training set in this way. Next, switch to the **Training set** tab, and locate the copied images, which will appear on the last page as unlabeled images. | ||
| - | [{{wiki:image_1356.jpg}}] | + | [{{wiki:image_1356.jpg?850}}] |
| Go ahead and correctly label the images as we've done before. | Go ahead and correctly label the images as we've done before. | ||
| - | [{{wiki:image_1357.jpg}}] | + | [{{wiki:image_1357.jpg?850}}] |
| Line 238: | Line 238: | ||
| Sometimes none of the object classes are in the image, yet the model erroneously detects an object. These false positive detections are fairly common and can be pretty silly: "I'm 65% sure this thing in the image (chair) is a banana", etc. For example, below the model sees a phone, something it hasn't seen before, and makes a guess: "I'm 45% sure this thing (phone) is paper." | Sometimes none of the object classes are in the image, yet the model erroneously detects an object. These false positive detections are fairly common and can be pretty silly: "I'm 65% sure this thing in the image (chair) is a banana", etc. For example, below the model sees a phone, something it hasn't seen before, and makes a guess: "I'm 45% sure this thing (phone) is paper." | ||
| - | {{wiki:image_1358.jpg}} | + | {{wiki:image_1358.jpg?500}} |
| Copy these images to the training set also, but instead of labeling them, you'll leave them blank (no labels), which will tell the CNN "there are no objects of interest in this image". By doing so, you're giving the model more information about the world -- what's //not// an object of interest in this case -- so it can make a more accurate inference. | Copy these images to the training set also, but instead of labeling them, you'll leave them blank (no labels), which will tell the CNN "there are no objects of interest in this image". By doing so, you're giving the model more information about the world -- what's //not// an object of interest in this case -- so it can make a more accurate inference. | ||
| Line 249: | Line 249: | ||
| After you download the re-trained model, you will now have two model versions. In our example, the first version is ''rock paper scissors_01.tflite'' and the re-trained model is ''rock paper scissors_02.tflite''. The model versions simply increment in this way, so you can keep track and compare previous versions with newer ones. Along those lines, you can easily check to see if the re-trained model has improved by enabling **Test models** at the bottom of the **Training set** tab. When enabling **Test models**, it automatically selects the most recent version as the first model version (in this case ''rock paper scissors_02.tflite''). Selecting another model version in the 2nd dropdown allows you to do a simultaneous comparison to see how the two model versions behave. This is shown below where the boxes are red or green depending on whether the detection is version 02 or version 01, respectively. | After you download the re-trained model, you will now have two model versions. In our example, the first version is ''rock paper scissors_01.tflite'' and the re-trained model is ''rock paper scissors_02.tflite''. The model versions simply increment in this way, so you can keep track and compare previous versions with newer ones. Along those lines, you can easily check to see if the re-trained model has improved by enabling **Test models** at the bottom of the **Training set** tab. When enabling **Test models**, it automatically selects the most recent version as the first model version (in this case ''rock paper scissors_02.tflite''). Selecting another model version in the 2nd dropdown allows you to do a simultaneous comparison to see how the two model versions behave. This is shown below where the boxes are red or green depending on whether the detection is version 02 or version 01, respectively. | ||
| - | [{{wiki:image_1370.jpg}}] | + | [{{wiki:image_1370.jpg?850}}] |
| We can see that we improved (see below). (You can click on the individual pictures within the **Training set** tab to examine them more closely.) | We can see that we improved (see below). (You can click on the individual pictures within the **Training set** tab to examine them more closely.) | ||
| Line 296: | Line 296: | ||
| - **Create an album and copy these pictures into the album**. This can be done from your phone via the [[https://play.google.com/store/apps/details?id=com.google.android.apps.photos|Google Photos App]] or from your computer via a browser. | - **Create an album and copy these pictures into the album**. This can be done from your phone via the [[https://play.google.com/store/apps/details?id=com.google.android.apps.photos|Google Photos App]] or from your computer via a browser. | ||
| - **Share the album with [[wiki:google_cloud_setup3#creating-a-google-account|Vizy's Google account]]**. Similarly, you can do this from your phone via the [[https://play.google.com/store/apps/details?id=com.google.android.apps.photos|Google Photos App]] or from your computer via a browser. From the album you can click on **share** or click on the context menu to select **share**, then type in the Gmail address of the account you wish to share the album with. You may need to type in the complete Gmail address associated with your Vizy camera for the account to show up. | - **Share the album with [[wiki:google_cloud_setup3#creating-a-google-account|Vizy's Google account]]**. Similarly, you can do this from your phone via the [[https://play.google.com/store/apps/details?id=com.google.android.apps.photos|Google Photos App]] or from your computer via a browser. From the album you can click on **share** or click on the context menu to select **share**, then type in the Gmail address of the account you wish to share the album with. You may need to type in the complete Gmail address associated with your Vizy camera for the account to show up. | ||
| - | - **Import album pictures into your Object Detector project**. This will copy the images in the Google Photos album into the training set of the currently open project. Start by selecting **Import photos...** from the **File** menu, then type the name of the album into the text box as shown below. \\ \\ {{wiki:image_1410.jpg}} \\ \\ After clicking on **Import**, Vizy will locate the album, retrieve the images, and add them to the end of the training set. So after it's done, go to the last page(s) of the training set to see the imported images. Bear in mind that the album name is case-sensitive. If Vizy has trouble finding the album, make sure you can see the album from the Google Photos page while logged in via Vizy's Google account. Once the images are imported, you can [[wiki:object_detector_app_3#labeling|label them]] as before. Easy-peasy! | + | - **Import album pictures into your Object Detector project**. This will copy the images in the Google Photos album into the training set of the currently open project. Start by selecting **Import photos...** from the **File** menu, then type the name of the album into the text box as shown below. \\ \\ {{wiki:image_1410.jpg?400}} \\ \\ After clicking on **Import**, Vizy will locate the album, retrieve the images, and add them to the end of the training set. So after it's done, go to the last page(s) of the training set to see the imported images. Bear in mind that the album name is case-sensitive. If Vizy has trouble finding the album, make sure you can see the album from the Google Photos page while logged in via Vizy's Google account. Once the images are imported, you can [[wiki:object_detector_app_3#labeling|label them]] as before. Easy-peasy! |
| - | ===== Exporting, importing, and sharing Object Detector projects ===== | + | ===== Importing, exporting, and sharing Object Detector projects ===== |
| Through the powers of the Internet (and Google Drive), you can export your project and share your CNN efforts with others. And they can import your project to evaluate, improve, and share it back with you and possibly others. When exporting a project, all of the training set images, models, and settings are zipped up and uploaded to Google Drive. (Note, the detection images are not included when you export a project.) | Through the powers of the Internet (and Google Drive), you can export your project and share your CNN efforts with others. And they can import your project to evaluate, improve, and share it back with you and possibly others. When exporting a project, all of the training set images, models, and settings are zipped up and uploaded to Google Drive. (Note, the detection images are not included when you export a project.) | ||
| - | ==== Exporting a project ==== | + | ==== Importing a project ==== |
| - | From the **File** menu select **Export project...** to bring up the export dialog. Click the **Export** button. Vizy will then get busy zipping up the project and copying it to Google Drive. When it's finished it will present you will a **Copy share key** button. | + | From the **File** menu select **Import project...** to bring up the import dialog. Copy the share key into the text box and click the **Import** button. Vizy will then get busy downloading the zip file, unzipping, and installing the project. Easy! Note, you don't need to have [[#Configuring Google services|Google services]] configured to import projects. |
| - | {{wiki:image_1412.jpg}} | + | {{wiki:image_1413.jpg?400}} |
| - | Pressing this will copy the "share key" to your clipboard. The share key is just a jumble of text. You can save the key to a text file, email it, etc. With this key, someone can import your project. | + | Below is a share key for our rock paper scissors project. **Copy and paste it to give it a try!** |
| - | ==== Importing a project ==== | + | ''VWyJPRFBHIiwgInJvY2sgcGFwZXIgc2Npc3NvcnMiLCAiMXpDc2c4aVlIVkpaSk53a0xIeTlPdmJvcjJKaWFjYURlIl0=V'' |
| - | From the **File** menu select **Import project...** to bring up the import dialog. Copy the share key into the text box and click the **Import** button. Vizy will then get busy downloading the zip file, unzipping, and installing the project. Easy! | + | ==== Exporting a project ==== |
| - | {{wiki:image_1413.jpg}} | + | From the **File** menu select **Export project...** to bring up the export dialog. Click the **Export** button. Vizy will then get busy zipping up the project and copying it to Google Drive. When it's finished it will present you will a **Copy share key** button. |
| - | Below is a share key for our rock paper scissors project. Copy and paste it to give it a try. | + | {{wiki:image_1412.jpg?400}} |
| + | |||
| + | Pressing this will copy the "share key" to your clipboard. The share key is just a jumble of text. You can save the key to a text file, email it, etc. With this key, someone can import your project. | ||
| - | ''VWyJPRFBHIiwgInJvY2sgcGFwZXIgc2Npc3NvcnMiLCAiMXpDc2c4aVlIVkpaSk53a0xIeTlPdmJvcjJKaWFjYURlIl0=V'' | ||
| ===== Customized handlers ===== | ===== Customized handlers ===== | ||
| Line 359: | Line 360: | ||
| ===== Using custom CNN's in other programs ===== | ===== Using custom CNN's in other programs ===== | ||
| - | The projects for Object Detector are located in ''/home/pi/vizy/etc/object_detector/''. Within this directory are the project directories -- one for each project, and within each project directory are the two files: ''<project name>.tflite'' and ''<project name>.json''. These files are the latest CNN model version for this project. You can run these files on other generic Raspberry Pis, or you can export them to AI accelerators such as [[https://coral.ai/products/|Coral]] or [[https://www.canaan.io/product/kendryteai|Kendryte]] using a TensorFlow Lite compiler. Or you can use them in other Vizy programs that use TensorFlow, like the Vizy Tflite example. You would only need to change a single line of code (where the Tflite detector is instantiated) to use the desired CNN model instead of the default common objects model. | + | The projects for Object Detector are located in ''/home/pi/vizy/etc/object_detector/''. Within this directory are the project directories -- one for each project, and within each project directory are the two files: ''<project name>.tflite'' and ''<project name>.json''. These files are the latest CNN model version for this project. You can run these files on other generic Raspberry Pis, or you can export them to AI accelerators such as [[https://coral.ai/products/|Coral]] or [[https://www.canaan.io/product/kendryteai|Kendryte]] using a TensorFlow Lite compiler. Or you can use them in other Vizy programs that use TensorFlow, like the Vizy Tflite example. You would only need to change a single line of code (where the Tflite detector is instantiated) to use the desired CNN model instead of the default common objects model, as shown below. |
| <sxh python; title: Tflite example main.py> | <sxh python; title: Tflite example main.py> | ||
wiki/object_detector_app_2.1673110349.txt.gz · Last modified: 2023/01/07 10:52 by vizycam
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
