Raspberry Pi Image Processing
|Sponsors||Biological & Chemical Engineering Department
|Duration||Fall 2020 - Spring 2021|
|Faculty Adviser||Bruce Bolden|
The goal of this project is to integrate machine learning with the Raspberry Pi 4, leveraging the computational power of the Intel Neural Compute Stick 2 in order to get sufficient speeds to identify patches of weeds in farm plots.
- 1 Problem Definition
- 2 Background
- 3 Deliverables
- 4 Value Proposition Statement
- 5 Design Considerations
- 6 Project Learning
- 7 Quickstart Guide
- 8 Final Design
- 9 Validation
- 10 Team Members
- 11 Additional Documentation
Current computers suffer from being heavy and using high amounts of electricity. This prevents them from being carried by drones, which means that any sort of aerial image recognition is impossible.
With the advent of artificial intelligence, computers are now able to manage computationally-heavy tasks, such as facial recognition and self-driving vehicles. However, machine learning is typically limited by the processing speed of the computer, and is usually limited to bulky computers that are stationary or draw large amounts of power. The purpose of this project is to demonstrate the viability of machine learning on smaller machines, such as the Raspberry Pi, which could then be used for crop identification via drones. Since big computers are too heavy to be carried by a drone, the extremely light-weight Raspberry Pi would eliminate the problem of weight as well as the problem of high power draw.
- Develop a benchmark for speed comparisons between the Raspberry Pi with the NCS2, and a standard computer with a modern processor and graphics card.
- Implement a pre-trained artificial neural network with the Raspberry Pi.
- Use a standard image processing library to process images in real-time.
- Compare processing times between the computer and the Raspberry Pi.
Value Proposition Statement
Most modern farmers use small, Kerosene-fueled airplanes to spray pesticides onto their fields. Our goal is to instead, use a small computing system atop a drone to detect weeds and trigger a spraying mechanism in order to save farmers time and money. If successful, modern farmers won’t need to use small airplanes that are dangerous, use fossil fuels, take up space, and pollute the atmosphere. Our weed spraying drone will be easier to store, cost-effective, more accurate, and require brief human interaction. This could improve the quantity as well as quality of foods. Our object detection will allow us to target exact locations that need spraying, so it will reduce the amount of toxic chemicals going into the soil. Our project will be using a raspberry pi to explore the benefits of using smaller computers to do computationally heavy tasks with computer weight limits.
- Develop a sufficiently light-weight and low-power system.
- Train a neural network to accurately identify different types of weeds, specifically dandelions, clovers, and grass.
- Develop a program to interface with the Raspberry Pi remotely.
- Ensure our model runs quick enough for real-time computations.
- Utilize pre-trained models on the TensorFlow Github, which allows us to get a prototype up and running quicker.
- See the models here: Models
- Incorporating the NCS2 with the Raspberry Pi is well-documented and supported by the Raspberry Pi operating system.
- This tutorial is what we used to get the NCS2 working: OpenCV, NCS2, and the Raspberry Pi
This guide will walk you through the entirety of the steps we took to make this project work. Over the course of this guide, you will learn to set up TensorFlow, install Ubuntu, install the Raspbian operating system on a Raspberry Pi, train a custom machine learning model, optimize the model for the NCS2, and finally run the model on your Raspberry Pi with the help of the NCS2.
One of the biggest issues with this project was the immense dependence on 3rd-party libraries. Between TensorFlow, OpenVINO, and the Raspbian operating system, we ran into numerous dependency issues due to libraries being updated, halting development until we could find a version of the offending package that worked with the project. TensorFlow itself is the worst example of this, where the newest version of the library is actually less supported than the older versions. Our suggestion is to install the exact same versions of libraries that we have when listed, which will guarantee that the project will work for you even if the libraries are older. If exact versions cannot be found, the closer to the target version the better.
- This section will cover hardware and software requirements and where to get them.
- Raspberry Pi 4
- Micro SD card for the operating system, the larger the better.
- External battery or power source.
- Intel Neural Compute Stick 2
- Raspberry Pi Camera Module
- Computer with Windows installed, for TensorFlow usage.
- NVIDIA graphics card suggested for faster training, TensorFlow GPU is NOT compatible with AMD cards.
- Computer with Linux installed for OpenVINO model optimizer usage.
Installing TensorFlow on Windows
- This section will describe setting up our version of TensorFlow on Windows.
- The first component necessary for this process to work is Anaconda. Anaconda will allow you to create separate Python development environments, and more importantly, install the specific versions of libraries that we need for our training purposes. It is not recommended to proceed without utilizing Anaconda.
- Once Anaconda is installed, open the start menu and search for "Anaconda Prompt". Run this program as administrator.
- In this prompt, run the command "conda create --name tfgpu". This will create a new Anaconda environment named tfgpu. The name doesn't matter, but for future training you'll need to remember it in order to enable your development environment.
- Type "conda activate tfgpu" to open your development environment. The far left side of the prompt should now list (tfgpu) instead of (base).
- Next, we'll install TensorFlow.
- If training with your CPU, run TODO.
- Otherwise, run the command "conda install -c conda-forge tensorflow-gpu=1.15".
- Note: yIt's absolutely vital that you install version 1.15, or you risk issues with optimizing the model later on.
- TensorFlow will install a variety of libraries, if any fail to install you'll need to find the exact version that TensorFlow 1.15 requires.
Setting up the Linux Installation
- This section will guide you through installing Ubuntu for the model optimizer usage.
Setting up the Raspberry Pi Installation
- This section will teach you how to download and install Raspbian, and also to enable the camera module on the Raspberry Pi.
Training a Custom Model
- This section will teach you how to train a model, and also how to freeze the graph for future usage.
Setting up the Model Optimizer
- This section will teach you how to install the model optimizer on your Ubuntu installation, and how to optimize your frozen model for the NCS2.
Raspberry Pi and NCS2 Integration
- This section will finally set up the NCS2 on the Raspberry Pi, and will run a simple script to show that the model is running correctly.
|Requirement||Test||Test Subject||Target Date||Result||Recommendation|
|Rpi communicates with NCS2 stick and nueral network||Sync Rpi and NCS2 with camera to evaluate functionality||Physical prototype||11/18/20||TBA||TBA|
|Detect any kind of object using Rpi, NCS2, and Camera.||Using any object and Machine Learning model with Rpi integration.||Fully functional prototype||12/01/20||TBA||TBA|
|Evaluating multiple training and testing images to ensure 90% accuracy in detection||Train various models to determine if further improvements are necessary||Physical prototype||12/01/20||TBA||TBA|
|Correctly identify various weeds||Integreate fully trained model with 90% accuracy and Rpi model to ensure benchmark is met||Fully functional prototype||01/30/21||TBA||TBA|
|Rpi model is correct weight for drone||Create a platform for model to attach to the drone and verify correct usage of drone with model attached||Physical prototype||2/22/2021||TBA||TBA|
|Durable in certain weather conditions||Physically test outdoor conditions against model attached to drone to confirm functionality||Physical prototype||2/22/2021||TBA||TBA|
|90% accuracy/benchmark||Run object detection model on various trained images and determine further model development||Fully functional prototype||3/1/2021||TBA||TBA|
|Evaluate speed of image recognition by RPI and RPI + NCS2 during flight||Compare speeds of RPI and RPI with NCS2, compare power consumption for how long the RPI can operate on a battery.||Fully functional prototype||3/15/2021||TBA||TBA|
|Check for over heating.||Run Object detection model for 30 minutes.||Fully functional prototype||4/5/2021||TBA||TBA|
|Bounding box is within +/- 4 inches of target||Run object detection model on physical test subject||Fully functional prototype||4/15/2021||TBA||TBA|
Computer Science Student
Hometown: Meridian, ID
Hobbies/Interests: My professional interests include object detection, machine learning, and cyber security. In my free time I enjoy volleyball, camping, and PC building.
Plan for the Future: My plan is to further my understanding of machine learning and object detection to optimize and automate processes.
Computer Science Student
Hometown: Bend, OR
Hobbies/Interests: I enjoy working with Python and C#, and I have a significant amount of experience with both Raspberry Pi's and the Unity engine. I also like to dance, rock climb, and play guitar.
Plan for the Future: My goal is to work at Intel in Portland and eventually teach computer science someday.
Computer Science Student
Hometown: Coeur d’Alene, Idaho
Hobbies/Interests: My professional interests include all things cybersecurity. I am most interested in the topic of Digital Forensics and filesystem analysis. In my free time I like to create art, listen to music, and hang out with my friends.
Plan for the Future: My goal is to work for a Federal Executive agency pursuing a career in a cybersecurity-related field.
Computer Science Student
Hometown: Kathmandu, Nepal
Hobbies/Interests: I like everything related to AI and Machine Learning. In leisure, I also like to play soccer and go fishing.
Plan for the Future: My goal is work as a Machine Learning Engineer.