Cloud-Based Lab Instrumentation System

The goal of the project is to build a lab instrumentation system that allows for remote management of devices used in a lab setting, which includes both parameter monitoring as well as control over these devices. We want to build a system that can control a lab while using state-of-the-art security protections to safeguard lab data.

=Problem Definition=

Background
A previous project, [|0 App Controlled Poinsettia Covering System] was completed in 2018 that allowed for remote control over a covering system at one of the University of Idaho's greenhouses. We want to expand on this project in creating a system that can respond to event triggers as well as handle more types of data for monitoring. Additionally, there is a growing problem with security in IoT enabled devices, as many of them have been developed and rushed to the market without information assurance in mind, or the devices are incapable of implementing standard security features. We want to gather measurement data for research and agricultural purposes and do this securely to prevent unauthorized access.

Deliverables
A web application that can manage lab instrumentation.

Specifications
 See and visualize lab data in real time via a web browser Record and store sensor data with at least 1 sample per second. Examples of sensor data include temperature, humidity, air pressure, etc. Capture photos and videos at 10 FPS or higher Upload/download multimedia data Ability to set triggers that perform an action based on sensor reading Support multiple sites as well as multiple users at each site 

=Design Considerations=
 * A Raspberry Pi will be used to collect all data from the lab to be monitored.
 * The Google Cloud Platform
 * On the back-end side, we needed to make many decisions regarding the services we would use to implement the server side of the project. The first option we explored was building the backend by ourselves from scratch. This would involve either PHP, Python with some framework like Django, or NodeJS for writing the API, which would interface with some type of database, such as MySQL. We would be able to host this either on our own server or use some type of cloud platform. This option provides great flexibility and control to the developer and requires a bit more time to set up. The next option was using a cloud platform. We looked at the Google Cloud Platform and found it had the following tools available:
 * BigQuery or CloudSQL (MySQL on GCP)- data storage and analysis
 * Cloud Run – Deploying stateless/serverless containers
 * Cloud Storage – secure data storage
 * Identity platform – identity management and access control
 * Firebase Authentication, another service of Google, is also a great option for identity and access control
 * Pub/Sub – messaging service for data transfer between two+ nodes
 * Cloud Functions – serverless execution environment for building/connecting cloud services

Cloud Functions vs. In-House API
Cloud Functions provide an easy way to write simple HTTP endpoints that the front end can make a request to. They can also easily interface with other GCP features without the need to install libraries and modules. Cloud Functions could be used to connect to a MySQL database and make requests to manage users, sites, devices, and triggers, as well as handle the transfer of collected data from the Pi to the database and from the database to the front end for viewing.

User Authentication

 * Both Identity Platform and Firebase Authentication would be a good choice for user identification and access control
 * Both options have excellent documentation and would be easy to implement in our project

Pub/Sub

 * May take the place of some cloud functions or used with them
 * Can easily implement two way messaging between server or web and the Raspberry Pis
 * Could be useful for handling upstream data from Pi devices or sending messages to the Pis

Database Considerations
Google BigQuery Cloud SQL
 * Learning curve to implement
 * Designed to handle large amounts of data
 * Can auto scale as demand grows
 * We are all already familiar with the MySQL query language
 * Easiest implementation
 * Will be more expensive than BigQuery for a small project
 * A future project could easily use our database implementation on their own server to avoid GCP costs.

Frontend Web Interface
The frontend interface would be responsible for accepting user input, communicating with the backend to retrieve data or take certain actions, and then displaying the results to the user. We considered three potential options for building this part of the system: Angular, React, or pure HTML and JavaScript. Angular is a frontend web development framework that was released by Google in 2016. It allows code written in TypeScript to specify how a web application looks and works, which can simplify the development process. TypeScript is a superset of JavaScript and adds features related to type checking. While some of our team members are familiar with JavaScript, none had TypeScript experience, so there would likely have been a learning curve involved with this option. After the learning curve, though, we would be able to take advantage of significant pre-built functionality for applications like this including processing HTTP requests and displaying data tables. React is another frontend framework that is a competitor to Angular. React was made public by Facebook in 2013, and it uses JSX as its language. Instead of being an entirely new language, JSX is a combination of JavaScript and XML that can be used to specify how a web application looks and works. Our team also had no experience with JSX, so this would have involved a learning curve as well. Based on our research and feedback from the developer communities, however, React has a much easier learning curve than Angular, so we likely would have gotten started more easily. React provides a similar set of features to Angular and is also designed for building this type of web application. The final option we considered was using pure HTML and JavaScript to build the user interface. Support for both of these languages is already part of every modern web browser, so there would be no additional software needed. We had team members already familiar with these languages, which meant we could get started right away with no learning curve. While this option did not offer the pre-built functionality of the frameworks, it did not require any external dependencies, making the application lighter and reducing the amount of maintenance needed over time.

=Original Plan/End of Semester 1 Design=
 * We decided against using Google BigQuery and instead are using Google Cloud SQL to store sensor data, which is a MySQL Database that runs on Google's servers.
 * Google Cloud Functions are triggered from the Pi publishing a message to a Pub/Sub topic. These functions connect to Cloud SQL to store the data collected by the Pi's sensors.
 * A Cloud Function is also invoked via HTTP requests from the web interface to return sensor data from the database.
 * Google Firebase is used to handle the sign in of users. The uid will be used in the database to identify users.

Once data is received from the Pi via Pub/Sub, it will be stored in Google BigQuery. Google Cloud Run, a serverless computing platform gets input from Pub/Sub and moves it to BigQuery to be stored. The dataflow diagram below depicts how data is to move through the system. An image of our initial prototype is shown below.

=Final Design=

Backend Design

 * Firebase Authentication was used to handle access control and user identity.
 * After a user signs in, Firebase returns a token that is valid for 1 hour. This token is passed in with every HTTP request to the Cloud Functions.
 * The cloud function makes a request to Firebase to validate the token, which returns the user's uid. The function then executes if the token was valid.
 * The uid is a unique string that identifies the user. It is used throughout the system to ensure the user has access to the correct sites/data.
 * Instead of using BigQuery, we decided to use Cloud SQL since we are more familiar with the MySQL query language and BigQuery is better suited to large amounts of data.
 * The Cloud SQL database holds all data collected for a site, as well as keeps track of the sites and their users that exist in the system.
 * Pub/Sub was cut entirely from the project. It was determined that Pub/Sub would only add another layer of complexity as all requirements could be met using Cloud Functions only
 * Cloud Functions are the backbone of our backend. They carry out all tasks that interface between the database, the front end, and the Raspberry Pi devices.

Site/User Management Database
The schema below is from our site/user management database. This database is responsible for keeping track of the sites and users that exist in the system and the sites that a user is a member of.
 * The "user" table has an entry for each user containing the user's uid obtained from Firebase Authentication,
 * The "site" table contains an entry for each site that exists on the system.
 * The "site_user_role" table connects a user to a site, indicating the user is a member of that site. The role_id value sets the privliges of the user. If role_id = 0, they are an owner, and if role_id = 1, they are a standard user and can only view/collect data.

Site Database
The below schema is an example of a database for a site. Each site database has a "devices" table with an entry for each Raspberry Pi device that is attached to the site. A "parameters" table that keeps track of every parameter (sensor) that a Raspberry Pi device is collecting data from on the site. The "device_parameter" table connects a device with a parameter. There is also a "triggers" table that holds every trigger that exists on a device in the site. The trigger contains all information about what the device needs to do when a parameter reaches a specified value. A "device_trigger" table connects a device to a trigger, making the device monitor for when the trigger occurs. Finally, each parameter has its own data table that contains all data collected for that sensor from every device on the site.

Video Streaming
Over the course of the semester we set up a way to stream video from a Raspberry Pi to a user's browser, anywhere in the world. The way we accomplished this was using a virtual machine on Google Compute Engine. The Raspberry Pi uses a built-in utility called "raspivid" to capture video from the camera, and then uses a utility called ffmpeg to send the video stream to our server using the RTMP protocol. Our server has NGINX running, which allows the pages to be displayed, and also accepts the stream from the Raspberry Pi. The server then uses ffmpeg to transcode the video to the RTP protocol so that it can be ingested by software called Janus WebRTC Gateway. Janus takes the video from ffmpeg and makes it available in WebRTC format, which all of today's major browsers support - including on mobile. The user can then access a page on the server (we didn't quite get to incorporating it into our front end) and start the stream. In the testing that we've done at a few locations in the pacific northwest, the delay of the video was about half a second at thirty frames per second. This was with the video at a fairly low resolution, so the higher resolution streamed from the Raspberry Pi, the longer the delay would be before the user is seeing the video. Below is a diagram showing the process of getting video from the Raspberry Pi to the end user.

Frontend Design
We ultimately decided to use pure HTML and JavaScript for the frontend interface. This would provide us the most flexibility in setting up the interface exactly as we wanted it and with full support for the Firebase and Google Charts APIs. We implemented a set of functions for handling HTTP requests to our APIs and for transforming the response data into tables for display to users. This choice allowed us to avoid the learning curve and processing overhead of the larger frameworks. Additionally, the new features introduced a few years ago in ES6, the newest version of JavaScript, have helped to greatly bridge the gap between JavaScript and the features needed to build a modern web application.

The Finished Web Interface


=Validation Plan= =Team Members=

=Additional Documentation=

[|0 Project Schedule]

[|0 Meeting Minutes]

Presentations

[|0 Snapshot Presentation 1] [|0 Snapshot Presentation 2] [|0 Snapshot Presentation 3] [|0 Concept Design Review] [|0 Engineering Release Review] [|0 Design Expo Technical Presentation] [|0 Design Expo Poster]