Speech Therapy Application 2019-2020

From Mindworks
Jump to navigation Jump to search
Sponsors Micron Foundation
Team Name Dialectric
Duration Fall 2019 - Spring 2020
Faculty Adviser
  • Feng Li
  • Eric Booth
Team Members
  • Lucas Jackson
  • Ian Wood
  • Yiqing Ma

The goal of the project has been to continue the work of two previous teams for a speech therapy application intended for children. Specifically, this project has implemented a front-end user interface to interact with the last team's database system, a back-end server that controls data flow, and an expansion upon security to ensure compliance with medical data laws.


About 8% of children need speech therapy globally and while there are many speech therapy apps intended to help these children, most only supplement typical sessions, require a therapist to use, or are of little use for research. This project’s aim has been to create an all-in-one computer application for speech therapy utilizing last year’s database system by implementing: a dynamic user-interface, an overhauled security system, and features useful to children, therapists, and researchers, alike.

Problem Definition[edit]

This project is a continuation of two previous capstone teams’ work on a speech therapy application intended for children. The first team developed a proof-of-concept speech analyzer using an open-source library called Sphinx. The second team developed a java wrapper and database structure for the back-end of the application. This year, we have developed a front-end for the application, connected the last team's database program, and successfully integrated a robust security layer.

Overall, this project is eventually intended to assist children undergoing speech therapy with an application that provides speech lessons. These lessons can either prompt words or phrases with a text-to-speech generator or by the patient’s therapist and the patient’s response will be scored by comparing it to a matching phrase. All of the patient data and their recordings will then be stored and can be downloaded within the application for research use. This project will be especially useful for research as certain users will be able to download and filter the patient data, which can then be used to train and improve certain machine learning models.


  • A complete user interface for the application including but not limited to: a login screen, menu screen, help menu, lessons screen, and data viewing screen.
  • Expansion upon security and authentication to keep the project on track for COPPA and HIPAA compliance
  • A system to give therapists, caregivers, and researchers a view of patient records
  • A system allowing for the upload of patient audio files
  • A back-end server to handle data transfer and to control the security layer
  • Meaningful integration of the previous database system

Overall Design[edit]

This project utilizes Angular as the framework for the user interface/front-end and Java for the back-end. The back-end runs accesses databases hosted on Google Cloud through MySQL queries and passes data to the front-end through a controller system. The controller system communicates through HTTP requests, utilizing the Spring Boot Web library. A security layer in the back-end controls the JWT authentication system and generates or validates tokens for users based on data entered from the front-end.

Current High-Level Flow[edit]


Previous Project Database Structure[edit]


Design Specifics[edit]

The Angular project is split into multiple main components, which are in the ‘main’ file of the app and include:

  • Login Component
  • Register Component
  • Records Component
  • Upload Component
  • Services Component

Outside of 'main' exists a Dashboard component that provides the navigation and routes through the app. These front-end components all contain unique HTML, CSS, and TypeScript files to generate the live web page. The HTML and CSS portions define what can be viewed and the TypeScript file defines data flow and logic for the component. The front-end handles security in the 'services' folder by preventing page navigation if no authentication token is stored for the user.

The uploading is handled by a small Flask Python script hosted on the Google Cloud and stores audio files uploaded into a bucket. Integration of this part of the Google Cloud and the project still has some issues due to conflicts with both the Flask and Google Cloud libraries in the Google App Engine, where the Flask project is running. The Flask script also only uploads the file and does not add the into the SQL database which will be needed to keep track of which accounts have access to which audio files.

Tying into the Java back-end is the security layer utilizing Spring Boot Security which is used to handle the JWT authentication. The security layer is mostly held within the “security” folder in the back-end but also utilizes components from the “controllers”, “service”, and “payload” folders. The JwtTokenUtil.java class within “security” contains the methods to generate and validate tokens and serves as the main class utilizing Spring Security. It generates tokens based on the HS512 encryption algorithm and utilizes BCrypt for hashing. The JwtAuthenticationController.java class controls the /authenticate request sent from the front-end and passes the username and password data to be authenticated or to generate a new token. Tokens are generated using. JwtConfig.java serves as the configuration class to identify which requests (e.g. /authenticate) are exempt from authentication. This whole system works by parsing user data sent from the front-end and authenticating it with the database. If the user exists and they are trying to login, a token is generated and sent back to the front-end to be stored. If the authentication call is made after logging in, the token is sent to the security layer and is validated. If it is valid with what was generated then the user can continue with their request, if it is invalid then the token is removed and the user will be redirected back to the login screen.

Initial Specifications[edit]

  • The username and password data must be encrypted
  • User interface must be intuitive enough that children can use it
  • Back-end server must be able to call database wrapper functions
  • Certain parts of the user interface should only be accessible by specific user types
  • User interface should load the correct records accessible to that user
  • The user interface should ensure data can only be changed by valid users
  • Patient data must be encrypted
  • Back-end server must pass data correctly to the front-end
  • Authentication must send valid and unique tokens to users

Alternative Designs[edit]

  • Use React for the front-end
    • We initially considered using React instead of Angular for our front-end framework but settled on Angular. React was simpler to learn and did not host as large projects, but was missing many of the features Angular offered. Angular was chosen for its longevity and strong support from Google, its real-time compiled view for developing, and its component system
  • Use Javascript as the front-end
    • Javascript is considerably more lightweight than React or Angular, but it loses the strong typing of TypeScript offered by Angular and too many additional components would have been needed to integrate it in
  • Use another server service other than Google Cloud
    • This would have been less complex than using Google Cloud, but porting the database to a different system was deemed unnecessary in the scope of our project so we continued to use the same server as the last team
  • Record vs. upload audio
    • Originally we wanted to record audio directly through the application, but after initial testing we realized too many issues arise from browser permissions so we opted for uploading audio files instead. While outside the scope of our project, recording through the browser will eventually be ideal for the intended final product

Manufacturing Costs[edit]

Dialectric Manufacturing Plan.png

Project Learning[edit]

  • Authentication
  • Authentication Comparison.jpg

  • Front-end and Back-end Communication
  • DialectricAppCommunication.png
  • Angular
  • DialectricAngular.png

Project Link[edit]


Team Members[edit]

Lucas Jackson

Major: CS
Hometown: Fruitland, Idaho
Responsibilities: Team Lead, Wikimaster, Security Design
Email: jack6202@vandals.uidaho.edu

Ian Wood

Major: CE
Hometown: Boise
Responsibilities: Budget, Upload Design
Email: wood3802@vandals.uidaho.edu

Yiqing Ma

Major: CS
Hometown: China
Responsibilities: Front-End Design
Email: ma6724@vandals.uidaho.edu

Additional Documentation[edit]

Project Schedule

Schedule v1.0

Meeting Minutes

Minutes: 9/11/19
Minutes: 9/20/19
Minutes: 9/27/19
Minutes: 10/4/19
Minutes: 10/11/19
Minutes: 10/25/19
Minutes: 11/1/19
Minutes: 11/8/19
Minutes: 1/24/20


Concept Design Review
Engineering Release Review
Engineering EXPO Presentation