Pocket Microscope Slide Scanner
|Team Name||Team Apollo|
|Duration||2019 - 2020|
The goal of the project is to develop a handheld device that will allow doctors to scan slides whilst out in the field in order to detect signs of cancer in patients.
- 1 Problem Definition
- 2 Design Considerations
- 3 Project Learning
- 4 Final Design
- 5 Design Validation Plan
- 6 Project Poster
- 7 Team Members
- 8 Additional Documentation
According to the World Health Organization, 70% of deaths caused by cancer worldwide occur in developing countries (Chan, 2010). The overwhelming majority of these cases when diagnosed are late-stage terminal cancers. To alleviate this calamity, it is imperative that these countries have the accessibility to affordable cancer diagnostic equipment. The objective of this product is to provide a solution to diagnosing childhood cancers in developing countries where access to biomedical tools are limited.
Handheld device encapsulated in a harden case to protect against the elements and user abuse. Will allow a user through an Apollo App interface to scan, import, export, store image acquisitions and handle image processing. To include a fully functional prototype of the device, by late April 2020. The fully functional prototype includes an imaging system that is able to accept a standard microscope slide and control the region of interest, among other controls, via a smartphone app. Then, the smartphone app will tell the device to begin image acquisition and within 30 minutes an image of a 1 cm2 area of the slide will be available for distribution.
- Image magnification 400x (equivalent to a 40x objective commercial slide scan)
- Optimization for histology using hematoxylin & eosin stains (pink & purple)
- Use of daylight illumination (LED illumination optional)
- Auto white-balance
- +/-use existing camera in the smartphone
- No moving parts
- Resistant to dirt and dust (ie, if a surface needs to be cleaned, you rub it on your pants)
- Transfer image to smartphone by Bluetooth, wifi or wire/cable
- Optional in-device jpeg compression (in order to transmit image by smartphone in the field)
User Interface Requirements
The device will feature a smartphone interface via an Apollo app that supports the device functionality. This app will allow the user a live-view of the slide and controls for fine and coarse adjustment to adjust the focus of the camera. The images captured via the app will be saved as TIFF and JPEG file types to allow for easy management outside of the app and device. The user will have the ease of using an inductive charger and only one port for loading the slide. These features aid in ruggedness but also in usability of the product.
Optics and Camera Requirements
To achieve the desired scope and objective of the product, a 0.25 micron per pixel resolution is needed, which corresponds roughly to 20-40x zoom. The product, by client requirement, will feature a large format imaging capability of up to 1 cm2. To aid in taking high quality images, a 5600K LED light source is desired. The camera system will also feature an auto white-balance and an auto-focus feature.
Storage and Battery Requirements
Per charge, the battery should accommodate a minimum of 8000 acquisitions. The device will feature connectivity to online cloud storage and utilization of phone storage with secondary MicroSD card support, as back-up
The device will feature Wi-Fi connectivity (802.11 b/g/n) and Bluetooth 4.1 compliant. In addition, 4G LTE will be available when paired with a smartphone.
The enclosure of the device will feature a durable plastic, potentially ABS, and maintain function at approximately 100 kPa of compressive pressure. To protect the delicate optics system, a shock dampening system will be implemented along with minimal ports to prevent water seepage. The durability of the product is denoted by an Ingress Protection rating of 67 (IP67). It will also be designed to withstand a drop of up to 2 meters and feature scratch resistant coating. The usage of continuous solid pieces will be given preference, with a minimalist and simple design. Spatially, the final product shall fit within a maximum of 150mm in length, 80mm in width, and 30mm in depth, all while staying under a weight of 1 kg. All components will work within a 95% reliability standard.
The power for the device will come from an onboard battery. This battery will have a capacity of 4000mAh, or 20 W-h at 5.0V. This provides a 5.0V power source and allows an operational voltage range of 3.3-5.0V. The battery will support USB or inductive charging also.
There will need to be full app support for Android. This will include controls to adjust the field of view from the smartphone, adjust the ROI by controlling the stage, on the app. The app would also feature a photo stitching algorithm, built-in. The app features some good user interface features, such as a full-screen view of the sample and a 1cm2 image acquisition progress bar. A file viewer within the app to view the TIFF and JPEG files is needed.
Environmental and Regulatory Requirements
The device will maintain full functionality at a temperature range of 0-70°C. Device enclosure should provide protection against harmful dust and protection from immersion in water with a depth of up to 1 meter for up to 30 minutes as dictated by IP67 guidelines. For regulatory requirements, there are no UL, shipping, or production requirements. The cost of the final prototype must not exceed $1000 to build, including purchasing parts and labor.
Cancer diagnosis is a difficult task many physicians. Many forms of cancer such as prostate and cervical cancer are curable if caught early enough. Modern technology has improved cancer diagnostic techniques over the past several decades by introducing computer assisted image analysis to histology samples. These computer programs use parameterized algorithms that detect unique features within the sample such as variance in cell shape, nucleation, and disorganization (Figure 1) (Srivastava et al, 2015). This technology is known as computer assisted diagnosis (CAD) and can successfully detect various forms of carcinomas (He, Lei et al 2012). This technology has great implications for providing low-cost diagnostics to remote regions of the world that do not have access to healthcare. According to the World Health Organization, 70% of deaths caused by cancer worldwide occur in developing countries (Chan, 2010). The overwhelming majority of these cases when diagnosed are late-stage terminal cancers. To alleviate this calamity, it is imperative that these countries have the accessibility to affordable cancer diagnostic equipment. Ideally, a portable device that could image histology samples could provide the data for CAD to fix its malady.
Current products available for portable microscope slide scanners are either too expensive or do not provide the adequate resolving power for successful CAD. The uSCOPE MXII (Figure 2a) is advertised as a low-cost microscope scanner that resolves images up to 60X optical zoom at 0.25 microns per pixel and provides stitching capabilities. Though the features are promising, the unit is hardly low-cost, retailing for $10,0000. Another available product is the PathScan Enabler 5 low power scanner (Figure 2b) provides 4X magnification up to 2.54 microns per pixel. While this device costs significantly less at $1,900, it does not provide enough resolution for affective CAD.
In order to achieve the proper resolution, size and cost for providing affective cancer diagnostic tools to remote regions of the world, the design philosophy of the device needs to be revaluated. Physicians in the field require battery powered device that is both hand-held and rugged to withstand dust, dirt and rain in hostile conditions. The device should operate at IP67 standards and maintain functionality while enclosing all components in a small enclosure (Association, National Electrical Manufacturers, 2004).
In order to maintain small size, a compact optics system is necessary. One-low cost compact lens product that is available is the Nurugo (Figure 3). This is compact lens provides up to 15X resolving power and has universal adapters to connect to most smart phones. This device is marketed toward amateur microscope enthusiasts at the price of $65 but proves that high-magnification can be achieved with a small device. Another option for achieving high magnification is using a compact device is using a Raspberry Pi Cam V2.1 and a Raspberry Pi Zero in conjunction with 40X microscope objective (Figure 4). This configuration can resolve 0.075 microns per pixel (PublicLab, 2019).
In order to montage large format images, the device must have a native image stitching feature. In conjunction with an electronic stage, images of the sample can be stitched together sequentially. The calculated image size for a 1cm2 field of view at 0.25 microns per pixel ~ 1.60 gb at 1600 megapixels (approximately 1mb= 1 megapixel) Figure 5 shows an example of an Open CV Python stitching alorithim that can be used in conjuction of an automated x-y stage to produce large format images (Roseblock, 2018).
The following link gave us better insight into image stitching algorithms:
- Image Stitching
OpenCV, or open source computer vision library, is a collection of open source computer vision and machine learning software libraries. We will be using OpenCV’s python libraries to construct our image stitching library. After multiple tests, we have found that most of the algorithms are factorial, O(n!). This means that we will need substantial hardware to run the algorithms on stitching 1600 image captures, which is not feasible to because of the size requirement of device. We will need to develop an algorithm that runs linear, O(n). Following are the links we have used for OpenCV:
- Python stitching
- Python and OpenCV stitching
- Real time panoramic stitching
- Install OpenCV on Windows
- OpenCV courses
- uSCOPE image stitching example
Flutter is the application we will be using to develop the app for either the standalone or addon device. This application allows us to program in a single language developed by Google, DART, while simultaneously developing for both Apple and Android platforms. In order to use Python with DART, we will need to use StarFlut which is a library that allows DART to communicate with Python. DART will be used to create the GUI while Python will be used to build the image stitching algorithm:
We have been using a Raspberry Pi 0 W for the standalone device build. The Pi will be what the phone communicates with in order to control the standalone as well as act as the main picture capture device. Because of the low computational power of the Pi 0, all image stitching will be done on the phone. Following is documentation on all Pi camera commands and a guide on how to setup a raspberry Pi:
Adafruit DC & Stepper Motor HAT for Raspberry Pi
To control the two stepper motors and single DC motor that controls the stage movement, we will be using a motor hat from Adafruit. The motor hat interfaces with the Raspberry Pi 4's GPIO pins and uses a 15V power source to supply power to the motors.
For early stage prototyping, VNC will be used to control the standalone device. VNC is a free remote desktop control software. Only drawback of VNC is the inability to show the camera preview from the Pi. Following is documentation on how to setup and run VNC:
The stage is a crucial component of our design that will move the microscope slide to take images of the desired area. The stage design is divided in to three major components: the X, Y and Z stage sub assembly.
The X and Y sub-assemblies have same working principle which resembles the design concept of our client’s scanner in their lab. These assemblies consist of lead screw, stepper motor, guide rails and flange nut. The stepper motor has a turn angle of 1.8 degrees for each step and is connected to a lead screw with travel distance of 0.3 mm. Two flange nuts will be used to hold the carriage platform which connect the X axis with the Y axis. Rails will be used in each axis to constrain the motion of the carriage in one axis. Ball bearing will be used at the end of lead screw to allow free rotation. Each step of the stepper motors will result in travel distance of 0.0015 mm.
The Z stage sub assembly has unique design to reduce the size and number of moving components. Two rails will be used to limit the motion in Z axis and two Nautilus Gear design gear will be used to move the Z stage and provide stability to its movement.
The exploded view, bill of materials and individual drawing is shown in this section.
The X and Y sub assembly share multiple parts, for example the stepper motor, guide rails, lead screw, ball bearing and fittings. The bill of materials and parts unique to Y sub assembly are included in this drawing packet.
The Z stage has a simple bill of materials, with most of the components being related to the DC motor.
In a cycle the software will take two images on column 1 (Y-Axis). This would be image Start and 2 of the row (X-Axis). It will then stitch these 2 images together into a new master image for that column. The program will then delete the old Start Image and Image 2 and place the new stitched image in the directory. Then it will increment the X-Axis index to then stitch the new master image with image 3. It will do this until it reaches the end of the column, stitching and replacing the column image. After it reaches the end it will do a search from the Y-Axis +1 image at the end of the column. Except this time it will decrement the X-Axis index to capture the n-1 image and n image to repeat the Stitch and Replace method. Once the program reaches the end by looking to the n+1 X-Axis and Y-Axis and seeing nothing it will then proceed to stitch all column images created.
Stage, Camera, & Limit Switch Code
Stage movement, picture taking, and limit switch detection will all be done in Python using an open source motor control library. The program will initialize the stage to zero position, setup limit switches and then call the auto focus function. From the auto focus function, the user will either manually focus the image using the Z-Stage or allow the program to automatically focus the image. Once all initialization is finished, the program will begin stage movement and picture taking.
Design Validation Plan
Major: Computer Science
Major: Computer Science