LED Animation

The Raspberry Pi and LED display used for this project.

LED Animation is a University of Idaho Fall 2013 Capstone Design project for the Collage of Engineering. The goal for this project is to stream a YouTube video to a 32x32 LED panel using a Raspberry Pi.

Sponsor

Dr. Robert Rinker:

University of Idaho Associate Professor in the Collage of Engineering (Department of Computer Science).

Problem Definition

Background

This project builds off of the framework created last semester by team RPLD (Raspberry Pi LED Display). Team RPLD developed display drivers for the Pi's serial port, and manufactured a custom 16-to-26 pin serial adapter needed for connecting the LED panel to the Raspberry Pi.

Problem Statement

Our main objective is to develop software for the Raspberry Pi that will:

Download a YouTube video
Decode the video stream frame by frame
Convert each frame to a representative 32x32 matrix, and
Display frames on a 32x32 LED panel with a reasonable frame rate

Target Specifications and Dependencies

Raspberry PI

Debian Operating System
RM1176JZF-S 700 MHz processor
512MB RAM

OpenCV

CMake 2.6 or higher.
GTK+2.x or higher, including headers (libgtk2.0-dev).
pkgconfig;
Python 2.6 or later and Numpy 1.5 or later with developer packages (python-dev, python-numpy).
ffmpeg or libav development packages: libavcodec-dev, libavformat-dev, libswscale-dev.

libcurl

libcurl4-gnutls-dev binary package
v. 7.9.6 or later
curl 7.14.0 or later

Qt

v. 4.5.2 or higher
qmake 4.5 or higher

LED Display

32x32 LED panel, this project's was acquired from Adafruit (adafruit.com).
Custom 16-to-26 pin adapter.
Image data in 3072x1 8-bit array, per frame, per LED display.

Design

Dataflow Diagram

Note: For a more detailed description of this project's design, refer to the SDD in the documentation section below.

The general design and construction of this project is partitioned into four major interfaces. Each of these interfaces are modular in design, allowing for our end product to provide a much more robust functionality than the specification calls for. These four interfaces are as follow:

User Interface
YouTube Interface
Video Stream Interface
LED Panel Interface

User Interface

The User Interface is the hub joining and streamlining the other interfaces in an appropriate manor. It offers the user several different options, such as streaming a video on the LED display, downloading a remote video to the local machine, buffering raw video data to provide greater throughput, and more. Each of these options utilizes the other interfaces, or modules, in order to achieve the desired result. The general architecture of module interaction can be seen in the module design picture on the left. The functionality of these three modules is described in more detailed below.

General module architecture

YouTube Interface

The YouTube Interface encapsulates the first module, Module 1, of the product. It is responsible for retrieving a proper YouTube video URL that will locate the remote video file hosted by YouTube. These URLs are different from the one that is used in a browser to play a YouTube video on their sight, these are URLs that access the compressed video file stored on the YouTube server. Due to YouTube's copyright protection policies, any YouTube video that is has a copyright cannot be retrieved from YouTube. However, when a video URL is successfully retrieved it is passed up to the User Interface to be utilized as desired.

If a local video, or a remote video that is not from YouTube, is requested by the user to be used, then this interface will not be utilized. This interface is strictly for retrieving the video location of a YouTube video, as the specifications require that the product can stream a video from YouTube.

Video Stream Interface

The Video Stream Interface encapsulates Module 2 of this project. It is responsible for downloading, buffering, and processing video data from the desired location. The end result is a stream of bits that correspond to a single video frame that can be displayed on the 32x32 LED panel. In order to achieve this, the buffered video data has to be decoded and extracted one video frame at a time, re-sized to a 32x32 resolution, and converted to a bit stream that represents the pixel map of the video frame. All common video files can be properly processed in this module, including .mp4, .avi, .mpeg, .flv, etc.

Due to the relatively high overhead of processing the video data, direct streaming on the Raspberry Pi yields very low frame rates, anywhere from 1 to 5 frames per second depending on video quality and resolution. To circumvent this issue, the raw frame data can be buffered into a local object file. This will remove the video processing overhead by pre-processing the video frames, so all that needs to be done is to stream the frames from the object file, one at a time.

LED Panel Interface

Pulse Width Modulation: A modulation technique used for estimating a LED color's brightness

The LED Panel Interface encapsulates Module 3 of this project. It is responsible for displaying the processed video data one frame at a time, at a reasonable frame rate. This is done by pushing the raw frame data to the LED panel, through the serial port, and redrawing the frame at a rate of about 30 Hz (about 30 times per second). This allows each frame to be seen with little to no flickering on the screen. To achieve a convincing depth of color, darker colors are displayed less frequently, using a technique called pulse width modulation. A pictorial description of how this works can be seen in the image to the right.

As each individual frame is being displayed on the LED panel, the LED Panel Interface is preparing to load in the next frame. Due to the Raspberry Pi's relatively low CPU speed, and the fact that part of this module's job is to maintain the display on the LED panel, proper timing to achieve a consistent frame rate cannot be done effectively. Timing interrupts required to achieve consistent frame rates introduce latency issues that increase the amount of flickering seen on the LED display. So videos that are displayed will vary speed depending on their required frames per second, since each frame will be displayed for the same amount of time (i.e. lower frame rate video means faster video displayed).

While the flickering on the screen can be minimized and improved upon by removing timing interrupts, there will always be some flickering do to operating system interrupts. This flicker cannot be resolved without porting the LED display drivers to a dedicated device that has the soul job of displaying the raw frame data. However this would be the task of a future design team, and is outside the scope of this project.

Team: Automaten

Grant Boomer (left) Everett Bloch (right)

Everett Bloch:

Everett is a Senior majoring in Computer Science, with fields of interest in computer networking / data communications, computer graphics, and artificial intelligence.

Grant Boomer:

Grant is a Senior majoring in Computer Science, with a background in art and graphic design, and interests in software management.