Virtual Control System Network

Welcome to the wiki page for the Virtualization of Industrial Control Systems capstone project at the University of Idaho. Here someone can find all the information you need to know about the project, such as the background, specifications, design choices, team information, and document archive.

Background
The goal of this project is to create a portable system for presentations about security in virtual control systems, which can be used as a test bed and demonstration platform for various industrial control system (ICS) devices. Previously this role has been filled in part by a physical system containing a relay, PLC, gateway, RTAC, and a HMI.

Problem Statement
The current physical system has two major drawbacks, flexibility and portability. Because the system is hard-wired with a fixed number of devices it cannot be reconfigured to run different types of simulations and presentations. This also means that as new technology comes out is difficult to implement it into the design. Although the previous system was designed with portability in mind, there is a limit to how portable a system can be while implementing physical devices. The bulk of the device means that it cannot be moved easily or brought on board a plane as a carry-on. This is an issue, as it limits the audiences it can be presented to due to the logistics of transport.

Problem Solution
By creating a virtual parallel of the previous model, we can have its full functionality while adding on key features. Because it will no longer be hard-wired together the new network will be reconfigurable with ease, allowing much needed flexibility. This also means that adding new devices and duplication of old devices will be quick and easy, allowing for large-scale complex networks to be created and tested. Because the entire system is virtual, it can be hosted on a remote server and accessed anywhere with an internet connection, making it as portable as a laptop. Additionally, this makes it easy to expand upon, even allowing for a hybrid network of physical and digital devices to be created for specific applications.



Project Goals
Below are listed the project goals of things that will be included in our virtual system

- Relays - Gateways - Firewall - RTAC or similar device - Backend data emulator - Human Machine Interface - Communication channels between devices using industry standard protocols - Documentation to allow the project to be taken up by other teams in the future

Overall Structure
The infrastructure for this project will be a Linux server hosted externally (during development this will be on an Amazon Web Services hosted server, then during use it will be hosted on a server owned by PNNL for use and further development). We will use Docker containers for each individual component of the industrial control system. These component Docker containers will communication over a Docker Virtual Network using the modbus protocol (a common protocol used in industrial control systems). The infrastructure will be created such that in the future this can be extended to include other common communication protocols such as DNP3. To enable rapid deployment of docker containers we will be using Docker Compose, and Docker Swarm will be used to network containers spread across multiple hosts (to ensure created environments can be scaled as much as possible). Access into the network will be provided by a VPN container to ensure that the environment is realistically separated from other networks and to ensure security. Data will be fed into the virtual relays through a plugin system, to allow flexibility in what programs and methods can be used to generate or simulate data.



Relay
The relay will be written from scratch in C++. the basic structure of it includes three threads; an input thread, a communication thread, and a control thread. The input thread waits for a signal that there is input inbound. Once this signal is received, it wakes up the control thread and the main thread handles the data input from the data plugin. The control thread is the core of the program, and implements user-defined logic on incoming data. Based on this data, it will update the control state and push both the input data and control state to the I/O thread. The I/O thread has two main jobs, outbound and inbound communication. Data and the control state are pushed to the communication thread by the control thread, the communication thread must then package these up as modbus packets and send them to a control device. For inbound communication, the input thread is constantly listening for a control order from the control devices on the network, when one is received it is sent to the control thread so it can be acted upon.

Gateway
The gateway will be handled by Docker. Included in Docker's base features are simple networking devices, this is the easiest way to set up the network without creating a convoluted setup process.

Firewall
The firewall will exist on the edge of the network to ensure no improper communication enters or exists the network.

Back-end Input Emulator
This will feed data into the relays, and will be implemented as a plug-in. The data this feeds will be based off of real world data, obfuscated to protect privacy. There will be options to play a simple loop of data specified by the user, or to randomized the data even more, for random input to be used in network testing.



Human Machine Interface
The Human Machine Interface (HMI) will be created with a simple base developed using python with flask and pymodbus for communications. The HMI will poll devices (such as relays) for data to display. It will have a simple file system to enable caching of information such as slave device IP addresses, and to enable other convenience features. The HMI can also send commands to devices on the network. For example, it can send an interrupt request to a virtual relay to manually flip a breaker. This functionality can be added for each new type of device on the network, so the HMI can be configured to seamlessly interact with any device added to the simulation. To add a new type of device to the HMI all that you need to do is expand the JSON file that it reads device types from.

Virtual Private Network
The Virtual Private Network (VPN) will be a docker container that allows the network to be accessed in a safe and secure way. You will be able to access the network either remotely or locally through the VPN container, with security set up as a password or with a public/private key combination. As mentioned earlier, this will allow for realistic modularity and improved security of the virtual network.

Configuration File Generating Script
In order to generate the configuration files for devices such as the PLC and HMI, we will have a script on the network. The script will run immediately after Docker Compose sets up the network, discovering the devices it creates to compile all the data each device will need in their configuration files. This will be done through Docker's Python API, by taking advantage of the services and tasks Docker Compose sets up to keep track of individual containers (this will allow us to set up multiple different containers with the same configuration settings, reducing some of the tedium in setting up a large network of similar devices). Next, the script will output this data in the format that each different device is able to read. These files will then be loaded through Docker Volumes onto each individual container that requires them, and an update will be forced in each of these containers. The reason this is needed is there are some things that are dynamically set when launching a network with Docker Compose, such as IP addresses, this reduces the need to have someone manually set all these such configurations by hand. Additionally, it allows for different configuration "profiles" to be made for different types of devices and is able to auto detect them, so their configuration doesn't need to be copied multiple times through each configuration file.

Contact
Ben Merritt   (merr4001@vandals.uidaho.edu)

Gabe Gibler (gibl3465@vandals.uidaho.edu)

Joey Chereck (cher3222@vandals.uidaho.edu)

Jessica Smith (jessica.smith@pnnl.gov)

Documents

 * [[Media:2017_VirtualControlSystemNetwork_MeetingMinutes.pdf|vGrid meeting minutes]]
 * [[Media:2017_VirtualControlSystemNetwork_ProjectLearningSummary.pdf|Project learning summary]]
 * [[Media:2017_VirtualControlSystemNetwork_AnaylsisofAlternatives.pdf|Analysis of alternatives]]