Currently, there are a lot of IoT tools and solutions, so why should we create something new? The majority of the existing frameworks focus on programming and controlling separate devices. Yet, we decided to apply our experience from delivering IoT development and integration services to develop a solution that would allow programming complex systems consisting of multiple devices that interact with each other.
Consequently, the framework had to provide the following set of features:
We saw that the majority of requirements could be implemented with Akka.
To understand the problems we wanted to solve with Pragmukko, let’s look at the hypothetical business case that can become a commonplace occurrence in the very near future. Assume we have several drones controlled by Raspberry Pi, a regular PC as a control station and a smart fridge with ice cream controlled by Intel Edison. All the devices are physically located in one room and are connected to one and the same network via Wi-Fi. A drone has to pick up and deliver some ice cream from the fridge when a user orders it to do so. Therefore, the framework’s main goal is to enable easy programming of this function.
Since the project team decided to use Akka, our first idea was to launch an Akka node on every component of our system: drones, control station and fridge, and then build the logic of every component as an actor in the node. This way, our system would look like a regular Akka cluster with some nodes capable of flying and other being fridges with ice cream. Akka enables the drones, the fridge and the control station to communicate with each other, control the state of the cluster, receive the list of the cluster members, etc. Under certain conditions, our system is decentralized and resistant to the failure of individual components. Moreover, we can easily add new components to the cluster - they will immediately get the information about other members and join the operation. This means that we can effortlessly install one more fridge, or, if any drone takes too much ice cream and falls down, we can employ additional drones.
In general, we had to write a hardware interaction layer and bring the whole IoT and drones thing together. That’s what we did, along with some other things.
To bring the system to life, the team took the following steps. First of all, we created a mechanism to automatically group the nodes in a cluster. Although Akka has this function, we ignored it because it required preliminary configuration of seed nodes. We also needed to know the network addresses of all cluster members, and these were the kind of complications we wanted to avoid. Instead, the project team developed the node grouping mechanism based on UDP broadcasting.
Then, we programmed the device nodes to interact with hardware by transforming the incoming Akka messages into commands. We named such nodes Embedded Nodes. Schematically, they can be presented in the following way:
Broadcast Actor is an actor that sends UDP-broadcast messages for the node to be identified and added to the cluster.
Embedded Actor is an actor that processes the messages from other nodes of the cluster and interacts with hardware. This actor can be extended to implement user logic.
Hardware Layer is the layer for hardware interaction. It is an actor that receives binary commands and writes them to the serial port.
Also, we introduced one more type of nodes, Manage Nodes, to work on a server, command the devices and provide interaction with other systems. Pragmukko allows extending the functions of a Manage Node with plugins that could also integrate with other systems. The plugins we have can integrate with MongoDB and HDFS as well as implement the REST interface based on Akka-http. Below is the illustration of this node:
UDP-Responder is an actor that adds new nodes to the cluster.
Unfortunately, the project team was unable to implement the business case described above since there’s no fridge with ice cream in our office. Therefore, we tested a simplified case. We had several quadcopters randomly flying over a certain territory. They were controlled by the control station that kept them inside the perimeter.
Here’s the example of the code that works on a quadcopter:
object EmbeddedMain extends App with DroneCommands { // Initialize initial speed of the drone var (vx, vy) = (Random.nextFloat(), Random.nextFloat()) // Special helper to create Embedded Node EmbeddedPragma { ctx => { // The Start message comes right after the node joins the cluster case Start => // Here, we notify the hardware that we want to receive telemetry ctx.subscribeHardwareEvents() // Notifying the hardware layer about the desired initial position and speed of the drone // We tell the drone to rise 10 meters up and start moving in directions vx and vy with the specified // speed ctx.self ! moveTo(0,0,-10) ctx.self ! direction(vx, vy, 0) // Here, we process the autopilot data // The data is received in the binary format // The tools for processing are available in the DroneCommands trait case TelemetryBatch(batch) => // TelemetryBatch - message contains drone // Checking if the received data contains the information about the drone’s position in space val position = batch.collect { case DronePositionLocal(p) => p }.lastOption // If the position is located, we send the information about it to all listeners ctx.listeners foreach ( _ ! position ) // Processing commands from the control station case "turn x" => vx = -vx ctx.self ! direction(vx, vy, 0) case "turn y" => vy = -vy ctx.self ! direction(vx, vy, 0) } } }
And the code of the control station:
// plug-in listing class DroneControlExt extends GCExtentions with DroneCommands { override def process(manager: ActorRef): Receive = { // Processing messages from a drone // If the drone crosses the 20x20 m perimeter // we send the command to turn along the required axis case DronePositionLocal(position) => if (position.x > 10 || position.x < -10) sender() ! "turn x" if (position.y > 10 || position.y < -10) sender() ! "turn y" } }
Basically, that’s all with the code. We compiled it and launched the Embedded Node on the drone. As for the control station, it can be launched anywhere: for instance on a laptop, provided that it is located in the same network with drones.
So, what’s happening behind the scenes? A drone, when launched, starts identifying itself with broadcast messages. As soon as a control station gets the message, it adds the drone to a cluster. Finding itself in the cluster, the drone adds the control station to the context.listeners list.
The control station and the drone’s Embedded Node implement the trait DroneCommands. This trait contains utilities for Pixhawk MAVLink protocol. For example, direction(dx:Float, dy:Float, dz:Float) method forms a binary command that sets the drone speed along certain axes. All incoming binary commands are automatically transmitted to the hardware interaction layer that communicates with the drone’s autopilot. Since we implemented the hardware interaction layer as a simple interface to the serial port, other autopilots would work if you change the implementation of DroneCommands.
Here are a few more words about the deployment. As the number of drones and IoT devices we experimented with grew bigger, the time to deploy them also increased. We had to go through the same routine for each device: install Java, upload the app, register it as daemon, and reboot the device. Only after all these steps could we deploy the Manage Node with all the components it interacted with. The process was so complicated that the team immediately wanted to automate it. On the advice of our partners from Cisco, we used Mantl.io. A platform for rapidly deploying globally distributed services, Mantl provides all the necessary components to start fast and improve often. As Cisco's CTO Zorawar Biri Singh outlines in his exclusive interview to InfoWorld, in the foreseeable future, this lightweight, high-level container PaaS can be used as an orchestrator solution for building tightly coupled systems.
With just a few Ansible scripts, we were capable of deploying the necessary infrastructure, regardless of the number of devices in it. The team really loved this method and, as a result, we highly recommend it.
Traditionally, Akka is used to build distributed server solutions. So, we faced a question - if Akka-based products are effective enough on devices with limited computing capabilities. We received the answer through an experiment. In our case, we used the Raspberry Pi 2 Model B as the drone’s onboard computer. The device had the following characteristics:
The characteristics might look quite impressive, but alongside our process, this mini computer also has to run two important processes with nearly real-time priorities. To prevent our software from inhibiting the neighbouring processes, we limited the CPU usage for it to a single core. Additionally, we set one more limitation: the use of memory by the Java process was restricted by the -Xmx32m parameter.
Before the test flight, the team decided to check how our Embedded Node would work on Raspberry Pi with such limitations. Once launched, the process immediately utilised 25% CPU (100% according to top data), or just one core out of four, and in a couple of seconds the CPU use dropped down to an acceptable 10-15%. The memory showed even better results: the process did not utilize all of the allocated 32 Mb, and the garbage collector was starting after a time-out.
During the flight, the performance parameters declined. CPU use reached 20% and the garbage collector worked intensely, because the autopilot generated a lot of telemetry data. If necessary, there is some possibility for optimization by limiting the telemetry stream only to autopilot configuration. We did not introduce such limitations because the parameters were within the normal range and the software did its job well - the control station directed the drone and kept it within the designated perimeter.
The project also revealed a number of challenges we still have to overcome. In particular, the higher the drone’s speed, the harder it is to control it. This is a complex problem and our framework cannot take all the blame for it. The control scheme that we used in the experiment is hardly suitable for real tasks. We can feed the drone the destination coordinates instead of the velocity vectors. Such an approach will add accuracy to the drone positioning, offsetting the dependance on the data link between the drone and the control station.
The second problem is the insufficient speed of our software. Akka is very slow to start. If the actor system is launched, joining the cluster takes from 5 to 20 seconds. Primarily, the speed depends on the quality of the network connection. We still work to improve this aspect.
Another interesting feature of Pragmukko is the possibility to emulate hardware cluster members. This can come in handy while testing the business logic that is implemented on the devices. To enable hardware emulation, we extended the framework with the possibility of switching the Hardware Interaction Layer to Mock. For the tests, we saved real telemetry in a file and then ran its instances on the Embedded Node. I can already see how we will run integration tests for our drones on Jenkins. Then, the expression “the tests have crashed” will not be nearly as dramatic as it is now. Follow the link to learn more about Pragmukko, its features and the project roadmap.
Pragmukko makes the system easy to program, configure and test. In spite of certain complications, we are satisfied with the result. We also believe that our framework has the potential for widespread use. If you have any thoughts on the topic or IoT-related questions you would like to discuss with our experts, please get in touch with us.
No drone was harmed in the making of this experiment.
The breadth of knowledge and understanding that ELEKS has within its walls allows us to leverage that expertise to make superior deliverables for our customers. When you work with ELEKS, you are working with the top 1% of the aptitude and engineering excellence of the whole country.
Right from the start, we really liked ELEKS’ commitment and engagement. They came to us with their best people to try to understand our context, our business idea, and developed the first prototype with us. They were very professional and very customer oriented. I think, without ELEKS it probably would not have been possible to have such a successful product in such a short period of time.
ELEKS has been involved in the development of a number of our consumer-facing websites and mobile applications that allow our customers to easily track their shipments, get the information they need as well as stay in touch with us. We’ve appreciated the level of ELEKS’ expertise, responsiveness and attention to details.