ICRS: Details and Goals


Infrastructure and construction robots are a group of modular robotic agents that can work cooperatively to automate various tasks such as construction, inspection, and repair. Each robot is capable of multiple functions, as capacities can be modified through the installation of different attachments on the unit. Each base unit is designed to be low cost to minimize the time and monetary investment and protect against significant losses due to the failure of a single robot. A group of robots can be teleoperated to increase the efficiency of skilled operators or can be given basic directives to automate simple or repetitive tasks.

Problem A: Infrastructure

  • U.S. infrastructure is currently rated as subpar and is continuously degrading (Graded D+ by ASCE)
  • Part of the issue lies in the extreme costs of repairs necessary across the nation (estimated $4.59 billion by 2025 to fix)
  • This massive increase in spending would cause a drag on the economy ($3.9 trillion GDP loss by 2025)
  • Too many problems, too expensive, takes too long

Problem B: Disaster Relief Management

  • Disasters are difficult to recover from and each disaster has unique requirements and problems that need to be addressed
  • Disasters are inherently unpredictable and so disaster preparation would require addressing all possible problems before it occurs, a prospect that can be prohibitively expensive
  • It’s also expensive to be reactionary and ship required relief materials for every disaster only after the disaster has occurred
  • Loss of life is worse immediately following the disaster before the relief supplies can make it to their destination
  • Massive, repeated shipments can be problematic due to bureaucratic and logistical delays

Problem C: Construction and Landscaping

  • Building homes and structures is expensive and time consuming
  • Machines for construction are expensive and have high specialization sometimes requiring many different machines for a single project
  • Cheaper, mass produced materials result in low levels of customization and customer satisfaction also requiring expensive shipment to the destination
  • Heavy earth moving machinery is often needed for landscaping projects which can be expensive or difficult to acquire
  • Using heavy machinery can be difficult and requires additional skills; alternatively hiring professionals only makes the work more expensive

Problem D: Non-Terrestrial Construction

  • Sending up satellites is limiting in both space and weight
  • After the heavy costs associated with initial construction, there are currently no options for repairs in the case of damage
  • Space junk is a problem without a good method of deorbiting debris
  • Expensive and restrictive to need to send all materials to construct a habitat on another planet
  • Materials to build habitats exist on planets but no machines currently capable of utilizing them or constructing habitats


  • Teleoperated robots!
  • Remote operated requiring skilled workers or semi-autonomous task completion
  • Perform checking and eventually construction that’s more cost effective
  • Makes workers more efficient and amplifies their capabilities
  • Increase in safety with cheap, expendable robots performing dangerous work
  • Flexible function using modular attachments means each robot is capable of many different actions
  • Small size means robots can work concurrently in the same location, reducing multiple phases of a project into a single, incremental step
  • Swarm methodology for a group of robots means tasks can be accomplished much quicker
  • A combination of a large swarm and the modularity means specific tasks can be dynamically allocated due to shifting requirements


Tier 1

  1. Robot with modular attachments that enable it to serve various functions
  2. Cheap base model that can be produced in quantity and has basic motion, communication, and sensor capabilities
  3. Assignable roles so that broken robots can easily be replaced by spare units or units can be re-assigned based on need
  4. Teleoperation with visual data so that users can get direct feedback about the status of the project’s target
  5. Limited physical intervention required by users
  6. Ability to direct movement and basics tasks to be performed by the robots

Tier 2

  1. Modular attachments can be swapped autonomously
  2. Dynamic role allocation so that the robots automatically determine the best distribution to get a task done efficiently
  3. Coordinated motion so the robots can be given simple direction to accomplish group behavior

Tier 3

  1. Machine learning can automatically flag problematic inspection data
  2. Augmented reality data so that the users can see the project target and the current progress side by side
  3. Advanced autonomy with the robots having the capabilities to perform a great deal of tasks with little user control required



  • Camera for teleoperation and visual data collection
  • GPS
  • IMU
  • Attachment identification


  • Central communication node to route mesh network packets
  • Master node compiles debug information and swarm state
  • Master establishes swarm requirements and passes them on to individual agents
  • Handled over WiFi with the master node providing the central access point

User Interface

  • Communication through a web interface hosted on the central node
  • Accessible by connecting to the swarm network
  • High level control abilities to direct general motion of the group and assign tasks
  • Lower level control also available for finer motion control
  • Individual robots are selectable so that debug information, robot state, and individual command interfaces are available

Job Allocation

  • Jobs are dynamically allocated and passed on to the individual robots via communication with the master node
  • Robots automatically connect to the attachments required to perform their assigned tasks

Potential Attachments

  • Gripper: Movement of structural material
  • Dumper: Transportation and removal of debris, earth, sand, etc.
  • Arm: Fine manipulation of objects
  • Screwdriver
  • Drill


1. First Functional Prototype

  • Single robotic unit
  • Direct control basic interface (command line, GPS coordinates, etc.)
  • Basic motion and movement commands with low accuracy

2. Attachment Prototypes

  • Several basic, modular attachments for the prototype robot so it can perform various functions (e.g. gripper, arm, digger, screwdriver, drill, material transportation)
  • Automatic identification and control of the different attachments

3. Prototype Swarm

  • Multiple units controllable via a single interface
  • Different attachments to show multi-use cooperation
  • Direct control basic interface for individual units as well as group movements
  • Central communication hub to route communication between robots and provide single access point that distributes commands to individual swarm members

4. Survey and Analysis Demo

  • Demonstration of basic survey capabilities using direct control interface and teleoperated swarm
  • Robots coordinate with each other to map out a structure and provide detailed pictures
  • Optional additional sensors for measuring other useful data (e.g. radiation, temperature, vibration)

5. Repair Demo

  • Demonstration of basic repair capabilities using direct control interface and teleoperated swarm
  • Robots are capable of moving repair materials into place and performing the repairs without physical operator intervention
  • Robots can cooperate as a cohesive group to transfer repair materials and/or remove broken material

6. Construction Demo

  • Demonstration of basic construction capabilities using direct control interface and teleoperated swarm
  • Robots are capable of assembling an entirely new structure without physical operator intervention
  • Robots are capable of working together to prepare construction area and build a structure

Infrastructure and Construction Robot Swarm

New project alert! I was throwing around some multidisciplinary project ideas with two Mechanical Engineering friends and we talked about having some sort of robotic swarm for construction, disaster relief, infrastructure inspection, etc. Right now it’s not really a fully formed idea but I did submit it to the Hackaday Prize contest anyway since they’re currently running an “idea” seed funding phase. Here’s the project link. I think this idea has a lot of potential and I’m hoping this can grow into something that’s useful and beneficial to society. More details to follow!

On Recent Inactivity

It’s been a while since I’ve posted here, not because I haven’t been working on projects but more because I’ve spent the better part of a year trying to add some diversity to my hobbies. Last April I decided to finally lose the weight I’d gained since high school so I did a little self-improvement and lost forty pounds while exploring some other interests. Aside from the weight loss I took up running and biking, started learning to play violin, wrote a novel for NaNoWriMo, and am currently experimenting with gardening and cooking. Luckily many of these activities provide opportunities for electronics hacks and designs so I anticipate a few smaller hacks centered around these new hobbies in addition to the large, ongoing projects I have going.

Since I last posted, I’ve redesigned the OpenSCAD to make the robot much smaller. I haven’t documented any of this and I’m pretty sure it’s a dead end, but I’ll try and write up the changes I’ve made and my results soon. I haven’t really done anything else on the PiGRRL Switch. Everything I need is sitting on my desk, I just need to buckle down and put it all together. I plan on starting with the console and verifying it works with a regular USB controller and then finishing the custom controllers.

PiGRRL Switch: Test Controller in Action!

2017-03-18 12.55.34

So I didn’t get a chance to take a video of the controller in action, but I do have a picture of me playing the original Zelda using my soldered up controller prototype. The Raspberry Pi and LCD is hooked up to one battery bank and the controller is hooked up to a separate one. Going through this test I noticed a few things that can be improved.

For starters, I had a hellish time getting enough power to both the Raspberry Pi and LCD. For whatever reason the power supply didn’t seem to be able to give enough juice to keep the Raspberry Pi from rebooting. The display has two micro USB ports on it that are used to power the screen and provide touchscreen feedback to the Pi. However since I currently don’t need to use the touchscreen I just powered the screen directly from the battery bank. To circumvent these power issues in the future I’ll need to see if I can get the Pi to provide more current from the USB ports. If that doesn’t work I can use one of the micro USB ports for touchscreen feedback and cut the red power wire to prevent the screen from drawing power from that port and use the second USB port to power the screen directly from the battery bank.

Another issue I noticed with my controller is the lack of menu buttons. The RetroPie interface requires using Start and Select buttons for navigation, so I had to map those to an attached keyboard since I only have four action buttons and four directional buttons on my prototype. Additionally I’ll also want shoulder buttons since the later consoles all have at least two and the PS1 has four.

The next step for the controller interface is designing the PCB!

PiGRRL Switch: Controller Driver

The next step in getting the controller to work as a RetroPie controller is writing a driver so the console interprets the serial data coming in from the Bluetooth module the same way it would interpret a USB controller.

I got this idea from petRockBlog who mapped their custom, two-player joystick to two separate virtual controllers in RetroPie.  Their code is available on GitHub.  After a quick review, I saw that their code is using a C user input emulation library called uinput.  In the interest of simplicity I found a Python uinput library and decided to use that to emulate my controller driver.

Below is a full snippet of the code from the GitHub repository followed by an explanation of how it works:

This is the definition of the Controller class.  A static variable called NumButtons is used to show the maximum number of buttons a Controller has.  A Controller instance is initialized with a path to the Bluetooth serial device which it then connects to.  The initialization also creates a uinput Device that it will use to simulate a virtual controller for providing input to the console.

An array of keys is created to represent the possible input keys that will be passed into the system.  Right now I’m just using the letters of the alphabet.  RetroPie offers full controller customization so the actual keys used don’t really matter.  I still plan on changing the controller driver to emulate a joystick with analog controls, however, so this method will need to be updated in the future.

This array will store the device names for all of the Bluetooth serial consoles.  At some point I’d like to autodetect all of the serial consoles from the controllers but for now a hardcoded array will work.

The program creates a controller for each serial console in the ControllerNames array and stores them in the Controllers array.  It also resets the serial input buffer so old input data doesn’t bog down the driver and it can start processing the most current data.

My driver operates on button transitions (unpressed->pressed and pressed->unpressed) rather than operating on the button state.  This is to be consistent with keyboards which work in the same way.  Because of this transition-based processing, the previous input packet needs to be stored so the previous packet can be compared to the current packet and differences between the two can be detected.  Here the previous packet is instantiated as a dictionary and is initialized to a known, totally unpressed state.

This is just an infinite loop which iterates over the list of controllers and processes each of them.

A line is read from the serial port and split at the colon.  The left side will be the key character which reflects the type of data and the right side will be the data itself.

This if statement detects which key is seen and executes the appropriate code.

This is the code for the key ‘K’ which is the only type of data I currently have.  It iterates over each character in the eight bytes of data and compares it against the data from the previous packet.  If the value went to ‘1’ from ‘0’ it emits an event to signal that the key was pressed and if the value went to ‘0’ from ‘1’ it emits an event to signal the opposite.

PiGRRL Switch: Controller Test Firmware

As a follow-up to my last post, I wanted to go over the Arduino firmware I used for my primary controller test.  Essentially what it does is read the states of multiple buttons and print a sequence of 1’s or 0’s (1 for unpressed and 0 for pressed) to the Bluetooth module.

The whole program is embedded above and I’ll go through each section individually and explain what it does below.

These enums serve the purpose of defining human-readable names for the pins that are hooked up to the buttons.  That way their usage in the code is immediately obvious upon reading.

Nasty literals!

This isn’t strictly necessary due to the simplicity of this program, but the general best practice is to not use integer literals in code.

This is the pin mode setup in the Arduino setup() function.  The buttons are hooked up directly to ground so that when the button is pressed, the pin sees 0V.  When not pressed the pin is floating so here I’m enabling the Arduino’s pullup resistors so that when the button isn’t pressed the pin sees 5V.

Here the state of the buttons is read and printed to the Bluetooth module over UART.  Because of the way pullups are handled, ‘1’ corresponds to unpressed and ‘0’ corresponds to pressed.  The printing of the button states is preceded by a “K:” to act as a key-value pair.  This means that when the controller driver on the Raspberry Pi sees a “K” it knows it’s about to get a list of digital buttons.  I’m planning on using this in the future to enable more complex data than just a few buttons.  For example, I could use the keys “JX” and “JY” to denote the two axis values of an analog joystick.  The last serial print is a Serial.println() function call to print a newline character.  This is an easy way to segment the data so the controller driver can differentiate between multiple packets.

Lastly is the delay that sets the update rate of the program.  With a delay of 10 milliseconds at the end of the loop, new button states will be sent to the driver at an update rate of roughly 100Hz.  Right now this rate was arbitrarily decided, but in the future I’ll have to tweak it to balance several factors.

First off is the bandwidth of the Bluetooth connection.  As far as I can tell, the HC-05 module uses Bluetooth 2.0 which has a data throughput of of 2.1 Mbit/s or 275251.2 bytes per second.  Divide this by the update rate and you get the total amount of data that can be sent in one cycle of the program.  This also assumes that the connection is perfect with no lost packets.  Bluetooth’s serial port profile uses re-transmission to verify that packets sent are received.  This means that wireless noise can cause the connection to slow down while the Bluetooth module is attempting to resend a bad packet.  Additionally, if I go back to the BLE module and am able to get it working, BLE only has a throughput of 35389.44 bytes per second, or 13% of Bluetooth 2.0.  So while I’d be saving power I’d be restricting the update rate and amount of data I can send from the controller.

Another thing to consider is the update rate of the controller driver running on the Raspberry Pi.  Serial ports have buffers to store incoming data.  This means that if the driver is updating slower than the controller, the driver won’t be able to process all of the data coming in and the buffer will overflow.  This problem will exhibit itself as lag in the controls.  The driver needs to be updating at the same rate, or faster, than the controller.

The last consideration is debouncing.  This is a topic that has been extensively covered elsewhere, but suffice it to say that if the controller updates too fast it will accidentally send phantom bounces in the switch contacts to the driver and cause erroneous button presses.

Once everything has come together and is working to a degree I’m happy with, I’ll come back to these small issues and tweak everything until it works perfectly.  Until then, my next step is writing the controller driver!  As always, my code is available on my GitHub in the PiGRRL_Switch repository.

PiGRRL Switch: Creating the Controllers

With the screen chosen and working, the next step for creating the PiGRRL Switch was prototyping the controllers.  Initially, I wanted something cheap and easy like a breakout board that acted as a Bluetooth HID joystick.  I was immediately drawn to Adafruit’s EZ-Key which acts as a Bluetooth keyboard.  At the time I’m writing this, however, it’s out of stock and seems to have been for a while.  Additionally, because it acts as a Bluetooth keyboard and not a joystick, it rules out any possibility of adding analog controls in the future.

Another alternative to a Bluetooth HID breakout would be taking apart a cheap Bluetooth joystick and putting it in a 3D printed casing.  However, I decided this would greatly reduce the design flexibility of the controllers and might make it difficult to reconfigure the controllers on the fly (i.e. using two JoyCons as one controller vs. using them as two separate controllers).

So with those two options off the table I decided instead to use a Bluetooth serial bridge.  The HM-10 BLE and HC-05 Bluetooth 2.0 modules are both cheap and plentiful and provide a good solution at the cost of some extra work.  These modules can be hooked up to the UART of an Arduino and paired via Bluetooth.  Once connected, it acts as a virtual serial port in Linux, allowing the serial data to be read just as if the Arduino was connected via USB or FTDI.  The only exception to this is that it doesn’t support firmware loading wirelessly.

2017-03-13 22.18.32

The next step was setting up the initial design on a breadboard.  Above is an Arduino Pro Mini, four pushbuttons wired to the digital pins, and the HM-10 BLE module.  I decided to use the HM-10 because of the lower power requirements (BLE being an initialism for Bluetooth Low Energy).  The code for the Arduino reads the values from the digital pins and prints out eight characters to signify which buttons are pressed (‘1’ for unpressed and ‘0’ for pressed).  Right now I’m using a byte for each button which is wasteful, so I’ll go back at some point in the future and make the code more efficient so each button is represented by a bit.

2017-03-14 22.50.16

Once everything was wired up and running I had a lot of trouble finding an app that could connect to the HM-10 as a serial terminal.  Apparently the BLE standard has a lot of bells and whistles that make configuration a bit more difficult.  After trying several different apps I eventually found Serial Bluetooth Terminal which can connect to both BLE and regular Bluetooth devices via a serial terminal.  Above is screenshot of my phone connected to the controller with the button status being transmitted.

2017-03-14 20.31.24

2017-03-14 20.31.37

With my proof of concept working, I soldered everything onto a proto-board, this time with eight buttons to serve as a D-pad and four action buttons.

With that complete the next step was connecting to the Raspberry Pi over a serial terminal.  Unfortunately, this was much more difficult than I expected.  I could pair and connect to the HM-10, but couldn’t find a way to mount it as a serial terminal.

2017-03-15 21.01.17

Rather than continue further down the rabbit hole, I decided to drop the BLE module for now and switch to the HC-05 modules I bought as a backup.  Those have been around for years and have been used extensively with Arduino and Raspberry Pi.  Once that module was paired and connected, mounting it as a serial terminal was as simple as using the following commands to connect and then print out the values read from the module:

sudo rfcomm bind /dev/rfcomm0 <MAC Address>
sudo cat /dev/rfcomm0

2017-03-17 19.20.25

2017-03-17 22.01.22

Lastly I connected the controller, screen, and Raspberry Pi to battery packs and verified everything still worked as suspected.  Success!  The next step is writing a program for Linux that reads the button data coming off the serial port and uses it to emulate a controller for the console.