Decentralized Control of a Hexapod Robot Using a Wireless Time Synchronized Network

08/23/2018 ∙ by James Fang, et al. ∙ 0

Robots and control systems rely upon precise timing of sensors and actuators in order to operate intelligently. We present a functioning hexapod robot that walks with a dual tripod gait; each tripod is actuated using its own local controller running on a separate wireless node. We compare and report the results of operating the robot using two different decentralized control schemes. With the first scheme, each controller relies on its own local clock to generate control signals for the tripod it controls. With the second scheme, each controller relies on a variable that is local to itself but that is necessarily the same across controllers as a by-product of their host nodes being part of a time synchronized IEEE802.15.4e network. The gait synchronization error (time difference between what both controllers believe is the start of the gait period) grows linearly when the controllers use their local clocks, but remains bounded to within 112 microseconds when the controllers use their nodes' time synchronized local variable.



There are no comments yet.


page 1

page 2

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Control systems can be fully centralized, fully decentralized, or a combination of both. In a centralized control system, all of the information about the system as well as the calculations based on that information take place at a single location. In a decentralized system, the information about the system as well as the calculations about that information take place at multiple locations [1].

Decentralized control is desirable for a multitude of reasons. Firstly, by modularizing a robotic system, lengthy control wires running from the central controller to various different parts can be cut down, leaving just the power wires needed to supply current. Wiring is one of the most vulnerable points on any robot and is thus one of the most common sources of faults and breaks [2]. By reducing the wiring required, the robustness of the entire system is improved. Additionally for microrobots, having lots of wires is far too costly in size and weight and can easily make micro-scale systems infeasible [3].

Secondly, decentralized control allows for more flexible and responsive robotic systems [2]. This becomes especially apparent in robot swarms [4] and self-assembling robots [5] where the components of the system may not be physically connected in any way.

One of the biggest challenges to decentralized control is time synchronization of the various distributed components. In this paper, we demonstrate the viability of implementing the low-power wireless protocol stack OpenWSN [6] on OpenMotes [7] for the time synchronization of decentralized robotic movement on Larry (Fig. 1), a hexapod robot that uses two independent but wirelessly connected OpenMotes as controllers for its legs.

Figure 1: Larry the hexapod with two on-bot OpenMotes sitting on his frame patiently awaits commands from the central controller or ”DAGroot” mote (not shown), which is connected via OpenUSB to a computer and controlled by a human driver using a python GUI.

Ii Time Synchronized Protocol

OpenWSN [6] and Contiki [8] provide open-source implementations of a low-power, high-reliability communication stack based on the IEEE 802.15.4e [9] protocol and the IETF 6LoWPAN [10] and 6TiSCH [11] protocols. IEEE802.15.4e added Timeslotted Channel Hopping (TSCH) to the IEEE802.15.4 standard, allowing for ultra low power communication with very high reliability. 6LoWPAN allows IPv6 packets to be sent over lossy and low-power wireless sensor networks, allowing them to interface directly with the Internet, and 6TiSCH provides standard ways of implementing and managing a TSCH schedule. CoAP [12] is a web transfer protocol specifically designed for constrained and low-power networks. Using IEEE 802.15.4e TSCH with centralized feedback control has been demonstrated [13]; however, the work in this paper uses IEEE802.15.4e TSCH for decentralized control by exploiting the extremely tight time-synchronization of nodes. Very small synchronization errors between 3-hop deep nodes in TSCH networks have been demonstrated using Contiki (2 microseconds) [14], OpenWSN (76 microseconds) [15], and Dust Networks’ SmartMesh software and LTC5800 chip (5 microseconds) [16].

Figure 2: The dual tripod gait in action (side view, walking to the right). (i) The right tripod lowers and the left tripod lifts at the hip. (ii) The right tripod propels Larry forward by rotating back at the knee while the left tripod rotates forward at the knee to reset. (iii) The right tripod lifts, and the left tripod lowers at the hip. (iv) The left tripod propels Larry forward by rotating back at the knee while the right tripod rotates forward at the knee to reset. These four actions loop to produce the forward-moving dual tripod gait.
Figure 3: An image panel of Larry walking and being controlled by and using scheme decentralized synchronized control (ASN synchronization). Larry’s approximate velocity is around 2 cm/s.

Iii Larry, an example platform

Iii-a Design and Construction

Larry, being a hexapod, has six legs, each with two points of rotation (i.e. joints), which are referred to as the “hip” and “knee”. These joints are each actuated by a servo motor for a total of twelve servos.

The hip-and-knee design allows for two rotational degrees of movement: the servo actuating the hip translates its rotational motion into linear vertical motion, allowing Larry to lift himself, while the servo actuating the knee translates to linear horizontal movement, allowing Larry to pull himself forwards or backwards. The advantage of this design is simplicity: there are only three moving parts on the leg. Legs also allow Larry to engage in step-like motions, which can be adapted to traverse over obstacles in the human world (i.e. stairs, gaps, etc.).

The hexapod frame and legs are fabricated by laser cutting inch plywood. The parts are assembled using small nails. Six servos are mounted directly on a cm sideboard. Two mirrored sideboards are then attached to a cm top frame. The legs are attached to the servos at the appropriate joints. The wiring is done using a breadboard, which sits on the top frame with two OpenMotes.

Iii-B Gait

Larry walks using a dual tripod gait. As the name suggests, there are two tripods, each one consisting of the front and back leg from one side of the robot and the middle leg from the other side. At any point in time, at least one tripod will be in contact with the ground while the other tripod resets in midair. The gait has four main positions which when executed in a loop results in step-like motion (Fig. 2).

A single tripod goes through the following four events:

  1. The tripod actuates the hip downwards, lifting Larry.

  2. The tripod actuates the knee backwards, propelling Larry forward.

  3. The tripod actuates the hip back upwards.

  4. The tripod actuates the knee forwards to reset for the next iteration.

Larry can be seen walking in Fig. 3. The tripods are offset by exactly 180 degrees, meaning that in our four-step gait, tripod 1 should be exactly two steps ahead of tripod 2 at all times. Since the gait motion is periodic with a period of one second, tripod 1 should be half a second ahead of tripod 2 at all times. This is where time synchronization becomes crucial for proper execution of the robotic gait. If the gait offset of the tripods is out-of-sync, Larry would not be able to walk properly. In the worse case, if tripod 1 and tripod 2 are 180 degrees out-of-sync (i.e. they are performing the same gait actions at the same time), Larry will not be walking at all but instead be squatting in place over and over.

The gait events are controlled using two OpenMote-cc2538 boards or “motes”. Let the two motes be and . controls all six hip servos, and controls all six knee servos. A circuit diagram of the two motes connected to the servos that they each control can be seen in Fig. 4.

Figure 4: Circuit diagram of the servos being controlled by two motes. (top mote) is controlling the hip servos (green control lines), and (bottom mote) is controlling the knee servos (yellow control lines). The servos are powered by an external supply drawing 2A at 5V.

Iv Firmware and Networking Implementation

Hexapod control is implemented with a modified version of the OpenWSN wireless protocol stack firmware, running on OpenMote-cc2538 Rev. A1 boards. OpenWSN designates one mote as a “DAGroot” that can send and receive messages from all motes connected in the network.

The firmware is flashed to , and the DAGroot motes. A human driver can interact using a CoAP application (Fig. 5), usable through either a Python GUI interface or the command line (e.g. the driver can send a CoAP message from the DAGroot to and telling them to start running their gait events). Three separate methods for gait control were explored: centralized control, decentralized open loop control, and decentralized synchronized control.

In centralized control, the DAGroot was responsible for timing the gait events and sending information over the network about the angle of each servo motor (i.e. turning it) to the two child motes, whose only task was actually implementing the pulse width modulation (PWM) signals to drive the motors. However, in this implementation, the DAGroot was constantly transmitting messages to the other motes, typically several times per second, and this caused latency issues since the messages to both motes cannot be sent at exactly the same time and the transmission itself introduces some delay.

To work around this issue, we tried decentralized open loop control. The DAGroot was now only responsible for sending high-level commands to the child motes such as “go forward”, “turn left”, or “stop”, and the child motes used their independent, internal timing schemes to control the gait events driving the servo motors. In practice, this worked well while driving Larry for short amounts of time (on the order of minutes). However, the on-board timers of the motes have a drift of at most 10 ppm, so for longer runs, the two motes will eventually grow out-of-sync. Thus, we need to periodically resynchronize the two child motes in order to mitigate timing drift between the gait events.

We addressed this issue in decentralized synchronized control by synchronizing the motes’ timing schemes using the network’s absolute slot number (ASN), a local variable on each mote that is incremented every 15 milliseconds according to the mote’s local clock. Anytime a message is sent between the child motes and the DAGroot, the child motes will resynchronize their clocks to the DAGroot’s clock - this makes the ASN a reliable timing reference across all motes. As per the default OpenWSN timing requirements, motes must be synchronized within 1 millisecond. Our implementation has the child motes communicating with the DAGroot every 30 seconds in a worst case scenario, allowing the child motes to be synchronized within 120 microseconds (see results below). This implementation allows Larry to walk for any duration of time without squatting (i.e. desynchronizing).

Figure 5: A human driver can operate Larry using a Python GUI interface, which is processed by a Python application that then sends a constrained application protocol (CoAP) message, via the connected DAGroot, telling the child motes carried on Larry’s back to start running their respective gait events which drive the servos on Larry’s legs. Top: physical system of a laptop with DAGroot connected communicating with the two child motes on Larry. Bottom: block diagram of the control logic occurring within the physical systems.
Figure 6: Gait synchronization error between the two motes starting from an initial state of zero error. Top: the motes are running on decentralized open loop control using independent on-board timers and are experiencing a drift of around 5 s/s (5 ppm) for a total of slighlty more than 2 ms over the 400 second experiment. This drift continues indefinitely. Bottom: the motes are running on decentralized synchronized control using the network’s ASN and are experiencing a drift of around 3 s/s (3 ppm). The motes resynchronize every 30 seconds, so the offset between them never exceeds 112 s.

V Results

With decentralized control, and are executing their respective gait events independently, yet the events must be time synchronized for proper gait performance. Our results compare the gait synchronization error using two gait control schemes: decentralized open loop control and decentralized synchronized control. Decentralized open loop control uses the free-running onboard clocks of and as independent references for time. Decentralized synchronized control relies on the ASN as a global time reference for making control decisions. If and have the same reference for the start of each gait period, their gait events will be performed in sync. The quality of the control will then be measured by the difference between the starts of and ’s gait period.

lets and ’s clocks run independently of each other, but they may have an error up to 10 ppm (as specified by the 32.768kHz crystal the OpenMote-cc2538 uses). This implies that one of the clocks will be running slightly faster than the other (less than 10 microseconds faster per second). As expected, this error is causing a drift between and ’s gait period starts, causing them to gradually fall out of sync over time (Fig. 6, top). For the 400-second sample run, the gait synchronization offset increases linearly to slightly over 2 milliseconds with a slope of -5 s/s. For a gait period of 1 second, a 2 millisecond difference does not have much noticeable effect on gait performance (i.e. , controlling the knees, is pulling the hexapod forwards 2 milliseconds later than it should). However, after slightly less than 28 hours of operation, the drift would cause and to be 180 degrees out of phase, causing Larry to squat instead of walk. He would then be in an uncontrollable state.

uses the ASN variable on and as a time reference to make control decisions. and will resynchronize their clocks anytime communication happens with the DAGroot (their time-source parent). For the 400-second sample run (Fig. 6, bottom), the offset between the two motes is never more than 112 microseconds, i.e. the two motes will never be out of sync by more than 112 microseconds. The slope of the line is around -3 microseconds per second, showing an error of 3 ppm between these two motes. The bounded gait synchronization error means that Larry can walk indefinitely without getting into an uncontrollable state.

The synchronization bound can be further reduced from 112 microseconds to 30 microseconds by having the child motes communicate with the DAGroot more often. The above data was collected using a worst case communication period of 30 seconds, resulting in a bound of 112 microseconds. Using the 32.768 kHz clock on the motes, in the best case we are limited to the clock period of kHz microseconds. By simply reducing the worse case communication period to 10 seconds, we were able to achieve a synchronization bound of around 30 microseconds (the 112 microsecond-bounded data is used here for ease of visual analysis). And with more sophisticated techniques such as adaptive synchronization this bound could be reduced even further [15]. Microsecond time synchronization using TSCH has been demonstrated by Elsts et al. [14].

The results show that for gait synchronization between and , the scheme , taking advantage of the TSCH mode of IEEE802.15.4 with the usage of the ASN implemented by OpenWSN, has the objectively better decentralized control performance.

Vi Conclusion

Decentralized control can be used to successfully coordinate actions in robotic systems. However, as the results show, success depends on keeping the various controllers time-synchronized. The controllers themselves cannot be relied on to stay time-synchronized due to their clock error of 10 ppm. Thus, the controllers need to be connected via a network to share a common time reference. The OpenWSN network presented here uses the TSCH mode of IEEE802.15.4 to ensure each controller’s time reference remains synchronized with respect to the DAGroot’s clock by usage of the ASN variable implemented by OpenWSN.

The application of a network-synchronized robotic system extends beyond the single-robot gait presented in this paper. Precisely coordinated decentralized control can be used in a wide range of robotic systems: for example, self-reconfiguring robots or intelligent robot swarms. Furthermore, the use of synchronized nodes implementing the OpenWSN protocol stack can be extended beyond just the three-node topology presented in this paper. There have been projects such as SensLab which uses 1024 synchronized nodes at once [17] and FIT IoT-LAB, a testbed for IoT technologies which is composed of 2728 synchronized wireless nodes and 117 mobile robots [18].


The authors would like to thank the Berkeley Sensor & Actuator Center and the UC Berkeley Swarm Lab for supporting this project.


  • [1] N. Sandell, P. Varaiya, M. Athans, and M. Safonov, “Survey of decentralized control methods for large scale systems,” IEEE Transactions on automatic Control, vol. 23, no. 2, pp. 108–128, 1978.
  • [2] A. M. Flynn and R. A. Brooks, “Building robots: Expectations and experiences,” in Intelligent Robots and Systems’ 89. The Autonomous Mobile Robots and Its Applications. IROS’89. Proceedings., IEEE/RSJ International Workshop on.   IEEE, 1989, pp. 236–243.
  • [3] D. S. Contreras and K. S. Pister, “Durability of silicon pin-joints for microrobotics,” in Manipulation, Automation and Robotics at Small Scales (MARSS), International Conference on.   IEEE, 2016, pp. 1–6.
  • [4] S. Bergbreiter and K. S. Pister, “Cotsbots: An off-the-shelf platform for distributed robotics,” in Intelligent Robots and Systems, 2003.(IROS 2003). Proceedings. 2003 IEEE/RSJ International Conference on, vol. 2.   IEEE, 2003, pp. 1632–1637.
  • [5] M. Rubenstein, A. Cornejo, and R. Nagpal, “Programmable self-assembly in a thousand-robot swarm,” Science, vol. 345, no. 6198, pp. 795–799, 2014.
  • [6] T. Watteyne, X. Vilajosana, B. Kerkez, F. Chraim, K. Weekly, Q. Wang, S. Glaser, and K. Pister, “Openwsn: a standards-based low-power wireless development environment,” Transactions on Emerging Telecommunications Technologies, vol. 23, no. 5, pp. 480–493, 2012.
  • [7] X. Vilajosana, P. Tuset, T. Watteyne, and K. Pister, “Openmote: open-source prototyping platform for the industrial iot,” in International Conference on Ad Hoc Networks.   Springer, 2015, pp. 211–222.
  • [8] A. Dunkels, B. Gronvall, and T. Voigt, “Contiki-a lightweight and flexible operating system for tiny networked sensors,” in Local Computer Networks, 2004. 29th Annual IEEE International Conference on.   IEEE, 2004, pp. 455–462.
  • [9] IEEE Std 802.15.4e-2012.
  • [10] IPv6 over Low power WPAN (6lowpan), IETF.
  • [11] IPv6 over the TSCH mode of IEEE 802.15.4e (6tisch), IETF.
  • [12] Z. Shelby, K. Hartke, and C. Bormann, “The constrained application protocol (coap),” IETF, 2014.
  • [13] C. B. Schindler, T. Watteyne, X. Vilajosana, and K. S. J. Pister, “Implementation and characterization of a multi-hop 6tisch network for experimental feedback control of an inverted pendulum,” in Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), 2017 15th International Symposium on.   IEEE, 2017, pp. 1–8.
  • [14] A. Elsts, S. Duquennoy, X. Fafoutis, G. Oikonomou, R. Piechocki, and I. Craddock, “Microsecond-accuracy time synchronization using the ieee 802.15. 4 tsch protocol,” in Local Computer Networks Workshops (LCN Workshops), 2016 IEEE 41st Conference on.   IEEE, 2016, pp. 156–164.
  • [15] T. Chang, T. Watteyne, K. Pister, and Q. Wang, “Adaptive synchronization in multi-hop tsch networks,” Computer Networks, vol. 76, pp. 165–176, 2015.
  • [16] LTC5800-IPM SmartMesh IP Node 2.4GHz 802.15.4e Wireless Mote-on-Chip,, Dust Networks, 2013.
  • [17] C. B. des Roziers, G. Chelius, T. Ducrocq, E. Fleury, A. Fraboulet, A. Gallais, N. Mitton, T. Noel, E. Valentin, and J. Vandaële, “Two demos using senslab: Very large scale open wsn testbed,” in Distributed Computing in Sensor Systems and Workshops (DCOSS), 2011 International Conference on.   IEEE, 2011, pp. 1–2.
  • [18] C. Adjih, E. Baccelli, E. Fleury, G. Harter, N. Mitton, T. Noel, R. Pissard-Gibollet, F. Saint-Marcel, G. Schreiner, J. Vandaele et al., “Fit iot-lab: A large scale open experimental iot testbed,” in Internet of Things (WF-IoT), 2015 IEEE 2nd World Forum on.   IEEE, 2015, pp. 459–464.