D-SLATS: Distributed Simultaneous Localization and Time Synchronization

Through the last decade, we have witnessed a surge of Internet of Things (IoT) devices, and with that a greater need to choreograph their actions across both time and space. Although these two problems, namely time synchronization and localization, share many aspects in common, they are traditionally treated separately or combined on centralized approaches that results in an ineffcient use of resources, or in solutions that are not scalable in terms of the number of IoT devices. Therefore, we propose D-SLATS, a framework comprised of three different and independent algorithms to jointly solve time synchronization and localization problems in a distributed fashion. The First two algorithms are based mainly on the distributed Extended Kalman Filter (EKF) whereas the third one uses optimization techniques. No fusion center is required, and the devices only communicate with their neighbors. The proposed methods are evaluated on custom Ultra-Wideband communication Testbed and a quadrotor, representing a network of both static and mobile nodes. Our algorithms achieve up to three microseconds time synchronization accuracy and 30 cm localization error.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

11/01/2017

TERSE-KF: Event-Trigger Diffusion Kalman Filter with Application to Localization and Time Synchronization

The performance of a distributed network state estimation problem depend...
10/19/2020

A Feasibility Study on SNTP and SPoT Protocols on Time Synchronization in Internet of Things

The new wave of computing allows users to explore their time in the Inte...
09/06/2018

Slotted ALOHA Overlay on LoRaWAN: a Distributed Synchronization Approach

LoRaWAN is one of the most promising standards for IoT applications. Nev...
12/10/2019

SENSE: Scalable Data Acquisition from Distributed Sensors with Guaranteed Time Coherence

Data analysis in the Internet of Things (IoT) requires us to combine eve...
02/12/2018

Localization in internets of mobile agents: A linear approach

Fifth generation (5G) networks providing much higher bandwidth and faste...
05/07/2019

Collaborative Localization and Tracking with Minimal Infrastructure

Localization and tracking are two very active areas of research for robo...
09/29/2021

DNN-assisted Particle-based Bayesian Joint Synchronization and Localization

In this work, we propose a Deep neural network-assisted Particle Filter-...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

With the dramatic increase in the number of wireless devices, it is important to coordinate timing among networked devices and to provide contextual information, such as location. Maintaining a shared notion of time is critical to the performance of many cyber-physical systems (CPS) such as wireless sensor networks, Big Science (Lipiński et al., 2011), swarm robotics (Regula and Lantos, 2013), high frequency trading (Estrela and Bonebakker, 2012), telesurgery (Natarajan and Ganz, 2009), and global-scale databases (Corbett et al., 2013)

. In addition, position estimation is necessary for different fields such as military

(Lowell, 2011), indoor and outdoor localization (Lingemann et al., 2004).

Since time and distance are causally associated with each other, seeking a solution for one of them provides information about the other. In many cases, accurate time synchronization among nodes is necessary for their precise localization, as is the case in techniques based on time-of-arrival (TOA) (Chan et al., 2006) and time-difference-of-arrival (TDOA) (Zhang et al., 2006) measurements. Moreover, a combined solution to time synchronization and localization problems in sensor networks could be obtained with less computational effort by tackling the two problems in a unified approach, instead of considering them as two separate problems. This type of approach has been addressed in previous works. The effect of clock mismatch on the accuracy of ranging systems is studied in (D’Amico et al., 2013), where TOA measurements are considered, and in (Gholami et al., 2013), where a maximum likelihood estimator for TDOA-based positioning was proposed. In (Wang et al., 2011) localization is investigated based on time of flight measurements for asynchronous networks using least squares methods. Least squares are also used in (Zhu and Ding, 2010) to achieve synchronization using TOA measurements, and in (Zheng and Wu, 2010) with accurate and inaccurate network anchors. In (Denis et al., 2006), the authors derived a distributed maximum log-likelihood estimator that fuses clock offset, bias, and range estimates. While these approaches present satisfactory theoretical results, they either make several non-practical assumptions of the underlying network, such as perfect synchronization among anchor nodes, or they lack experimental hardware evaluation of the proposed methods.

Centralized algorithms for localization and time synchronization, though theoretically might attain the optimal solution, are neither robust nor scalable to complex large-scale dynamical systems, where the nodes are distributed over a large geographical region. In order to perform simultaneous time synchronization and localization using centralized algorithms, all nodes must send their measurements to a fusion center that computes the estimates of position and clock parameters for every node. The information is then sent back to every node and the process is repeated. This strategy requires a large communication overhead, might not be energy efficient, and has a potentially critical failure point at the fusion center.

This paper introduces D-SLATS, a framework that is comprised of three different, independent, and distributed algorithms to achieve network time synchronization and accurate position estimates using time stamped message exchanges, filters and optimization techniques. DKAL is the first proposed algorithm which is based mainly on the distributed Extended Kalman Filter (EKF) with diffusion between wireless nodes. DKAL improves some aspects of (Cattivelli and Sayed, 2010) by decreasing some of the computational efforts. The second algorithm, DKALarge is based on the results from (Khan and Moura, 2008) and deals with extremely large scale systems where DKAL might not be the best option. It is important to note that the modifications proposed in our work to the algorithms presented in (Khan and Moura, 2008) and (Cattivelli and Sayed, 2010), can be used in other domains as well. Finally, we present DOPT, an optimization technique to synchronize and localize the wireless nodes in a distributed fashion. Since D-SLATS operates in a distributed fashion, it does not require a fusion center and the results from the three algorithms can work perfectly with different network topologies.

The three algorithms proposed are evaluated on custom ultra-wideband wireless hardware for networks with different topologies containing both static and mobile nodes. This work leverages ultra-wideband RF communication to make precise timing measurements. While UWB has improved non-line-of-sight performance in comparison to signal strength methods like those based on RSSI, it should be noted that UWB timing accuracy can deteriorate with increased environmental clutter and signal attenuation, as described in (Medvesek et al., 2015). We compare the proposed algorithms with a conventional centralized EKF, and we present the accuracy of the node position estimation, as well as, the time synchronization.

The rest of the paper is organized as the following: Related work is shown in Section 2. Section 3 provides an overall overview about the system model. We then go through D-SLATS algorithm by algorithm in Section 4. Then, Section 5 evaluates the introduced algorithms on static and mobile network of nodes. Finally, Section 6 concludes this paper.

2. Related Work

We investigate the related work in concurrent synchronization and localization. One category of previous work was based on distributed maximum likelihood estimators, theoretical Cramér Rao Lower Bounds (CRLB), batch estimation techniques such as least squares regression, and experimental heuristics. Gholami

et al. (Gholami et al., 2013)

derived a maximum likelihood estimator (MLE) for joint clock skew and position estimation. However, they assume that a number of reference nodes are perfectly synchronized with a reference clock and transmit their signals at a common time instant. Similarly, joint time synchronization and localization is solved in

(Zheng and Wu, 2010) with accurate and inaccurate anchors in terms of time synchronization and localization. In (Denis et al., 2006), Denis et al. derived a distributed maximum log likelihood estimator that fuses clock offset, bias, and range estimates. While these approaches outline theoretical estimators, they make several non practical assumptions of the underlying network such as perfect synchronization among anchor nodes, and do not evaluate their methods on actual hardware. Recently, ATLAS estimates locations using a maximum likelihood method and known-position beacon transmitters to synchronize the clocks of receivers (Weiser et al., 2016). Positioning using time-difference of arrival measurements is proposed in (Gustafsson and Gunnarsson, 2003).

Lately, accurate RF time-stamping, spearheaded by impulse-radio ultra-wideband (IR-UWB) devices (Dardari et al., 2009), has encouraged research on low error ranging techniques and their related time synchronization. Polypoint (Kempke et al., 2015), for instance, introduced a RF localization system which enables the real-time tracking and navigating of quadrotors through complex indoor environments. Another quadrotor localization framework was introduced in (Tiemann et al., 2015). Furthermore, clock bias measurements were used in (Ledergerber et al., 2015) to allow for simpler range measurements using one-way TOA/TDOA messages compared to the more expensive two-way protocols. These works are based on the popular DecaWave DW1000 radio (??, dw1). Recent developments in UWB communications offer high precision positioning through which a new range of applications is enabled (Vossiek et al., 2003). Moreover, (Kempke et al., 2015) demonstrated a 56 cm localization error with frequency 20 Hz and density 0.005 node/. On the other hand, (Ledergerber et al., 2015) achieves a 28 cm localization error. However, it used 100 Hz frequency with 0.037 node/. Finally, (Tiemann et al., 2015) shows a 20 cm localization error with frequency 10 Hz given a density of 0.054 node/.

A linear data model for joint localization and clock synchronization is proposed in (Chepuri et al., 2012). Its estimates included batch offline least squares methods. Estimators to precisely estimate range and clock parameters from the measurements are suggested in (Dwivedi et al., 2015). It used master-slave clock synchronization to improve round-trip time estimates. Impact of clock frequency offsets on the accuracy of ranging systems based on TOA measurements is analyzed in (D’Amico et al., 2013). More application-specific solutions include using unmanned aerial vehicles to relay GPS information to a network of energy constrained nodes, who then localize and synchronize themselves to the mobile GPS receiver, saving energy (Villas et al., 2015) and leveraging clock bias estimation to improve RF time-of-flight estimates for non-UWB transceivers (Etzlinger et al., 2014). For more complete treatments of localization and synchronization methods, we refer the reader to (Patwari et al., 2005) and (Sivrikaya and Yener, 2004), respectively.

3. System Model

Consider a set of nodes indexed by distributed geographically over some region. We say that two nodes are connected if they can communicate with each other, and we denote the neighborhood of a given node by the set that contains all the nodes that are connected to node . Our state-space model is

(1)

where the state of node at time is denoted by , and the measurement available to node from the neighborhood node is represented by . The process and measurement noise are and , respectively, and assumed to be Gaussian. The state update function is and

is the measurement function. The state vector consists of three components,

, where is the three dimensional position vector, is the clock time offset, and is the clock frequency bias. We adopt a convention where both and are described with respect to the master node, which can be any node. It is recommended to pick the master node in the middle of the network graph to achieve faster time synchronization. It is assumed that the nodes are static and that the clock parameters evolve according to the first-order affine approximation of the following dynamics,

(2)

where given that is the root node time which is the global time. Therefore, we can write the update function for a static node as

(3)

3.1. Measurement types

The D-SLATS architecture supports three types of measurements which are distinguished by the number of messages exchanged between a pair of nodes. These measurements types are shown in detail in Figure 1, where time stamps through denote the locally measured transmission (TX) and reception (RX) times stamps, and and define, respectively, the response and the round-trip durations between the appropriate pair of these timestamps. The propagation velocity of radio is taken to be the speed of light in a vacuum, denoted by .

The three message types are:

  • TYPE 1: a single transmission is sent from to , yielding two timestamps, one from the TX instant and another from the RX instant. TYPE 1 results in finding the counter difference at time denoted by which is the measurement of the difference between the clocks of each node. This is a time measurement that includes the effects of propagation delay .

    (4)
  • TYPE 2: when a TYPE 1 message is followed by a reply message we allow for round-trip timing calculations. TYPE 2 results in the single-sided two-way range which is a distance measurement between pair of nodes j and k, with error proportional to the response turnaround time . This is a noisy measurement due to frequency bias discrepancies between and .

    (5)
  • TYPE 3: one last transmission completes a handshake trio allowing for a more precise round-trip timing calculation. Type 3 results in finding the double-sided two-way range which is another distance measurement between nodes and based on a trio of messages between the nodes at time . The error is proportional to the relative frequency bias between the two devices integrated over the period. This is a more accurate estimate than due to mitigation of frequency bias errors from the additional message.

    (6)
Figure 1. The top of this diagram shows six synchronization events between three devices, labeled , , and

. Each event is classified as TYPE 1, TYPE 2 or TYPE 3 depending on the number of transmissions sent.

With that, the measurement vector at node , from node has the form

(7)

It is important to note that a subset of these measurements may be used rather than the full set, i.e, we can have experiments involving just , , or . The three measurements types can be translated to the state vector according to the clock model from and the physics relating the propagation speed, the duration between the propagation and the distance between the nodes. Thus, we have measurement function in following form

(8)

where

4. Algorithms

The time synchronization and localization problem focus on finding an estimate for the clock parameters and the 3D position (i.e the state of the system) using the timestamp measurements and the model of the system. We denote by the estimate of given the observations up to time where every node seeks to minimize the mean-square error .

4.1. Dkal

Our DKAL algorithm is derived from the Diffusion Extended Kalman filter presented in (Cattivelli and Sayed, 2010), which uses the information form rather than the conventional form. The advantage of the information form over the covariance form becomes more evident for some categories of problems. For instance, the information filter form is easier to distribute, initialize, and fuse compared to the covariance filter form in multisensor environments (Kim and Hong, 2005). Furthermore, it can reduce dramatically the computation and storage which is involved in the estimation of specific classes of large-scale interconnected systems (Bierman, 1977). Also, the update equations for the estimator are computationally simpler than the equations of the covariance form.

By proposing DKAL, we are seeking a distributed implementation that avoids the use of a fusion center and instead distributes the processing and communication across the sensor network. Among distributed algorithms, diffusion algorithms are amenable for easy real-time implementations, for good performance in terms of synchronization and localization accuracy, and for the fact that they are robust to node and link failure. Some of the disadvantages of the algorithm in (Cattivelli and Sayed, 2010) that we try to address include:

  • Taking the inverse of a large matrix is not feasible on embedded devices. A large-scale network implies inverting an error covariance matrix on each node with dimension .

  • Every node monitors the whole network state which is a disadvantage in large scale networks, as the size of and will increase dramatically.

To address the first issue, we propose modifying the measurement update in Algorithm 2 in (Cattivelli and Sayed, 2010) by using the Binomial inverse theorem. This provides a formula to compute from without having to invert any matrix. We propose the following modifications:

(9)
(10)

where , and . Thus we have . Since the Binomial inverse theorem states that

(11)

It follows that

(12)
(13)

The other issue of monitoring the whole network state will be addressed by the DKALarge and DOPT algorithms. The DKAL algorithm is comprised of three main steps: measurement update, diffusion update section, and time update steps. First, we define the following matrices obtained by linearizing the state update and the measurement update functions around some point .

(14)
(15)

The algorithm starts with the measurement update where every node obtains at time step . Next, in the diffusion step, information from the neighbors of node are combined in a convex manner to produce a new state estimate for the node. Finally, every node performs the time update step. The proposed algorithm can be summarized as shown in Algorithm 1.

Algorithm 1: DKAL

Start with and for all , and at every time instant , compute at every node :

Step 1: Measurement update:

(16)
(17)
(18)

Step 2: Diffusion update:

(20)

Step 3: Time update:

(21)
(22)

In the algorithm, and are the process covariance matrix at time , and the measurement noise covariance matrix of node at time , respectively. The elements represent the weights that are used by the diffusion algorithm to combine neighborhood estimates.

4.2. DKALarge

Figure 2. An example of how a system can be divided in three subsystems.
Figure 3. Example of values out of original .

As mentioned before, the disadvantage of DKAL relies on the fact that each node in the network has to monitor the state of the entire network and carried out expensive matrix operations. Therefore, we are going to tackle these disadvantages by proposing DKALarge.

The key idea behind this algorithm is to let every node monitor only its neighbors, or we can say its subsystem. Therefore, the size of the and will depend only on the number of neighbors which solves the two disadvantages of DKAL. DKALarge spatially decomposes a sparsely connected large-scale network before carrying out Kalman filtering operations. Network connectivity is used to break down the global system into sets of smaller subsystems, while maintaining inter-subsystem graph rigidity.

Each subsystem is indexed by and we denote the covariance of the subsystem by and the state of the subsystem by . Figure 2 shows an example of how a network can be broken down into three subsystems. Subsystem , which corresponds to node , monitors only nodes , , and so that the subsystem state vector and covariance matrix will be of reduced dimension only containing information about estimates for those three nodes. Similarly, subsystem monitors only nodes , , and , whereas subsystem considers only nodes and . The size of the state vectors and the covariance matrices are thus of a reduced form representative of the nodes defined to be in a given subsystem as shown in Figure 3.

The important step in this algorithm is to decompose the whole systems into smaller subsystems and let every node monitors only the nodes that are in its own subsystem. For the local error covariance matrices, it is known that , which is the normal inverse of the reduced matrix, does not equal which is the submatrix of and this represents a challenge in decomposing the whole system into subsystems. We will remove the superscript for simplicity from now.

The DKALarge algorithm addresses the previous challenge of distributed matrix inversions by employing the L-Banded inverse on the global covariance matrix, followed by the DICI-OR method presented in (Khan and Moura, 2008) in order to approximate the inverse of the large covariance matrix. These two methods can be summarized as follows.

  • L-Banded Inverse: We use the L-banded inversion provided in (Kavcic and Moura, 2000) in order to obtain an L-banded approximate matrix from . The L-banded structure is necessary in the application of the DICI-OR method stated below. The L-banded inverse method contains properties for approximating gaussian error process, which help with inverses of submatrices. This has also been presented in (Frakt et al., 2003) and (Grone et al., 1984).

  • DICI-OR Method: The DICI-OR method uses the L-banded structure of the matrix in achieving distributed matrix inversion from to . With the assumption that the non L-band elements are not specified in the covariance matrix , a collapse step is effected in order to fill in the non L-band elements. The iterate step is then employed to produce the elements that lie within the L-band of . The iterate step can be repeated to achieve quicker convergence in the state estimates.

The DKALarge algorithm leverages L-banded inverse and the DICI-OR method to achieve distributed matrix inversion. Unlike DKAL where global covariance matrices and state vectors are needed to be maintained at each node, these two methods enable us to decompose the problem into smaller parts. While (Khan and Moura, 2008) proposes operations in the information filter domain, the algorithm has been adapted to run in the Kalman filter domain in this work. is defined as and is the overrelaxation parameter in the initial conditions. Using equations (25) and (26) as initial conditions, we seek to move from to using the DICI-OR method. This uses the collapse step to calculate non L-band elements before proceeding to the iterate step to reconstruct the local matrix. The subscripts and are representative of the row and column indices of the matrix. The DICI-OR method can be summarized as following:

Collapse Step:

(23)

Iterate Step:

(24)

where is the th row of matrix S which is defined in the initial conditions of DICI-OR, i.e, equation 25. is the th row of matrix P, and is the th diagonal element of the diagonal matrix . The iterations index is determined by . This step is only done for elements within the L-band. The DKALarge can be divided mainly into three steps, namely, measurement update, publish measurements, and time update.

Algorithm 2: DKALarge

Start with and for all , and compute at every node :

Initial Conditions for DICI-OR:

(25)
(26)

Step 1: Measurement update:

(27)
(28)
(29)

Step 2: Publish measurements among neighbors which will replace their estimates.

Step 3: Time update:

(30)
(31)

4.3. Dopt

DOPT operates by letting each node monitors its neighbor nodes only, so the size of is limited locally by the number of connecting nodes, an improvement from the DKAL algorithm. In this algorithm, the unconstrained minimizations are solved numerically and simultaneously at each node by computing the values for the parameters that satisfy the first order optimality conditions. DOPT can be summarize in the steps presented in Algorithm 3. A more detailed discussion with the theoretical results for convergence of the current algorithm can be found in (Ferraz et al., 2017).

Algorithm 3: DOPT

Start with . At every time instant , do at every node :

Step 1: Get from neighboring nodes their estimated state .

Step 2: Find optimal estimate of node based on the received estimates and measured distances:

(32)
(33)
(34)

Step 3: Combine the results from the optimization step and send the new estimate to the neighbors .

The steps in Algorithm 3 have considered Type 3 messages which are more accurate. However, depending on the application considered, Type 2 can be enabled instead by replacing equation 34 with equation 35.

(35)

5. Evaluation

Figure 4. Experimental setup overview, including, UWB Anchor nodes, motion capture cameras, and mobile quadrotor UWB nodes
Figure 5. Custom anchor node with ARM Cortex M4 processor and UWB expansion.
Figure 6. Ceiling-mounted anchor with DW1000 UWB radio in 3D-printed enclosure.
Figure 7. CrazyFlie 2.0 quadrotor helicopter with DW1000 UWB expansion.

We evaluated the performance of the proposed algorithms in a custom ultra-wideband RF test bed based on the DecaWave DW1000 IR-UWB radio (??, dw1). The setup is shown in Figure 4. We can summarize the main components of our test bed as follows: (i) Fixed anchor nodes powered by an ARM Cortex M4 processor with Power over Ethernet (PoE) and an expansion slot for a custom daughter board containing the DW1000, as shown in Figure 7 and Figure 7. This radio is equipped with a temperature-compensated crystal oscillator with frequency equals 38.4 MHz and a stated frequency stability of ppm. We installed eight UWB anchor nodes in different positions in a 109 lab. Six anchors are placed on the ceiling (roughly 2.5 m high) and 2 were placed at waist height (about 1 m) to better disambiguate positions in the vertical axis. Each anchor node is connected to an Ethernet backbone both for power and for communication to the central server, and each is capable of sending messages of type 1, 2, and 3, and is fully controllable over a TCP/IP command structure from the central server. These nodes are placed so as to remain mostly free from obstructions, maximizing line-of-sight barring pedestrian interference. (ii) Battery-powered mobile nodes also with ARM Cortex M4 processors based on the CrazyFlie 2.0 helicopter (??, cra) and equipped with the very same DW1000 radio as shown in Figure 7; (iii) A motion capture system capable of 3D rigid body position measurement with less than 0.5 mm accuracy; and finally (iv) a centralized server for aggregation of UWB timing information and ground truth position estimates from the motion capture cameras. We adopt a right-handed coordinate system where is the vertical axis and and make up the horizontal plane. We used 2 Hz as a rate of broadcasting the local time of each node.

5.1. Case Study: Static Nodes

We demonstrate D-SLATS performance in distributed simultaneous localization and time synchronization of static nodes. To begin, nodes are placed in 8 distinct locations around the 10 9 area, roughly in the positions indicated by Figure 4. The goal of D-SLATS is to accurately estimate the positions of all network devices relative to one another, as well as, the relative clock offsets and frequency biases. This relative localization, or graph realization as it is sometimes called, is a well-researched field. Local minima, high computational complexity, and restrictions on graph rigidity are the main challenges in graph realization problems (Aspnes et al., 2004; Jackson and Jordán, 2005). D-SLATS does not put restrictions on the connectivity of the graph. A centralized Kalman filter (CKAL) is implemented as a baseline for comparison.

5.1.1. Position Estimation

The D-SLATS algorithms find relative positions. Therefore, we first superimpose the estimated positions onto the true positions of each node by use of a Procrustes transformation (Zhang and Wang, 2014). Specifically, the network topology as a whole is rotated and translated without scaling, until it most closely matches the true node positions. Once transformed, the error of a given node’s position is defined as the norm of the transformed position minus the true position.

We begin by showing the localization error of a fully connected network for all algorithms. Figure 8 shows the localization errors for DKAL, DKALarge, and DOPT, respectively, with Type 3 enabled for all algorithms. Enabling Type 3 inherits enabling Type 1 and Type 2, as Type 3 is a replicate of Type 1 and Type 2. Table 1 summarizes the localization error of the eight nodes using DKAL, DKALarge, DOPT, and CKAL. DKAL achieves 0.311 . While DKALarge and DOPT report 0.330 and 0.299 , respectively. Also, we should not that enabling Type 2 only instead of Type 3 in our experiments does not have great effect of the overall performance.

(a) DKAL
(b) DKALarge
(c) DOPT
Figure 8. Fully connected localization error.
Figure 9. Average localization error with fully connected network.
Figure 10. Average localization error where each node has 4 neighbors only.

We summarize the mean error of the position estimation for all nodes and for different localization algorithms in Figure 9. The centralized algorithm outperforms the distributed algorithms in D-SLATS, as expected. Note that DOPT has the fastest convergence time, and DKAL has the worst performance among the algorithms studied. In a second experiment, we show that D-SLATS supports different network topologies without requiring the graph to be fully connected. Figure 10 shows the localization error for the network topology where every node is only connected to four neighbors. DKALarge gave the worst performance as the DICI-OR algorithms is approximating the full covariance matrix inverse, as described before.

Algorithm node 0 node 1 node 2 node 3 node 4 node 5 node 6 node 7 mean std
DKAL 0.518 0.189 0.638 0.228 0.021 0.433 0.126 0.336 0.311 0.209
DKALarge 0.232 0.536 0.394 0.318 0.093 0.400 0.246 0.418 0.330 0.137
DOPT 0.402 0.202 0.530 0.193 0.156 0.309 0.199 0.397 0.299 0.133
CKAL 0.205 0.189 0.208 0.147 0.218 0.075 0.168 0.143 0.169 0.047
Table 1. localization error (m) of different static nodes.

5.1.2. Time synchronization

In order to test the time synchronization, we could configure a third party node to send a query message and compare the timestamp upon receiving that message as done in (Maróti et al., 2004). However, in order to decrease the uncertainty in our test mechanism, we choose the root node to do this job. This testing mechanism is better than others as reported in (Alanwar et al., 2016), and been used also in (C Lenzen, C Lenzen). As mentioned before, we choose node 0 to be the reference node, i.e, and . Table 2 shows the synchronization errors for all nodes with respect to node 0. DKAL comes with the best performance, then DKALarge and DOPT. We should note that the synchronization errors in Table 2 are reported for a fully connected network. The effect of decreasing the connectivity on the D-SLATS algorithms is shown in Figure 11. DKALarge starts in a good shape with a fully connected network, but then the effect of approximating the covariance inverse appears when decreasing the connectivity. DOPT performs better than DKAL for smaller connectivity networks.

Algorithm node 1 node 2 node 3 node 4 node 5 node 6 node 7 mean std
DKAL 0.807 9.088 1.868 2.332 5.936 9.624 5.342 5.000 3.502
DKALarge 5.223 6.448 5.203 5.339 5.863 3.313 4.222 5.087 1.036
DOPT 2.15 0.891 2.090 2.343 1.651 5.215 3.293 2.520 1.391
CKAL 1.362 2.045 1.440 1.517 1.792 0.267 0.708 1.304 0.617
Table 2. Synchronization error ( seconds) of different nodes with respect to node 0.
Figure 11. Log scale of the effect of decreasing the connectivity on the synchronization error.

5.2. Case Study: Mobile Nodes

We have shown that D-SLATS can be used to localize and synchronize a network of static nodes. We now present the case of a heterogeneous network containing both static and mobile nodes. To the eight anchor nodes topology used in Section 5.1, we add one mobile node in the form of a CrazyFlie quadrotor as shown in Figure 7. We analyzed the results of running D-SLATS based on Type 3 measurements with a fully connected graph.

A number of experiments were performed with quadrotors traveling with variable velocities. Figures 12, 13, and 14 show the results of traveling quadrotors self-localizing themselves for 2 minutes using DKAL, DKALarge, and DOPT, respectively, using Type 3 messages. The left plot of the three figures show a 3D comparison of the self-localizing estimated position and the ground truth position reported by the motion capture cameras. DKAL achieved the best self localization estimation with an RMSE of 75 cm. DKALarge and DOPT reported 89 cm and 116 cm, respectively. The top left subfigures of the three figures show the 3D localization error. The localization errors in each axis separately are shown in the top right sub figure. Finally, the location of the mobile node in the network determines to some extent the accuracy with which it can be localized. A device whose location is more central to the network (i.e. closer to the centroid defined as the mean of all node positions) is more likely to be fully constrained in terms of its relative position. This correlation can be loosely seen in the bottom right of the three figures.

Figure 12. Localization errors for DKAL in 3D for a single mobile node. Spatial errors (left) are shown with corresponding per-axis errors by time (top right). Additionally, the error is plotted against the mobile node’s distance from the network centroid (bottom right).
Figure 13. Localization errors for DKALarge in 3D for a single mobile node. Spatial errors (left) are shown with corresponding per-axis errors by time (top right). Additionally, the error is plotted against the mobile node’s distance from the network centroid (bottom right).
Figure 14. Localization errors for DOPT in 3D for a single mobile node. Spatial errors (left) are shown with corresponding per-axis errors by time (top right). Additionally, the error is plotted against the mobile node’s distance from the network centroid (bottom right).

6. Conclusion

This paper has described and evaluated D-SLATS, an architecture for distributed simultaneous time synchronization and localization of static and mobile nodes in a networks. Three different algorithms were proposed, namely DKAL, DKALarge, and DOPT, that perform distributed estimation in a scalable fashion. Several experiments using real, custom ultra-wideband wireless anchor nodes and mobile quadrotor nodes were conducted and they indicate that the proposed architecture is reliable in terms of performance, and efficient in the use of computational resources. D-SLATS is made possible by state-of-the-art advances in commercial ultra-wideband radios, and continued improvements to these devices will further underscore the importance of treating temporal and spatial variables in a joint fashion. Future directions will deal with testing over real large scale network by considering more nodes. Also, secure estimation of nodes under malicious attacks is another direction.

Acknowledgments

This research is funded in part by the National Science Foundation under awards CNS-1329755 and CNS-1329644. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of NSF, or the U.S. Government.

References

  • (1)
  • ?? (cra) Bitcraze CrazyFlie 2.0. https://www.bitcraze.io/. Accessed: 2017-01-01.
  • ?? (dw1) DecaWave DW1000 IR-UWB. http://www.decawave.com/products/dw1000. Accessed: 2017-01-01.
  • Alanwar et al. (2016) Amr Alanwar, Fatima Anwar, João P. Hespanha, and Mani Srivastava. 2016. Realizing Uncertainty-Aware Timing Stack in Embedded Operating System. In Proc. of the Embedded Operating Systems Workshop.
  • Aspnes et al. (2004) James Aspnes, David Goldenberg, and Yang Richard Yang. 2004. On the computational complexity of sensor network localization. In Algorithmic aspects of wireless sensor networks. Springer, 32–44.
  • Bierman (1977) Gerald J Bierman. 1977. An application of the square root information filter to large scale linear interconnected systems. (1977).
  • C Lenzen (C Lenzen) R Wattenhofer C Lenzen, P Sommer. PulseSync: An Efficient and Scalable Clock Synchronization Protocol. In IEEE/ACM TRANSACTIONS ON NETWORKING.
  • Cattivelli and Sayed (2010) Federico S Cattivelli and Ali H Sayed. 2010. Distributed nonlinear Kalman filtering with applications to wireless localization. In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 3522–3525.
  • Chan et al. (2006) Yiu-Tong Chan, Wing-Yue Tsui, Hing-Cheung So, and Pak-chung Ching. 2006. Time-of-arrival based localization under NLOS conditions. IEEE Transactions on Vehicular Technology 55, 1 (2006), 17–24.
  • Chepuri et al. (2012) Sundeep Prabhakar Chepuri, Geert Leus, and A van der Veen. 2012. Joint localization and clock synchronization for wireless sensor networks. In Signals, Systems and Computers (ASILOMAR), 2012 Conference Record of the Forty Sixth Asilomar Conference on. IEEE, 1432–1436.
  • Corbett et al. (2013) James C Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, Jeffrey John Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, and others. 2013. Spanner: Google’s globally distributed database. ACM Transactions on Computer Systems (TOCS) 31, 3 (2013), 8.
  • D’Amico et al. (2013) A.A. D’Amico, L. Taponecco, and U. Mengali. 2013. Ultra-Wideband TOA Estimation in the Presence of Clock Frequency Offset. Wireless Communications, IEEE Transactions on 12, 4 (April 2013), 1606–1616. DOI:http://dx.doi.org/10.1109/TWC.2013.013013.120319 
  • Dardari et al. (2009) D. Dardari, A. Conti, U. Ferner, A. Giorgetti, and M.Z. Win. 2009. Ranging With Ultrawide Bandwidth Signals in Multipath Environments. Proc. IEEE 97, 2 (Feb 2009), 404–426. DOI:http://dx.doi.org/10.1109/JPROC.2008.2008846 
  • Denis et al. (2006) Benoit Denis, Jean-Benoît Pierrot, and Chadi Abou-Rjeily. 2006. Joint distributed synchronization and positioning in UWB ad hoc networks using TOA. Microwave Theory and Techniques, IEEE Transactions on 54, 4 (2006), 1896–1911.
  • Dwivedi et al. (2015) Satyam Dwivedi, Alessio De Angelis, Dave Zachariah, and Peter Händel. 2015. Joint Ranging and Clock Parameter Estimation by Wireless Round Trip Time Measurements. CoRR abs/1501.05450 (2015). http://arxiv.org/abs/1501.05450
  • Estrela and Bonebakker (2012) Pedro V Estrela and Lodewijk Bonebakker. 2012. Challenges deploying PTPv2 in a global financial company. In 2012 IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication Proceedings. IEEE, 1–6.
  • Etzlinger et al. (2014) Bernhard Etzlinger, Folker Meyer, Henk Wymeersch, Franz Hlawatsch, Gunter Muller, and Andreas Springer. 2014. Cooperative simultaneous localization and synchronization: Toward a low-cost hardware implementation. In Sensor Array and Multichannel Signal Processing Workshop (SAM), 2014 IEEE 8th. IEEE, 33–36.
  • Ferraz et al. (2017) Henrique Ferraz, Amr Alanwar, Mani Srivastava, and João P. Hespanha. 2017. Node Localization Based on Distributed Constrained Optimization using Jacobi’s Method. Submitted to conference publication.
  • Frakt et al. (2003) Austin B Frakt, Hanoch Lev-Ari, and Alan S Willsky. 2003.

    A generalized Levinson algorithm for covariance extension with application to multiscale autoregressive modeling.

    IEEE Transactions on Information Theory 49, 2 (2003), 411–424.
  • Gholami et al. (2013) Mohammad Reza Gholami, Sinan Gezici, and Erik G Strom. 2013. TDOA based positioning in the presence of unknown clock skew. Communications, IEEE Transactions on 61, 6 (2013), 2522–2534.
  • Grone et al. (1984) Robert Grone, Charles R Johnson, Eduardo M Sá, and Henry Wolkowicz. 1984. Positive definite completions of partial Hermitian matrices. Linear algebra and its applications 58 (1984), 109–124.
  • Gustafsson and Gunnarsson (2003) Fredrik Gustafsson and Fredrik Gunnarsson. 2003. Positioning using time-difference of arrival measurements. In Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP’03). 2003 IEEE International Conference on, Vol. 6. IEEE, VI–553.
  • Jackson and Jordán (2005) Bill Jackson and Tibor Jordán. 2005. Connected rigidity matroids and unique realizations of graphs. Journal of Combinatorial Theory, Series B 94, 1 (2005), 1–29.
  • Kavcic and Moura (2000) Aleksandar Kavcic and José MF Moura. 2000. Matrices with banded inverses: Inversion algorithms and factorization of Gauss-Markov processes. IEEE Transactions on Information Theory 46, 4 (2000), 1495–1509.
  • Kempke et al. (2015) Benjamin Kempke, Pat Pannuto, and Prabal Dutta. 2015. PolyPoint: Guiding Indoor Quadrotors with Ultra-Wideband Localization. In Proceedings of the 2Nd International Workshop on Hot Topics in Wireless (HotWireless ’15). ACM, New York, NY, USA, 16–20. DOI:http://dx.doi.org/10.1145/2799650.2799651 
  • Khan and Moura (2008) Usman A Khan and José MF Moura. 2008. Distributing the Kalman filter for large-scale systems. IEEE Transactions on Signal Processing 56, 10 (2008), 4919–4935.
  • Kim and Hong (2005) Yong-Shik Kim and Keum-Shik Hong. 2005. Federated information mode-matched filters in ACC environment. International Journal of Control Automation and Systems 3, 2 (2005), 173–182.
  • Ledergerber et al. (2015) Anton Ledergerber, Michael Hamer, and Raffaello D’Andrea. 2015. A robot self-localization system using one-way ultra-wideband communication. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on. 3131–3137. DOI:http://dx.doi.org/10.1109/IROS.2015.7353810 
  • Lingemann et al. (2004) Kai Lingemann, Hartmut Surmann, Andreas Nuchter, and Joachim Hertzberg. 2004. Indoor and outdoor localization for fast mobile robots. In Intelligent Robots and Systems, 2004.(IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on, Vol. 3. IEEE, 2185–2190.
  • Lipiński et al. (2011) Maciej Lipiński, Tomasz Włostowski, Javier Serrano, and Pablo Alvarez. 2011. White rabbit: A PTP application for robust sub-nanosecond synchronization. In Precision Clock Synchronization for Measurement Control and Communication (ISPCS), 2011 International IEEE Symposium on. IEEE, 25–30.
  • Lowell (2011) John R Lowell. 2011. Military applications of localization, tracking, and targeting. IEEE Wireless Communications 2, 18 (2011), 60–65.
  • Maróti et al. (2004) Miklós Maróti, Branislav Kusy, Gyula Simon, and Ákos Lédeczi. 2004. The Flooding Time Synchronization Protocol. In Proceedings of the 2Nd International Conference on Embedded Networked Sensor Systems (SenSys ’04). ACM, New York, NY, USA, 39–49. DOI:http://dx.doi.org/10.1145/1031495.1031501 
  • Medvesek et al. (2015) J Medvesek, A Symington, A Trost, and S Hailes. 2015. Gaussian process inference approximation for indoor pedestrian localisation. Electronics Letters 51, 5 (2015), 417–419.
  • Natarajan and Ganz (2009) Sriram Natarajan and Aura Ganz. 2009. SURGNET: an integrated surgical data transmission system for telesurgery. International journal of telemedicine and applications 2009 (2009), 3.
  • Patwari et al. (2005) N. Patwari, J.N. Ash, S. Kyperountas, A.O. Hero, R.L. Moses, and N.S. Correal. 2005. Locating the nodes: cooperative localization in wireless sensor networks. Signal Processing Magazine, IEEE 22, 4 (July 2005), 54–69. DOI:http://dx.doi.org/10.1109/MSP.2005.1458287 
  • Regula and Lantos (2013) Gergely Regula and Bela Lantos. 2013. Formation control of a large group of UAVs with safe path planning. In Control & Automation (MED), 2013 21st Mediterranean Conference on. IEEE, 987–993.
  • Sivrikaya and Yener (2004) F. Sivrikaya and B. Yener. 2004. Time synchronization in sensor networks: a survey. Network, IEEE 18, 4 (July 2004), 45–50. DOI:http://dx.doi.org/10.1109/MNET.2004.1316761 
  • Tiemann et al. (2015) J. Tiemann, F. Schweikowski, and C. Wietfeld. 2015. Design of an UWB indoor-positioning system for UAV navigation in GNSS-denied environments. In Indoor Positioning and Indoor Navigation (IPIN), 2015 International Conference on. 1–7. DOI:http://dx.doi.org/10.1109/IPIN.2015.7346960 
  • Villas et al. (2015) Leandro A. Villas, Daniel L. Guidoni, Guilherme Maia, Richard W. Pazzi, Jó Ueyama, and Antonio A. Loureiro. 2015. An Energy Efficient Joint Localization and Synchronization Solution for Wireless Sensor Networks Using Unmanned Aerial Vehicle. Wirel. Netw. 21, 2 (Feb. 2015), 485–498. DOI:http://dx.doi.org/10.1007/s11276-014-0802-2 
  • Vossiek et al. (2003) Martin Vossiek, Leif Wiebking, Peter Gulden, J Weighardt, and Clemens Hoffmann. 2003. Wireless local positioning-concepts, solutions, applications. In Radio and Wireless Conference, 2003. RAWCON’03. Proceedings. IEEE, 219–224.
  • Wang et al. (2011) Yiyin Wang, Xiaoli Ma, and Geert Leus. 2011. Robust time-based localization for asynchronous networks. IEEE Transactions on Signal Processing 59, 9 (2011), 4397–4410.
  • Weiser et al. (2016) Adi Weller Weiser, Yotam Orchan, Ran Nathan, Motti Charter, Anthony J Weiss, and Sivan Toledo. 2016. Characterizing the Accuracy of a Self-Synchronized Reverse-GPS Wildlife Localization System. In 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 1–12.
  • Zhang et al. (2006) Cemin Zhang, Michael Kuhn, Brandon Merkl, Aly E Fathy, and Mohamed Mahfouz. 2006. Accurate UWB indoor localization system utilizing time difference of arrival approach. In 2006 IEEE Radio and Wireless Symposium.
  • Zhang and Wang (2014) P. Zhang and Q. Wang. 2014. On Using the Relative Configuration to Explore Cooperative Localization. IEEE Transactions on Signal Processing 62, 4 (Feb 2014), 968–980. DOI:http://dx.doi.org/10.1109/TSP.2013.2297680 
  • Zheng and Wu (2010) Jun Zheng and Yik-Chung Wu. 2010. Joint time synchronization and localization of an unknown node in wireless sensor networks. Signal Processing, IEEE Transactions on 58, 3 (2010), 1309–1320.
  • Zhu and Ding (2010) Shouhong Zhu and Zhiguo Ding. 2010. Joint synchronization and localization using TOAs: A linearization based WLS solution. IEEE Journal On Selected areas in communications 28, 7 (2010), 1017–1025.