A Multi-Static Radar Network with Ultra-Wideband Radio-Equipped Devices

Abstract

A growing number of devices, from car key fobs to mobile phones to WiFi-routers, are equipped with ultra-wideband radios. In the network formed by these devices, communicating modules often estimate the channel impulse response to employ a matched filter to decode transmitted data or to accurately time stamp incoming messages when estimating the time-of-flight for localization. This paper investigates how such measurements of the channel impulse response can be utilized to augment existing ultra-wideband communication and localization networks to a multi-static radar network. The approach is experimentally evaluated using off-the-shelf hardware and simple, distributed filtering, and shows that a tag-free human walking in the space equipped with ultra-wideband modules can be tracked in real time. This opens the door for various location-based smart home applications, ranging from smart audio and light systems to elderly monitoring and security systems.

1. Introduction

Ultra-wideband (UWB) technology is built into a growing number of devices. Larger companies are actively starting to invest in it [1,2]. Car key fobs equipped with UWB radios enable secure locking and unlocking of cars [3], phones equipped with UWB radios can share data based on their relative position [4], and WiFi-routers equipped with UWB radios offer an even larger range of localization-based services [5]. These applications are possible as UWB technology not only allows devices to communicate with each other, but also to localize with respect to each other. However, the devices are only aware of other UWB radio-equipped devices and need to rely on cameras or other sensors to sense and react to their environment.

The work presented in this paper aims to augment UWB radio-equipped devices with the sensing capabilities of a multi-static radar network, thereby increasing their range of applications for entertainment purposes [6], elderly monitoring [7], security [8], or smart home applications in general (e.g., light and temperature control) [9]. In contrast to most existing UWB radar systems, which require specialized hardware, the system presented herein only relies on three or more communicating devices equipped with Decawave’s off-the-shelf, low-cost DWM1000 chip [10]. Figure 1 schematically shows how these devices can form a multi-static radar network and thereby track a nearby person,

• by first localizing themselves with respect to each other,
• by tracking changes in the channel impulse response of each communication channel,
• by fusing these observed changes into a position estimate of a nearby person.

Since the presented signal processing algorithms are lightweight, they can be deployed in real time on devices with limited computational power, as for example found in Internet of Things (IoT) applications. The focus of this paper lies on step 2, but step 3 is also briefly discussed for evaluation purposes as experimental results are presented for the complete pipeline. For step 1, the reader is referred to [11] and the references therein. The dataset on which the experimental results presented are based is publicly available at [12]. See [13] or Supplementary Materials Video S2 for a video showing the system’s performance.

The remainder of this paper is structured as follows: Section 2 discusses related work. Section 3 introduces the necessary preliminaries on the channel impulse response (CIR), before its filtering is discussed in Section 4. The fusion of multiple filtered CIR to a position estimate of a tracked person is discussed in Section 5 and experimental results are presented in Section 6. Concluding remarks on the work, its limitations, and on future research directions are made in Section 7.

2. Related Work

Radio frequency (RF)-based, device-free localization has a long history, starting with the invention of the first radar systems in the early 20th century [14]. With the advent of low-power radio systems, human-centric, device-free localization recently started to emerge in indoor and IoT applications. For a general overview of these recent systems, the reader is referred to the survey papers [15,16], and, for an overview focusing on assisted living applications, the reader is referred to [17].

The work on human centric, device-free RF based localization can be split into systems where the transmitter and receiver are collocated, and into systems where they are separated. The former systems are known as monostatic radar systems, and usually have the transmitter and receiver driven by the same RF-clocks [18,19]. This rigid synchronization between the signal source and receiver enables accurate phase difference and correlation measurements of the transmitted and received signals. The latter systems are referred to as bi-static or multi-static radar systems and either combine multiple monostatic radars (as presented in [20,21]), or have the transmitters and receivers separated and driven by independent RF-clocks (as was done in [22]). In this case, they need to employ either optical fibres, wireless links, or early-late locked loops to synchronize their RF-clocks ([23], p. 293), which is also the case for the DWM1000 modules used in this work. Different transmitter and receiver positions or the measurement fusion of multiple distributed monostatic radars have the ability to improve radar performance as further discussed in [24].

Given the widespread use of WiFi-enabled devices, it is convenient to use their narrow band radio signals for localization. However, susceptibility to multi-path fading and the low temporal resolution make it hard to develop accurate tracking systems. Fingerprinting techniques help to improve the accuracy of such systems [25] but generally require labeled data gathering, which precludes fast deployment. An alternative approach to achieve a better accuracy is to increase the bandwidth by combining all available 5 GHz WiFi channels [26]. This is similar to systems which directly employ UWB signals for localization. Two frequently used UWB approaches are pulse-based UWB radars [22,23] and frequency-modulated continuous wave radars [18,19,27]. Both methods have their respective advantages [28], but, as pulse-based UWB is currently mainly used for device to device communication and active localization, it is convenient to extend such pulse-based UWB networks with the ability to sense their environment. This was also proposed in [22], where small temporal variations in the received UWB signals are utilized to localize a stationary human in a post-processing step. In addition, the 5G cell-phone networks utilize a large bandwidth, which makes them suitable for passive localization as well. First, experimental studies on passive localization using 5G networks to increase safety in vehicular environments are presented in [29,30]. Body proximity and motion detection have already proved feasible with previous generation cell-phone networks in [31].

Most of the work on UWB-based, human-centric, device-free localization is based on commercial, or custom-built, specific radar hardware. Therefore, these systems are able to achieve a high accuracy and resolution enabling them not only to track and identify multiple humans [19], but also to track them behind walls [32], and to remotely monitor their heart rate [33] and breathing frequency [34]. In contrast to employing these purposely built devices, researchers have also focused on how Decawave’s low cost, off-the-shelf UWB radios can be utilized for these applications, as also done in this work. For example, the authors in [35] demonstrated heart rate monitoring by analyzing the CIR between a DWM1000 radio placed on the chest and another placed on the back of a person. Passive localization using Decawave’s radios was investigated in [36]. By post-processing CIR measurements, they were able to detect a human in a corridor with up to 20 $\mathrm{m}$ distance to the UWB modules. The work presented in this paper can be seen as a significant progression of their work and previous work in as much as we track a human in real time by means of a self-localizing, multi-static radar network with off-the-shelf UWB radio modules. Unlike the systems discussed in [19,22], the method presented in this paper is not able to track multiple targets or to detect a static target. These limitations should be addressed in future work and would further close the gap to systems employing radar-specific hardware. Potential mitigation approaches are discussed in subsection of Section 7.

3. Channel Impulse Response

In UWB communication and localization systems, the channel impulse response between a transmitter and receiver is often estimated. It is used for matched filtering techniques to decode transmitted data [37,38], for accurately time-stamping received signals [39] and to establish a secure communication link between any two communicating modules [40]. In addition, in radar applications, the channel impulse response is of interest, as objects can be detected based on the trace they leave behind in the measured CIR, which will also be utilized in this work.

A good overview of the CIR and UWB propagation channels is given in [41]. The CIR is informative of the different paths a signal takes when traveling from a transmitter to a receiver, as illustrated in Figure 2. Due to the short pulse duration of UWB signals, it is often possible to separate the first peak caused by the direct path from the peaks of other reflected and scattered paths. The location of first path ${\tau }_{\mathrm{FP}}$ within the CIR is generally found by means of a leading edge detection algorithm and allows accurate measurement of the time of flight.

If a new object appears in the scene, the newly measured CIR will differ from the previously measured CIR without the object. By subtracting the previous CIR measurement, termed background, from the current CIR measurement, and by applying a leading edge detection algorithm on the difference of the two measurements, the location within the CIR of the signal reflected from the object is found. This location is termed the target path location ${\tau }_{\mathrm{TP}}$ and constrains the position of the target object, ${\mathit{p}}_{\mathrm{T}}$, to an ellipse in two dimensions. This ellipse has the position of the transmitter, ${\mathit{p}}_{\mathrm{tx}}$, and receiver, ${\mathit{p}}_{\mathrm{rx}}$, as foci points and a major axis of length $c·{\tau }_{\mathrm{TP}}$, i.e.,

$$\begin{array}{c}\hfill \Vert {\mathit{p}}_{\mathrm{tx}}-{\mathit{p}}_{\mathrm{T}}\Vert +\Vert {\mathit{p}}_{\mathrm{rx}}-{\mathit{p}}_{\mathrm{T}}\Vert =c·{\tau }_{\mathrm{TP}}\end{array}$$

(1)

where we used c to denote the speed of light. This principle of subtracting the background from the current signal is termed “background subtraction” and is visualized in Figure 2. It is a widely used clutter removal technique in radar applications [42].

By having multiple transmitter and receiver pairs, multiple ellipses can be combined and the new object can be localized at the intersection point of these ellipses as visualized in Figure 1.

CIR Measurements with Two DWM1000 Transceivers

In contrast to traditional mono-static radar systems, the transmitting and receiving radio modules in a UWB network are not triggered by the same RF-clock, but each have their own independent RF-clock running with slightly different and not always constant speed. Hence, CIR measurements between two such modules taken at different times generally also sample the CIR at different times. By aligning these different CIR measurements with the estimated first path location ${\tau }_{\mathrm{FP}}$, as outlined in [36], a high resolution estimate of the CIR can be obtained. Note that the DWM1000 modules do not directly estimate the CIR, but its complex envelope ([23], p. 281). However, for simplicity, we refer to the magnitude of this envelope simply as CIR in the following.

Plot (a) in Figure 3 shows such a high resolution, accumulated CIR consisting of 50 CIR measurements obtained with two DWM1000 within a time period $\Delta t=27 \mathrm{m}\mathrm{s}$ in a static environment. The DWM1000 module provides CIR measurements with a resolution of approximately $\Delta {\tau }_{\mathrm{s}}=1/\left(2f_c\right)\approx 1 \mathrm{n}\mathrm{s}$, where $f_c=499.2 \mathrm{M}\mathrm{Hz}$ is the chipping frequency. Furthermore, it estimates the location of the first path ${\tau }_{\mathrm{FP}}$ within each CIR measurement with a resolution of $\frac{1}{64}\Delta {\tau }_{\mathrm{s}}$ [43]. The samples of one such measurement are accentuated with red dots in Figure 3.

Plot (b) in Figure 3 shows the accumulated CIR for the same channel, but at a different time and with a person walking in the vicinity of the transmitting and receiving antenna as shown in Figure 4. It again consists of 50 CIR measurements, also obtained within a time period $\Delta t=27 \mathrm{m}\mathrm{s}$. While the mean signal of the accumulated CIR seems to be similar for the two plots, the variance of the accumulated CIR in the region $12 \mathrm{n}\mathrm{s}\le \tau -{\tau }_{\mathrm{FP}}<16 \mathrm{n}\mathrm{s}$ is significantly larger in plot b). This is due to the person walking somewhere on the ellipse with a major axis of length $c·{\tau }_{\mathrm{TP}}$ with ${\tau }_{\mathrm{TP}}\approx {\tau }_{\mathrm{FP}}+12 \mathrm{n}\mathrm{s}$ and the receiver and transmitter as foci points. Therefore, the filtering algorithm presented in the following section is not based on the change in the mean (as illustrated in Figure 1), but on the change in the variance of the accumulated CIR.

4. CIR Filtering

It was shown in the previous section that a moving person in the scene leads to regions with a comparably higher variance in the accumulated CIR. The location of the first such region corresponds to the location of the target path ${\tau }_{\mathrm{TP}}$ used in Equation (1) to define the ellipse including the target position ${\mathit{p}}_{\mathrm{T}}$. The goal of the following filtering algorithm is to extract the location of the target path ${\tau }_{\mathrm{TP}}$ from the CIR measurements.

Instead of transmitting the measured CIR to a central server and performing the filtering there, it is proposed to process the measured CIR on the host microcontroller of each DWM1000 module and to only transmit the extracted location of the target path. This frees up airtime due to the shorter package length, but also requires a lightweight filtering algorithm such that it can be run on the host microcontroller. The proposed recursive filtering approach is light both in terms of memory and computations. Note that more elaborate, finite impulse response or non-causal filtering approaches could improve the performance but were not investigated as they were not in line with our hardware and real-time constraints.

A flowchart of the proposed algorithm is shown in Figure 5. Its individual steps are discussed in the following.

4.1. Mean Filtering

The proposed algorithm tracks the mean of the currently measured CIR in form of the coefficients $\mathit{h}=\left(h_0,h_1,\cdots ,h_{N_{\mathrm{knots}}-1}\right)$ of a uniformly-spaced, piece-wise linear function, i.e.,

$$\begin{array}{c}\hfill h\left(\tau \right)=\left(1-\frac{\tau -{\tau }_i}{\Delta {\tau }_{\mathrm{knots}}}\right)h_i+\left(\frac{\tau -{\tau }_i}{\Delta {\tau }_{\mathrm{knots}}}\right)h_{i+1},\end{array}$$

(2)

where i is such that ${\tau }_i\le \tau <{\tau }_{i+1}$, and where the knots ${\tau }_i$ of the piece-wise linear parameterization are given as

$$\begin{array}{c}\hfill {\tau }_i={\tau }_{\mathrm{start}}+\Delta {\tau }_{\mathrm{knots}}·i\mathrm{for}i=\left\{0,\cdots ,N_{\mathrm{knots}}-1\right\}\end{array}$$

(3)

with

$$\begin{array}{c}\hfill N_{\mathrm{knots}}=\frac{{\tau }_{\mathrm{end}}-{\tau }_{\mathrm{start}}}{\Delta {\tau }_{\mathrm{knots}}}+1.\end{array}$$

(4)

The knot spacing $\Delta {\tau }_{\mathrm{knots}}$ is chosen such that it is a divisor of the CIR measurement sampling period $\Delta {\tau }_{\mathrm{s}}$, i.e.,

$$\begin{array}{c}\hfill \Delta {\tau }_{\mathrm{knots}}=\frac{\Delta {\tau }_{\mathrm{s}}}{m},\end{array}$$

(5)

where $m\in \{1,2,\cdots ,64\}$. The smallest spacing $m=64$ is given by the resolution of the DWM1000’s leading edge detection algorithm, which estimates the first path estimate location ${\tau }_{\mathrm{FP}}$ within the measured CIR with a resolution of $\frac{1}{64}\Delta {\tau }_{\mathrm{s}}$. Such knot spacing reduces the number of necessary computations to update knot coefficients as explained in more detail in Appendix A. In order to have an estimate of the noise floor, ${\tau }_{\mathrm{start}}$ is chosen to be before the first path location, such that $h_0$ is indicative of the noise floor. Given the DW1000 settings as specified in

Table A1. The settings of the DWM1000 modules.

Channel Number	4 (Carrier Frequency 3993.6 MHz, Bandwith 900 MHz)
Pulse Repetition Frequency	16 MHz
Data Rate	6.8 Mbps
Preamble Length	128 Symbols
Preamble Accumulation Size	8
Preamble Code	7
Transmit Power Control	19 dB Gain

, the CIR starts to rise approximately 1 $\mathrm{n}$$\mathrm{s}$ before the estimated first path location as visible in Figure 3; hence, ${\tau }_{\mathrm{FP}}-{\tau }_{\mathrm{start}}$ must be larger than 1 $\mathrm{n}$$\mathrm{s}$. To add a safety margin, we choose ${\tau }_{\mathrm{FP}}-{\tau }_{\mathrm{start}}=4 \mathrm{n}\mathrm{s}$.

Upon reception of a CIR measurement

$$\begin{array}{cc}\hfill \mathit{z}& =\left(z_0,z_1,\cdots ,z_{N_{\mathrm{s}}-1}\right)\hfill \end{array}$$

(6)

composed of N_s CIR samples with the corresponding sampling times

$$\begin{array}{cc}\hfill {\mathit{\tau }}_{\mathrm{meas}}& =\left({\tau }_{\mathrm{meas},0},{\tau }_{\mathrm{meas},1},\cdots ,{\tau }_{\mathrm{meas},N_{\mathrm{s}}-1}\right)\hfill \end{array}$$

(7)

the innovation signal $\mathit{y}=\left(y_0,y_1,\cdots ,y_{N_{\mathrm{s}}-1}\right)$ with respect to the current piecewise parameterization (Equation (2)) is calculated, i.e.,

$$\begin{array}{c}\hfill y_j=z_j-h\left({\tau }_{\mathrm{meas},j}\right)\mathrm{for}j=\{0,1,\cdots ,N_{\mathrm{s}}-1\}.\end{array}$$

(8)

This innovation signal is then used to recursively update the coefficients $\mathit{h}$

$$\begin{array}{c}\hfill \mathit{h}\leftarrow \mathit{h}+\mathit{K}\mathit{y},\end{array}$$

(9)

where the gain matrix $\mathit{K}\in {\mathbb{R}}^{N_{\mathrm{knots}}\times N_{\mathrm{s}}}$ is further specified in Appendix A. A visualization of this parametrization is shown in Figure 6.

The coefficients $\mathit{h}$ are initialized with the first received measurement $\mathit{z}$ as

$$\begin{array}{c}\hfill h_i\leftarrow \left\{\begin{array}{cc}{z}_0,\hfill & \mathrm{if}{\tau }_i<{\tau }_{\mathrm{meas},0}\hfill \\ z_j,\hfill & \mathrm{if}{\tau }_{\mathrm{meas},j}\le {\tau }_i<{\tau }_{\mathrm{meas},j+1}\hfill \\ z_{N_{\mathrm{s}}-1},\hfill & \mathrm{if}{\tau }_i\ge {\tau }_{\mathrm{meas},N_{\mathrm{s}}-1}\hfill \end{array}\right.\end{array}$$

(10)

for all $i\in \{0,1,\cdots ,N_{\mathrm{knots}}-1\}$, before they are updated according to Equation (9) with the consecutive measurements.

4.2. Innovation Signal Filtering

In addition to the mean, the proposed algorithm also tracks a variance metric of the currently measured CIR. In contrast to the mean, the variance metric is parameterized as a piecewise constant function with coefficients $\tilde{\mathit{h}}=({\tilde{h}}_0,{\tilde{h}}_1,\cdots ,{\tilde{h}}_{N_{\mathrm{knots}}-2})$, which are updated according to

$$\begin{array}{c}\hfill {\tilde{h}}_l\leftarrow {\tilde{h}}_l+\tilde{\alpha }(|y_j|-{\tilde{h}}_l)\end{array}$$

(11)

where $l\in \{0,1,\cdots ,N_{\mathrm{knots}}-2\}$ is such that ${\tau }_l\le {\tau }_j<{\tau }_{l+1}$ and where the gain $\tilde{\alpha }\in (0,1)$. See Figure 6 for a visualization of the parametrization.

In order to distinguish variations in the tracked CIR from variations due to parameterization errors, system imperfections and environmental background noise, we track the background variation coefficients ${\tilde{\mathit{h}}}^{\mathrm{B}}$ which are updated in the same way as $\tilde{\mathit{h}}$, i.e.,

$$\begin{array}{c}\hfill {\tilde{h}}_l^{\mathrm{B}}\leftarrow {\tilde{h}}_l^{\mathrm{B}}+{\tilde{\alpha }}^{\mathrm{B}}(|y_j|-{\tilde{h}}_l^{\mathrm{B}}),\end{array}$$

(12)

but where ${\tilde{\alpha }}^{\mathrm{B}}\ll \tilde{\alpha }$. All coefficients $\tilde{\mathit{h}}$ and ${\tilde{\mathit{h}}}^{\mathrm{B}}$ are initialized with ${\tilde{h}}_{\mathrm{init}}$ (see

Table 1. The parameters with which the experimental results presented in Section 6 were obtained. Note that in the dataset only $N_{\mathrm{s}}=31$ CIR samples were logged, i.e., ${\tau }_{\mathrm{end}}-{\tau }_{\mathrm{FP}}=27 \mathrm{n}\mathrm{s}$. In addition, note that the step size to update the background variance metric is set to ${\tilde{\alpha }}^{\mathrm{B}}=0.1$ for the first thousand measurements, before being set to ${\tilde{\alpha }}^{\mathrm{B}}=0.001$.

${\mathit{\tau }}_{\mathbf{FP}}-{\mathit{\tau }}_{\mathbf{start}}$	${\mathit{\tau }}_{\mathbf{end}}-{\mathit{\tau }}_{\mathbf{FP}}$	m	$\mathit{\Delta }{\mathit{\tau }}_{\mathbf{knots}}$	${\mathit{N}}_{\mathbf{knots}}$	${\mathit{N}}_{\mathbf{s}}$	$\mathit{\alpha }$	$\tilde{\mathit{\alpha }}$
4 $\mathrm{n}$$\mathrm{s}$	36 $\mathrm{n}$$\mathrm{s}$	4	$0.25$$\mathrm{n}$$\mathrm{s}$	161	40	0.05	0.1
${\tilde{\mathit{\alpha }}}^{\mathrm{B}}$	${\tilde{\mathit{h}}}_{\mathrm{init}}$	${\mathit{N}}_{\mathrm{win}}$	${\mathit{N}}_{\mathrm{seg}}$	$\mathit{\beta }$	${\mathit{\sigma }}_{\mathit{\eta }}$	$\mathit{\gamma }$	${\mathit{N}}_{\mathrm{p}}$
0.001	4	8	5	1.3	10 m s−1	1 $\mathrm{n}$$\mathrm{s}$	200

and Section 6.1 for parameter choices).

4.3. Background Subtraction and Leading Edge Detection

With the previously defined variance metrics for the currently measured CIR and the background CIR, regions in the CIR with temporarily higher variance can be detected by subtracting the background variance metric from the current variance metric

$$\begin{array}{c}\hfill \tilde{\mathit{s}}=\tilde{\mathit{h}}-\beta {\tilde{\mathit{h}}}^{\mathrm{B}},\end{array}$$

(13)

where $\beta >1$ is a constant scalar sensitivity factor. Regions in the CIR with higher than usual variances are found in segments ${\tau }_l\le \tau <{\tau }_{l+1}$, where l is such that ${\tilde{s}}_l>0$. The location of the target path ${\tau }_{\mathrm{TP}}$ is measured by finding the first cluster of such segments, i.e.,

$$\begin{array}{c}\hfill {\widehat{\tau }}_{\mathrm{TP}}=min{\tau }_l\mathrm{s}.\mathrm{t}.{\tilde{s}}_l>0\mathrm{and}\sum _{n=l}^{l+N_{\mathrm{win}}}\left[{\tilde{s}}_n>0\right]\ge N_{\mathrm{seg}},\end{array}$$

(14)

with $l\in \{0,1,\cdots ,N_{\mathrm{knots}}-2\}$, $\left[·\right]$ the Iverson bracket converting a logical proposition into zero or one, $N_{\mathrm{win}}$ the window length, and $N_{\mathrm{seg}}$ the number of segments in the window which must have a higher than usual variance. This is visualized in plot (c) of Figure 6.

4.4. Outlier Rejection

An erroneous CIR measurement used to update the recursive filters for the mean of the CIR and variance metrics can corrupt the filtered signals and hence the measured target path location ${\widehat{\tau }}_{\mathrm{TP}}$ for a significant amount of time. Therefore, CIR measurements are rejected if they do not fulfill the following criteria:

• A CIR measurement must be based on a minimal number of accumulated preamble symbols picked as half the number of preamble symbols transmitted (see Table A1 in Appendix B). We observed that a CIR measurement based on a lower number of preamble symbols is much more likely to correspond to an outlier.
• The CIR samples before the estimated first path location ${\tau }_{\mathrm{FP}}$ must be smaller than a multiple of the first knot coefficient $h_0$ which is indicative of the noise floor. A factor of five was used for the experimental results presented later. In case the estimated first path location ${\tau }_{\mathrm{FP}}$ is too late, CIR samples before the estimated first path are likely to be significantly higher than the noise floor.
• The largest sample of the measured CIR must not deviate too much from the maximal knot coefficient. In the experimental results presented, a measurement was rejected when its largest sample was more than five times smaller or two times larger than the maximal knot coefficient.

Many reasons can lead to such erroneous CIR measurements, such as the DWM1000’s leading edge detection algorithm failing to properly detect the first path location ${\tau }_{\mathrm{FP}}$ or packets colliding when two modules transmit at the same time, to name only a few. This outlier rejection strategy proved to be sufficient, but other, more elaborate strategies based on statistics could be devised.

5. Network Sensing

Section 3 briefly described how a moving person can be localized on an ellipse by tracking changes in the channel impulse response of one transmitter and receiver pair. If three or more UWB radio equipped devices track and broadcast such changes in the CIR, any device participating in the broadcasting or only listening to these messages can localize a person by intersecting the ellipses. To this end, it also needs to know the location of these devices.

5.1. Protocol

To enable such a sensing network, we commanded the devices to broadcast messages as given in

Table 2. Content of broadcasted messages.

transmission time (in network time)

device ID

device position/location

device ID of transmitter of last received message

reception time of last received message (in network time)

target path location in CIR obtained with transmitter of last received message

other payload

. By listening to (and by broadcasting) these messages, a device can

• localize itself with respect to the other devices,
• synchronize its clock with respect to the network clock,
• localize a person within the space occupied by the network.

As already mentioned in the Introduction, we only discuss the last point and refer the reader to [44] for more details on the ALOHA protocol with which we scheduled message transmissions, and to [11,45] and the references therein for clock calibration and localization.

5.2. Particle Filter

The measured target path locations ${\widehat{\tau }}_{\mathrm{TP}}$ contained in the network messages given in Table 2 can be fused in multiple ways to obtain the target position estimate ${\widehat{\mathit{p}}}_{\mathrm{T}}$. An analytical way is presented in [21] in which the intersection points of the different ellipses are calculated. Alternatively, the space of interest can be gridded and each cell can be assigned a probability that an object is moving within it (as described in [46]). Instead of spatially fixed cells, the probabilities can also be assigned to moving particles of a particle filter as done in this work (see [47] for an introduction to particle filtering). A random walk is assumed for the particles’ movements, i.e., each particle p’s position ${\mathit{p}}^p=(x^p,y^p)$ is assumed to evolve as

$$\begin{array}{c}\hfill x^p(t+\Delta t)=x^p\left(t\right)+\Delta t·{\eta }_x^p\left(t\right)\end{array}$$

(15)

$$\begin{array}{c}\hfill y^p(t+\Delta t)=y^p\left(t\right)+\Delta t·{\eta }_y^p\left(t\right)\end{array}$$

(16)

where $\Delta t$ is the time period since the last prediction and where ${\eta }_x^p\left(t\right),{\eta }_y^p\left(t\right)$ are drawn from a zero-mean, normal distribution with standard deviation of ${\sigma }_{\eta }$ at each prediction step. When a message is received, the particles’ weights are updated according to the likelihood conditioned on the measurement ${\widehat{\tau }}_{\mathrm{TP}}$, i.e.,

$$\begin{array}{c}\hfill w^p=p\left({\tau }_{\mathrm{TP}}^p\right|{\widehat{\tau }}_{\mathrm{TP}}),\end{array}$$

(17)

where we assume that the measurement error of ${\widehat{\tau }}_{\mathrm{TP}}$ has Cauchy distribution centered at zero with scale parameter $\gamma$. This fat tail distribution was found to better represent the actual measurement error distribution than a normal distribution. The measured target path location ${\widehat{\tau }}_{\mathrm{TP}}$ is contained in the message and the expected ${\tau }_{\mathrm{TP}}^p$ of each particle p is calculated according to Equation (1), i.e.,

$$\begin{array}{c}\hfill {\tau }_{\mathrm{TP}}^p=\frac{1}{c}\left(\Vert {\mathit{p}}_{\mathrm{tx}}-{\mathit{p}}^p\Vert +\Vert {\mathit{p}}_{\mathrm{rx}}-{\mathit{p}}^p\Vert \right).\end{array}$$

(18)

Note that the position of the receiver ${\mathit{p}}_{\mathrm{rx}}$ and transmitter ${\mathit{p}}_{\mathrm{tx}}$ can be extracted from the current message and from previously received messages, respectively. Additionally, with every received message, a CIR measurement is obtained which can also be filtered and used to update the particle filter. After the weights of all $N_{\mathrm{P}}$ particles have been calculated, the particles are resampled to get $N_{\mathrm{P}}$ posterior particles, all with equal weights.

Note that this is a very basic approach. For improved performance, more elaborate measurement fusion approaches could be implemented, as, for example, the tailored particle filter outlined in [48].

6. Experimental Evaluation

In order to develop and test the algorithms before implementing them on the actual UWB radio equipped devices, a dataset of CIR measurements was recorded. The first part of this section discusses the dataset acquisition and tests the algorithms on it. Parameter settings are discussed which are also summarized in Table 1. The second part of this section discusses implementation details and real-time experimental results as also shown in the accompanying videos, see [13] or Supplementary Materials Video S2. In this paper, we focused on a lightweight filtering approach for devices with limited memory and computational power. However, to enable development of non-causal and more elaborate filtering techniques, we make the dataset available here [12].

6.1. Evaluation on CIR Measurements Dataset

Four devices equipped with DWM1000 modules connected to STM32F4 microprocessors were placed in a square with a side length of about 4 $\mathrm{m}$, about $1.2$ $\mathrm{m}$ above the ground, as shown in Figure 4. They were commanded to send messages to their peer modules at random intervals such that each device received messages with an average frequency of 564 $\mathrm{Hz}$. In other words, each device received CIR measurements for each of its three communication channels with an average frequency of 188 $\mathrm{Hz}$. Not the complete CIR, but only N_s $=31$ samples starting three samples before the estimated first path location were read from the DWM1000 register values. These samples were continuously sent via USB to a laptop which also logged ground truth measurements provided by an overhead motion capture system. During the first 17 $\mathrm{s}$, the person inside the space was at rest and then started walking for 87 $\mathrm{s}$. During the last 16 $\mathrm{s}$, the person was at rest again. The settings employed by the DWM1000 are given in Appendix B, Table A1.

When applying the algorithms on this dataset, we look at the performance of the target path location filtering (described in Section 4) and at the performance of the particle filter, fusing the measured target path locations (described in Section 5.2). It is clear that the latter is heavily dependent on the first.

Plot (a) in Figure 7 shows the measured target path location ${\widehat{\tau }}_{\mathrm{TP}}$ (in blue) and the ground truth target path location (in red) as calculated with the data provided by the overhead motion capture system for the transmitter and receiver marked in plot (b) of the figure (as also marked in Figure 4). Plot (b) shows the corresponding target position trajectories provided by the motion capture system (in red) and by the particle filter (in blue), respectively. For better visibility, only a representative 30 $\mathrm{s}$ segment of the complete dataset is shown. Plot (a) also shows where the background subtracted variance metric $\tilde{\mathit{s}}$ (as defined in Equation (13)) is positive (white) and where it is negative (gray). This facilitates understanding the influence of the different parameters. For example, if the parameter $N_{\mathrm{seg}}$ utilized in Equation (14) is chosen too large with respect to $N_{\mathrm{win}}$, movement might not be detected, and, if it is chosen to be too small, noise or outliers might be detected. The sensitivity factor $\beta$ has a similar effect. These parameters are kept constant at $N_{\mathrm{win}}=8$, $N_{\mathrm{seg}}=5$ and $\beta =1.3$ for the complete duration of the experiment. Environmental changes happening during the experiment can be dealt with to a certain extent, as we keep updating the background variance metric. Nevertheless, automatic parameter adjustment similar to the constant false alarm rate detection [49] might further increase performance. In addition, note that the measured target path location ${\widehat{\tau }}_{\mathrm{TP}}$ tends to be smaller than the actual target path location in time periods when it is increasing. This effect can be explained by the recursive filtering approach used to estimate the variance metric of the CIR. Before the background subtracted variance $\tilde{\mathit{s}}$ can again become negative in a certain region, multiple CIR measurements with a low variance in this region need to be received. A higher value of $\tilde{\alpha }$ as given in Equation (11) helps to reduce this settling time, but also makes the algorithm more sensitive to noise. To speed up the initial convergence of the background variation coefficients ${\tilde{\mathit{h}}}^{\mathrm{B}}$, the step size of the background variance metric, ${\tilde{\alpha }}^{\mathrm{B}}$ as given in Equation (12), is initially set to ${\tilde{\alpha }}^{\mathrm{B}}=0.1$ for quick convergence, and then set to ${\tilde{\alpha }}^{\mathrm{B}}=0.001$ after having received a thousand CIR measurements. This number of measurements is received after about 5 $\mathrm{s}$ given the CIR measurement update frequency of 188 $\mathrm{Hz}$ for each communication channel.

The measured target path locations are fused using the particle filter outlined in Section 5.2. The initial position of the $N_{\mathrm{p}}=200$ particles are randomly sampled from the region shown in plot (b) of Figure 7. The scale of the Cauchy distribution assumed for the measurement error is set to $\gamma =1 \mathrm{n}\mathrm{s}$ and the variance of the process noise is set to ${\sigma }_{\eta }=10$m s⁻¹. The positions of the devices are directly given by the motion capture system (see Section 6.2 for a results where the devices self-localize). The estimated position of the particle filter (as shown in blue in plot (b) of Figure 7) is taken as the particles’ mean position. The root-mean-squared error (RMSE) of the particle filter’s position estimate ${\widehat{\mathit{p}}}_{\mathrm{T}}$ was $\mathrm{RMSE}\left({\widehat{\mathit{p}}}_{\mathrm{T}}\right)=0.35 \mathrm{m}$ when discarding the initial segment where the particles have not yet converged.

6.2. Implementation and Real-Time Results

The filtering algorithm discussed in Section 4, as well as the messaging protocol discussed in Section 5.1, were implemented on devices equipped with Decawave’s DWM1000 UWB radio modules and STM32F4 microprocessors (32-bit, 168 MHz, single-precision floating point uint, 196 kB RAM, 1 MB flash), which served as host microcontrollers to the UWB radios. These devices were placed in the experimental area, where they self-localized with respect to each other and where they measured and broadcasted changes in the CIRs.

An additional device only listened to network communication and forwarded it to a laptop running the particle filter described in Section 5.2. The same algorithms and parameters as used to evaluate the dataset in Section 6.1 were applied, with the difference that the particle filter only ran on the messages received by the additional device and used the devices’ positions as calculated from the self-localization algorithm instead of the ones provided by the motion capture system.

The particle filter’s position estimate for two such real-time experiments are shown in Figure 8, once for a network with three, and once for a network with four devices. The corresponding distributions of the error in the measured target path location ${\widehat{\tau }}_{\mathrm{TP}}$ are shown in plot (a) of Figure 9. The Cauchy distribution assumed to update the particle filter with the measured target path location ${\widehat{\tau }}_{\mathrm{TP}}$ is shown in red. The actual distributions which are shown in blue and orange both have a median of $0.8$ $\mathrm{n}$$\mathrm{s}$. In addition, in other experiments, a median in between 0 $\mathrm{n}$$\mathrm{s}$ and 1 $\mathrm{n}$$\mathrm{s}$ was observed, which we assign to the recursive filtering approach as discussed in Section 6.1. The performance of the particle filter could be further improved by centering the Cauchy distribution at around $0.5$ $\mathrm{n}$$\mathrm{s}$, which was however not investigated further. The particle filter’s position estimates for the network with three devices had an RMSE of $\mathrm{RMSE}\left({\widehat{\mathit{p}}}_{\mathrm{T}}\right)=0.42 \mathrm{m}$, a median error of $\mathrm{MEDE}\left({\widehat{\mathit{p}}}_{\mathrm{T}}\right)=0.36 \mathrm{m}$ and a maximum error of $\mathrm{MAXE}\left({\widehat{\mathit{p}}}_{\mathrm{T}}\right)=1.03 \mathrm{m}$ when discarding the initial segment where the particles have not yet converged. For the network with four devices, these values were reduced to $\mathrm{RMSE}\left({\widehat{\mathit{p}}}_{\mathrm{T}}\right)=0.33 \mathrm{m}$, $\mathrm{MEDE}\left({\widehat{\mathit{p}}}_{\mathrm{T}}\right)=0.29 \mathrm{m}$ and $\mathrm{MAXE}\left({\widehat{\mathit{p}}}_{\mathrm{T}}\right)=0.77 \mathrm{m}$. The cumulative error distribution of the two position estimates are shown in plot (b) of Figure 9. In order to calculate these metrics, the coordinate systems found by the devices had to be aligned with the coordinate system of the motion capture system. This was done by calculating the best-fitting rigid transformation that aligned the estimated and ground truth positions of the devices as outlined in [50]. These alignments are shown in Figure 8, where the red ground truth triangles marking the position of the devices are barely visible under the blue triangles representing the estimated positions. The error in the position estimates was on average only $3 \mathrm{c}\mathrm{m}$, and therefore had little impact on the error in the target position estimate.

Footage of such real-time experiments performed in the space shown in Figure 4 and in the environments shown in Figure 10 is found in the accompanying video, see [13] or Supplementary Materials Video S2. There it is visible that, even with plants obstructing the line-of-sight between the modules and a lot of metallic objects in the environment, the system is able to track a person with the background subtraction technique. However, if the metallic objects also obstructed the line-of-sight between the devices, the self-localization accuracy would be impaired. This would also result in an impairment in the target tracking performance, and would have to be circumvented by manually determining the devices’ locations with respect to each other.

7. Conclusions

This concept paper demonstrated that devices equipped with off-the-shelf DWM1000 UWB modules can form a multi-static radar network and track a moving person in their surroundings. The algorithms are deliberately kept simple and lightweight, such that they can be run on devices with limited computational power. This makes this approach interesting for location-based, IoT applications. However, performance can be improved by refining each of the presented steps. Additionally, the system can be further enhanced by addressing the following limitations.

Limitations and Potential Mitigations

Motion requirement: The proposed system does not store or update the mean of the background, but only the background variance metric. Therefore, persons or objects which are not moving cannot be localized. By also storing the background mean and by performing leading edge detection on the background subtracted mean, this problem could be addressed. However, as we have seen in Figure 3, the mean might not vary much, making this a harder filtering problem when employing the DWM1000 instead of specialized hardware. Not only using the magnitude, but also the phase of the complex CIR envelope could help.

Monitoring range: We did not specify the monitoring range, but only recorded ${\tau }_{\mathrm{end}}-{\tau }_{\mathrm{FP}}=36 \mathrm{n}\mathrm{s}$ of the CIR after the first path location. It is possible to read up to 1 $\mathsf{\mu }$$\mathrm{s}$ (one symbol time) of the CIR from the DW1000 register values [51]. However, the power of a signal reflected from a far away object is low, resulting in a low signal to noise ratio. As this ratio is also heavily-dependent on the environment in which the measurements are made, it is difficult to indicate a meaningful monitoring range.

Three-dimensional setup: We only performed experiments with a two-dimensional setup. However, when four or more UWB radio-equipped devices are not all positioned in the same plane, it is also possible to localize an object in a three-dimensional setup at the intersection point of ellipsoids instead of ellipses as used in two dimensions.

Multiple targets: We only investigated tracking a single person. To track multiple people, the same algorithm to find the target path location within a CIR can be used, but a more evolved fusion algorithm than the simple particle filter outlined in Section 5.2 must be employed. The Cauchy distribution used for the measurement likelihood in this paper assumes a single target and is not suited for multiple targets moving in the scene. Instead, a multi-hypothesis tracking approach similar to the one proposed in [19] could be employed, where each target path location measurement would be assigned to a certain target position hypothesis. Other, particle filter-based, multi-target tracking approaches are discussed [52].

Message Scheduling Protocol: The ALOHA protocol employed allows easy handling of dynamic addition and removal of devices, and easy handling of partially connected networks with overlapping areas of coverage. However, it also leads to packet collisions, which is further discussed in [44]. A time-division multiple access (TDMA) protocol would prevent packet collision and allow for an increased CIR measurement update rate, which could further improve performance.

Power consumption: Contrary to what might be expected, the DWM1000 consumes most power ( $0.5$ Watt) when in receive mode searching for preamble symbols and not when transmitting a message [53]. Therefore, it would be best if the UWB radios transmitted messages only at a high rate when a person is present and would only search for preamble symbols if a message is certain to arrive. A possible approach might be to only sound a single channel between two devices having a large battery and once a consistent target path location has been discovered, to wake up the other devices. Furthermore, with a TDMA protocol instead of an ALOHA protocol, the devices could limit the search for preambles to certain time periods in the TDMA protocol.