Where Freshness Matters in the Control Loop: Mixed Age-of-Information and Event-Based Co-Design for Multi-Loop Networked Control Systems

Abstract

In the design of multi-loop Networked Control Systems (NCSs), wherein each control system is characterized by heterogeneous dynamics and associated with a certain set of timing specifications, appropriate metrics need to be employed for the synthesis of control and networking policies to efficiently respond to the requirements of each control loop. The majority of the design approaches for sampling, scheduling, and control policies include either time-based or event-based metrics to perform pertinent actions in response to the changes of the parameters of interest. We specifically focus in this article on Age-of-Information (AoI) as a recently-developed time-based metric and threshold-based triggering function as a generic Event-Triggered (ET) metric. We consider multiple heterogeneous stochastic linear control systems that close their feedback loops over a shared communication network. We investigate the co-design across the NCS and discuss the pros and cons with the AoI and ET approaches in terms of asymptotic control performance measured by Linear-Quadratic Gaussian (LQG) cost functions. In particular, sampling and scheduling policies combining AoI and stochastic ET metrics are proposed. It is argued that pure AoI functions that generate decision variables solely upon minimizing the average age irrespective of control systems dynamics may not be able to improve the overall NCS performance even compared with purely randomized policies. Our theoretical analysis is validated through several simulation scenarios.

1. Introduction

Networked Control Systems (NCSs) generally refer to multiple dynamical systems controlled by possibly remotely located controllers with information exchange supported by a wired or wireless communication infrastructure. The applications of such systems range from smart energy grids, autonomous driving, and industrial production, to healthcare, agriculture, and smart homes [1,2]. The two main layers of a networked system—control and communication—strongly influence each other and face heterogeneous and time-varying conditions, constraints, and demands [3]. Hence, the efficient design of networked systems requires novel and integrated strategies that are responsive to the heterogeneity of the control systems and the real-time variations of individual layers, and at the same time possess flexibility and scalability [4,5,6].

Considering state-of-the-art communication technology, there is a need for novel approaches to the modeling, analysis, and design of network protocols and control mechanisms capable of jointly supporting information exchange required to make decisions at the right component and at the right time. This is the basic motivation behind employing appropriate utility functions to coordinate the process of data exchange in a network of many dynamical users. Over the last two decades, there have been many attempts from the control and the communication communities to develop, evaluate, and improve such utility functions compared to the conventional fixed-period and randomized data coordination approaches. Notions such as Value-of-Information (VoI) [7,8], Age-of-Information (AoI) [9,10], and Event-Triggered (ET) [11,12], are metrics that have been separately shown to be capable of coordinating information distribution, taking into account the integrated and coupled context of NCSs. Traditionally, however, two rather distinct paths on addressing the NCS design have been followed: From the communication perspective, the focus mainly results in the design approaches that maximize the network throughput or minimize the end-to-end latency and jitter often ignoring the dynamics, requirements, and characteristics of the sending and receiving entities and the specific data that are being transmitted [13,14,15]. From the control perspective, on the other hand, the major goal has been to maximize Quality-of-Control (QoC), and the communication network is usually abstracted as one or more maximum-rate and delay-negligible transmission channels with enough computation and functional capability to resolve contentions [16,17]. Hence, to fill this research void, it is essential to develop systematic and applicable co-design principles for NCSs that bring both QoC and QoS together by studying novel architectures that take into account the requirements, limitations, and tolerances of both network and control systems.

1.1. Contributions

In this article, the goal is to propose an efficient co-design architecture for heterogeneous NCSs where the influence of both control and network systems is taken into account. Specifically, we study a sampling-scheduling-control co-design problem for stochastic NCSs comprised of multiple heterogeneous Linear Time-Invariant (LTI) control systems. The sampling and control units reside at the control system layer and are designed distributedly, i.e., they are locally installed in every control loop and generate decision variables for their corresponding local control systems. The scheduling unit resides at the network layer and arbitrates the channel access in a centralized fashion, i.e., a unique scheduler coordinates the allocation process of the limited resources among the control loops to avoid contention and consequently data loss. We consider a realistic communication model in that the data packets that are not scheduled for immediate transmissions, if not updated by a newer data sample, are stored in a buffer for possible transmissions in future time instances. If a current sample is not successfully transmitted due to resource limitations, it is not discarded and remains in the buffer to be either replaced by a newer sample or transmitted with some delay whenever the communication resource is assigned to it. Therefore, end-to-end delay in our formulation is comprised of an inter-sampling duration induced by the local samplers and a network-induced delay due to the resource limitations. The performance of each local control system is asymptotically measured by the local Linear-Quadratic Gaussian (LQG) cost function, and the overall asymptotic NCS performance is determined by the average sum of their local LQG costs. Note that the performance is influenced by the resource constraints and the end-to-end transmission delays.

Motivated by the existing results for the design of control and communication systems, in this article, we focus on two celebrated notions of utility metrics: AoI- and ET-based functions. We first discuss if these two design concepts may properly co-exist in a networked control scenario and study where each of them excels in terms of decision-making efficiency. We evaluate them based on two crucial aspects: first, which class of policies result in lower local and overall cost values, and second, how much information is required for a policy maker to generate appropriate decisions. The first one, as explained earlier, is evaluated based on asymptotic LQG cost functions, while the second is basically judged based on a policy maker needing less information, and distributed parts of the networked system may not be willing to disclose too much information. Therefore, a desirable and applicable co-design architecture would result in sampling, scheduling, and control decisions that jointly induce low local and overall control costs, while they require local or partially accessible information to generate their assigned decision variables at the expense of a viable level of computational complexity.

Under some mild assumptions on the information structures of the policy makers, we first show that the optimal control policy can be obtained independently of the sampling and scheduling policies. In fact, we show that the optimal controllers are of the Certainty Equivalence (CE) form, which technically means the optimal control inputs are identical as they would be obtained in the absence of the additive stochastic disturbances. This is really helpful as it provides a decomposition opportunity for the cross-layer co-design in the sense that the control law remains fixed for a variety of sampling and scheduling policies within the specified classes that satisfy those assumptions on their information structures. We then propose a joint sampling-scheduling co-design where the local samplers are ET and the centralized scheduler uses AoI-based prioritization for resource management. Considering the asymptotic average LQG cost function as the overall NCS performance metric, we show that the ET function is indeed a more efficient candidate for sampling, compared with its AoI counterpart, in the sense of the asymptotic average sum of LQG functions, while AoI performs efficiently for governing the resource allocation process. We compare the performance of the AoI-based scheduling design with conventional random access resource scheduling and show that the AoI scheduling has the design flexibility to be appropriately adjusted to outperform the randomized access policy.

To the best of our knowledge, there is no result available in the literature that considers the co-design of control and communication systems with joint ET and AoI-based policies and compares their joint performance, although both policies have separately been studied extensively from both the control and communication perspectives.

1.2. Related Works

Since the seminal work [18], many results have shown that the event-based approach outperforms the conventional time-triggered and periodic schemes in the sense that it is capable of achieving the same control performance with significantly less usage of computation and communication resources [19,20,21,22]. The event-based approach is also widely studied in the context of NCSs [23,24,25,26], and it is shown that the event-based functions can be employed to efficiently govern the information sampling and scheduling processes taking into account not only the control requirements, but also the communication conditions such as resource scarcity and channel properties [27,28,29,30].

Many researchers have demonstrated that ET policies preserve the stability of NCSs despite updating the controllers less often. In [31], the ${\mathcal{L}}_2$ stability of ET output feedback control was shown in the presence of network-induced delay. The stability of stochastic ET NCSs was also extensively studied, employing appropriate stochastic stability notions such as almost-sure and moment stability, with various sources of randomness such as model uncertainty, sensor noise, and erroneous channels [32,33,34,35]. Additionally, event-based Medium Access Control (MAC) and Contention Resolution (CR) protocols for resource-limited or contention-based communication networks have been proposed in the literature, both in the form of centralized and decentralized MAC and CR algorithms [36,37,38,39]. Centralized MAC and CR approaches are shown to be capable of fully resolving contentions, yet at the expense of not being scalable as they require a huge volume of information exchange, while easily deployable decentralized event-based MAC and CR counterparts can substantially decrease contentions, but not fully resolve them.

The design of optimal ET policies for either control and communication systems or cross-layer joint design has been an active area of research. The results suggest that finding global optimal event-based functions is often nontrivial, especially for multi-loop NCSs or more realistic models of communication networks [40,41,42,43]. The major difficulty lies behind the tight couplings and inter-layer dependencies between the distributed time-varying parameters of control and network systems, obliging to search for less computationally complex sub-optimal or approximative solutions. Network-induced delays are regarded as major coupling parameters in ET NCS design that, depending on the model of the sampling and communication network, might possess different dynamic characteristics. In fact, delay affects the states of the control systems, and the states themselves affect the decision outcomes of the event-based policies; those decisions also affect the network-induced delays [44]. Therefore, an optimal co-design needs to keep track of the network-induced delays, which might not be feasible for stochastic networked systems.

The AoI metric, proposed in [45], has emerged to quantify the freshness of the received status updates at the estimator and has attracted significant attention from the communication and networking communities. The AoI is defined as the time elapsed since the generation of the latest successfully received status update at the estimator. Several authors have studied the problem of minimizing some functions of AoI under different queuing and communication models [46,47,48,49,50]. While the works in [46,47,48] considered time averaged AoI, the authors in [49] considered minimizing the tail of the AoI, and the authors in [50] considered any non-decreasing and measurable function of AoI. Apart from studying the effects of communication scheduling on AoI, none of the above works considered estimation or control objectives for networked systems. Nonetheless, the general consensus is that a lower AoI in an NCS may result in a lower estimation and control cost, because having access to fresher state information often improves the performance. However, only a handful of works considered the performance of the solutions proposed for AoI with respect to such costs. The authors in [51] studied the minimum mean squared error problem with independent and identically distributed (i.i.d.) transmission delays for Wiener process estimation. They showed that the estimation error is a function of AoI if the sampling decisions are independent of the observed Wiener process; otherwise, the estimation error is not a function of AoI. In [52], we studied a state estimator of a single-loop stochastic LTI system with i.i.d. transmission delays and derived the relation between AoI and the estimation error, assuming that the sampling decisions are independent of the observed states.

There has been an increasing interest recently from the control community to consider AoI utility functions due to their simpler evolution and characteristics compared to ET or VoI metrics. Despite some progress, however, there exist results suggesting that AoI-based approaches with the original linear formulation of AoI may not be sensitive enough to dynamic changes of control systems and their QoC requirements [53,54]. In [55], various nonlinear functions of AoI were considered to be minimized instead of the conventional average linear AoI, and it was shown that these variations of AoI utility functions can be beneficial to improve the control performance. The authors in [56] showed in a recent work that a discounted AoI-dependent monotonic function can be employed to optimally govern wireless network scheduling to maximize control performance over an infinite horizon. Despite recent efforts reflected in the literature, there are still many challenges. Specifically, there is no result, to the best of our knowledge, on combined ET and AoI-based co-design across control systems and communication network layers.

1.3. Outline

In the remainder of this article, the NCS model and the problem statement are described in Section 2. The co-design architecture with CE controllers and sampling and scheduling policies is presented in Section 3. Performance analysis and comparisons with other co-design architectures are presented in Section 4. Simulation results are demonstrated in Section 5, and the concluding remarks are summarized in Section 6.

1.4. Notations

We denote the expectation, conditional expectation, conditional probability, transpose, and trace operators by $\mathsf{E}[·]$, $\mathsf{E}[·|·]$, $\mathsf{P}[·|·]$, ${[·]}^{\top }$, and $tr(·)$, respectively. A multivariate Gaussian distributed random vector X with mean vector $\mu$ and covariance matrix $W\succ 0$ is denoted by $X\sim \mathcal{N}(\mu ,W)$, where $A\succ B$ denotes that $A-B$ is positive definite. The Q-weighted squared 2-norm of a column vector X is denoted by ${\parallel X\parallel }_Q^2\triangleq X^{\top }QX$ and ${\parallel X\parallel }_2^2\triangleq X^{\top }X$. A time-varying column vector $X_t^i$ includes an array of variables belonging to the sub-system indexed by i at time t, while we define $X_{[t_1,t_2]}^i\triangleq \{X_{t_1}^i,X_{t_1+1}^i,\dots ,X_{t_2-1}^i,X_{t_2}^i\}$ and $X^i\triangleq \{X_0^i,X_1^i,\dots \}$. For constant matrices, a subscript indicates the corresponding sub-system, and a superscript denotes matrix power. An optimal decision variable/policy X is represented by $X^{*}$. The sets of natural, real, non-negative integer, and non-negative real numbers are denoted by $\mathbb{N}$, $\mathbb{R}$, ${\mathbb{N}}_0$, and ${\mathbb{R}}_{\ge 0}$, respectively. For an n-by-m-dimensional real space, we use the notation ${\mathbb{R}}^{n\times m}$.

2. NCS Model and Problem Description

2.1. NCS Model

We consider an NCS consisting of N heterogeneous stochastic LTI controlled dynamical processes that synchronously exchange their sensory information with their corresponding controllers via a common resource-limited communication network; see Figure 1. Each process $i\in \mathrm{N}\triangleq \{1,\dots ,N\}$ is comprised of a plant ${\mathcal{P}}_i$, a noisy sensor ${\mathcal{S}}_i$, and a feedback control unit including a feedback controller ${\mathcal{C}}_i$ and an estimator ${\mathcal{E}}_i$. Each process $i\in \mathrm{N}$ is described as follows:

$$\begin{array}{cc}\hfill x_{k+1}^i& =A_i{x}_k^i+B_i{u}_k^i+w_k^i,\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill y_k^i& =x_k^i+v_k^i,\hfill \end{array}$$

(2)

where $x_k^i\in {\mathbb{R}}^{n^i}$, $u_k^i\in {\mathbb{R}}^{m^i}$, and $y_k^i\in {\mathbb{R}}^{n^i}$ represent the state vector, control input, and sensor measurement of the process i at a time-step $k\in {\mathbb{N}}_0$, respectively. Constant matrices $A_i\in {\mathbb{R}}^{n^i\times n^i}$ and $B_i\in {\mathbb{R}}^{n^i\times m^i}$ describe the system matrix and input matrix, respectively, and we assume that each pair $(A_i,B_i)$ is controllable. To allow for heterogeneity, the $A_i$ and $B_i$ matrices may differ for different processes and may also adopt different dimensions. The random processes $w_k^i\in {\mathbb{R}}^{n^i}$ and $v_k^i\in {\mathbb{R}}^{n^i}$ are, respectively, the exogenous disturbance acting on the process dynamics and the measurement noise. They are assumed to be Gaussian distributed independent random sequences with mutually i.i.d. realizations $w_k^i\sim \mathcal{N}(0,{\Sigma }_{w^i})$ and $v_k^i\sim \mathcal{N}(0,{\Sigma }_{v^i})$, $\forall k$ and $i\in \mathrm{N}$, where ${\Sigma }_{w^i}\succ 0$ and ${\Sigma }_{v^i}\succ 0$. The initial states $x_0^i$’s, $i\in \mathrm{N}$, are also presumed to be randomly selected from an arbitrary finite-moment distribution with mean ${\mu }_{x_0^i}$ and variance ${\Sigma }_{x_0^i}\succ 0$.

At every time-step k, the decision on whether the state measurement $y_k^i$ of sub-system i is sent for transmission is made by a local sampler ${\mathcal{S}}_i$ located at the sensor station. The sampling decision is assumed to be the outcome of a local sampling policy ${\xi }_i:{\mathcal{I}}_k^i\mapsto \{0,1\}$, where ${\mathcal{I}}_k^i$ represents the information available at ${\mathcal{S}}_i$ at time-step k and will be formally defined later. The sampling decision outcome, denoted by the binary-valued variable ${\delta }_k^i$, is as follows:

$${\delta }_k^i={\xi }_i\left({\mathcal{I}}_k^i\right)=\left\{\begin{array}{cc}1,\hfill & y_k^i\mathrm{sent\; to\; the\; network\; for}\mathrm{transmission},\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(3)

At every time-step k, those sub-systems that locally decided to update their corresponding controllers will forward their sensor measurements to the communication network. We assume that the communication network has capacity limitations such that not all N sub-systems can simultaneously close their sensor-to-controller links at a time instance, i.e., if the network capacity at every single time-step is denoted by the constant $c\in \mathbb{N}$, the following resource constraint holds:

$$1\le c<N,\forall k\in {\mathbb{N}}_0.$$

(4)

The communication network is assumed to consist of a queue to store the received data packets and a scheduling unit that decides which data packets are to be transmitted at each time-step. It should be mentioned that transmissions of data from sensors to the buffer and from the buffer to the controllers are not subject to communication delay, i.e., if the sampler or scheduler decides on a sample being sent to the buffer or a buffered data sent to the controller, the transmissions are completed instantaneously. The scheduling decision at every time-step k is assumed to be the outcome of a centralized resource allocation policy $\pi :{\mathcal{I}}_k^s\mapsto \{0,1\}\times \dots \times \{0,1\}={\{0,1\}}^c$, where ${\mathcal{I}}_k^s$ denotes the information available at the network scheduling unit at time-step k which will be formally defined later, and c is the constant capacity constraint. The scheduling decision associated with sub-system i at time-step k is denoted by the binary variable ${\varphi }_k^i$ and is defined as:

$${\varphi }_k^i=\pi \left({\mathcal{I}}_k^s\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{send\; the\; latest\; measurement\; of}\mathrm{sub-system}i\mathrm{in\; the\; buffer\; to}{\mathcal{E}}_i,\hfill \\ 0,\hfill & \mathrm{send\; nothing\; from}\mathrm{sub-system}i\mathrm{to}{\mathcal{E}}_i.\hfill \end{array}\right.$$

(5)

The network queue buffers at most one data packet from each sub-system at every time instance. Hence, in case a new measurement belonging to a certain sub-system arrives at the queue, the fresher data packet replaces the formerly buffered data of that sub-system. The older data packet will be discarded. Therefore, for each sub-system, there is either no buffered data packet in the queue or there is one that is the latest measurement sent to the network by the local sampler. This means even the freshest data packet of a sub-system in the queue might contain the measurement that corresponds to a previous time-step.

When the bandwidth is assigned to a certain sub-system, its freshest measurement in the queue will be forwarded to the corresponding control unit. The received state measurement by the control unit of a sub-system i at a time-step k, denoted by $z_k^i$, might belong to a previous time $\overline{k}<k$ due to the communication delay imposed by the scheduling unit. Therefore, $z_k^i$ is determined as a function of the scheduling variable, as discussed in the following. Before that, we define the notion of AoI at the control unit in our NCS model as follows:

Definition 1.

AoI at the control side of a sub-system $i\in \mathrm{N}$, at time-step $k\in {\mathbb{N}}_0$, is defined as ${\Delta }_k^i=k-{\overline{k}}^i$, where $y_{{\overline{k}}^i}^i$ is the latest received measurement by the estimator ${\mathcal{E}}_i$ up to time k, which confirms ${\delta }_{{\overline{k}}^i}^i=1$.

Assume that at a time-step k, $y_{{\overline{k}}^i}^i=y_{k-{\Delta }_k^i}^i$ is the freshest measurement of sub-system i in the queue, which ensures ${\delta }_{k-{\Delta }_k^i}^i=1$ and ${\delta }_{k-{\Delta }_k^i+1}^i=\dots ={\delta }_k^i=0$, because otherwise, $y_{k-{\Delta }_k^i}^i$ would have been replaced by a fresher measurement. In addition, this confirms that ${\varphi }_{k-{\Delta }_k^i}^i={\varphi }_{k-{\Delta }_k^i+1}^i=\dots ={\varphi }_{k-1}^i=0$, since otherwise, no data belonging to sub-system i would be in the queue at time-step k. To conveniently denote this, we use the notation ${\varphi }_k^i(k-{\Delta }_k^i)=1$ to express the time index of the freshest buffered measurement belonging to sub-system i at time-step k that is scheduled to be transmitted to the estimator ${\mathcal{E}}_i$. Hence, by ${\varphi }_k^i(k-{\Delta }_k^i)=1$, we denote that $y_{k-{\Delta }_k^i}^i$ will be received by ${\mathcal{E}}_i$ at k. If no measurement of sub-system i is scheduled to be transmitted at k, we simply write ${\varphi }_k^i=0$. With this notation, we declare two essential aspects of the information structure: (1) if a sample is scheduled for transmission, then the estimator knows which time instance the received measurement belongs to, and (2) receiving no measurement update might correspond to having no measurement sample of sub-system i in the queue and not necessarily to resource limitations. It should be noted that if there is no data belonging to a sub-system i buffered at a time-step k, then we certainly have ${\varphi }_k^i=0$. In the other words, if the scheduler decides for ${\varphi }_k^i=1$, then there exists exactly one buffered data packet of sub-system i to be sent to its corresponding control unit. Therefore, ${\varphi }_k^i=0$ might correspond to either having no measurement sample of sub-system i in the buffer to forward or having not enough resources to schedule the available sample at that specific time. In the latter case, $y_{k-{\Delta }_k^i}^i$ remains in the queue to be either serviced in future time-steps or replaced by a fresher sampled measurement. Finally, according to Definition 1, the information update at an estimator ${\mathcal{E}}_i$ can be stated as:

$$z_k^i=\left\{\begin{array}{cc}{y}_{k-{\Delta }_k^i}^i\hfill & \mathrm{if}{\varphi }_k^i(k-{\Delta }_k^i)=1,{\Delta }_k^i\in [0,k],\hfill \\ \mathsf{⌀}\hfill & \mathrm{if}{\varphi }_k^i=0.\hfill \end{array}\right.$$

(6)

Note that the estimator ${\mathcal{E}}_i$ receives the current measurement sample $y_k^i$, only if ${\varphi }_k^i\left(k\right)=1$, which ensures ${\delta }_k^i=1$ and ${\Delta }_k^i=0$. Depending on the information received at the estimator and the state estimate computed, the control input $u_k^i$ is assumed to be generated as the outcome of a causal mapping ${\gamma }_i:{\tilde{\mathcal{I}}}_k^i\mapsto {\mathbb{R}}^{m^i}$, where ${\tilde{\mathcal{I}}}_k^i$ represents the set of available information at the controller and will be formally defined later.

Remark 1.

In the absence of a measurement sample at the control side at a certain time k, i.e., if ${\varphi }_k^i=0$, the estimator ${\mathcal{E}}_i$ may use the information contained in the sampling variable, i.e., knowing the outcome of ${\delta }_k^i$, and incorporate it in computing ${\widehat{x}}_k^i$. This extra knowledge is known as the side-information contained in the sampling variable. In this article, we do not investigate the impact of the side-information when no measurement update is received by an estimator. As we will see later when we introduce the information structures, we assume that the control unit of a sub-system keeps the history of the sampling variables ${\delta }_{[0,k]}^i$, however, it does not incorporate this side-information in computing ${\widehat{x}}_k^i$ in the absence of a measurement sample. Incorporating side-information results in a nonlinear estimator and possibly non-tractable state estimator design problem, especially for threshold-based sampling policies in the presence of resource limitations. We assume that if no update is received at the estimator at some certain time-steps, then the estimator constructs ${\widehat{x}}_k^i$ in a model-based fashion using the previous estimate ${\widehat{x}}_{k-1}^i$.

Depending on the sampling and scheduling decision variables $\{{\delta }_0^i,\dots ,{\delta }_k^i\}$ and $\{{\varphi }_0^i,\dots ,{\varphi }_k^i\}$, we can derive the dynamics of the AoI at the estimator ${\mathcal{E}}_i$. It is straightforward to derive the dynamics of ${\Delta }_k^i$, as functions of the sampling and scheduling variables:

$$\begin{array}{cc}\hfill {\Delta }_k^i& =\sum _{t=1}^{k-r}\prod _{l=t}^{k-r}(1-{\delta }_l^i)+r,r=\sum _{t=1}^k\prod _{l=t}^k(1-{\varphi }_l^i).\hfill \end{array}$$

(7)

It can be seen from Equation (7) that the AoI at the estimator depends on both sampling and scheduling decision outcomes.

Having the outcomes of the sampling and scheduling policies determined in Equations (3) and (5), we can introduce the information sets ${\mathcal{I}}_k^i$ and ${\mathcal{I}}_k^s$, available, respectively, for the local sampler of sub-system i and the centralized scheduling unit, as follows:

$$\begin{array}{cc}\hfill {\mathcal{I}}_k^i& =\{{\mathcal{I}}_{\mathrm{prim}}^i,{\delta }_0^i,\dots ,{\delta }_{k-1}^i,{\varphi }_0^i,\dots ,{\varphi }_{k-1}^i,z_0^i,\dots ,z_{k-1}^i\},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\mathcal{I}}_k^s& ={\cup }_{i\in \mathrm{N}}\{{\mathcal{I}}_{\mathrm{prim}}^i,{\mathrm{N}}_0^b,\dots ,{\mathrm{N}}_k^b,{\delta }_0^i,\dots ,{\delta }_k^i,{\varphi }_0^i,\dots ,{\varphi }_{k-1}^i,z_0^i,\dots ,z_{k-1}^i\},\hfill \end{array}$$

(9)

where ${\mathcal{I}}_{\mathrm{prim}}^i\triangleq \{A_i,B_i,{\Sigma }_{w^i},{\Sigma }_{v^i},{\mu }_{x_0^i},{\Sigma }_{x_0^i}\}$, and ${\mathrm{N}}_k^b$ denotes the set of buffered state measurements at time-step k. Additionally, we introduce the set of available information for the estimator and controller of sub-system i at time-step k:

$${\tilde{\mathcal{I}}}_k^i={\mathcal{I}}_k^i\cup \{{\delta }_k^i,{\varphi }_k^i,z_k^i\}\cup \{u_0^i,\dots ,u_{k-1}^i\}=\{{\mathcal{I}}_{\mathrm{prim}}^i,u_0^i,\dots ,u_{k-1}^i,{\delta }_0^i,\dots ,{\delta }_k^i,{\varphi }_0^i,\dots ,{\varphi }_k^i,z_0^i,\dots ,z_k^i\}.$$

(10)

Note that, with the information about sampling and scheduling variables in Equations (8)–(10) and the expression for the AoI in Equation (7), the sampler ${\mathcal{S}}_i$ is aware of the sequence ${\Delta }_{[0,k-1]}^i$, the controller ${\mathcal{C}}_i$ is aware of ${\Delta }_{[0,k]}^i$, and the centralized sampler has the knowledge of ${\cup }_{i\in \mathrm{N}}\left\{{\Delta }_{[0,k-1]}^i\right\}$.

Having the information set ${\tilde{\mathcal{I}}}_k^i$ introduced, we can construct the state estimate and compute the estimation error at the estimator of sub-system i. We denote the state estimate at the estimator of sub-system i at time-step k by $\mathsf{E}\left[x_k^i\right|{\tilde{\mathcal{I}}}_k^i]$ and define the corresponding estimation error as:

$$\begin{array}{cc}\hfill {\tilde{e}}_k^i& =y_k^i-\mathsf{E}\left[x_k^i\right|{\tilde{\mathcal{I}}}_k^i].\hfill \end{array}$$

(11)

The dynamics of the estimation error ${\tilde{e}}_k^i$ can be obtained as:

$$\begin{array}{cc}\hfill {\tilde{e}}_k^i& =y_k^i-\mathsf{E}\left[x_k^i\right|{\tilde{\mathcal{I}}}_k^i]=A_i{x}_{k-1}^i+B_i{u}_{k-1}^i+w_{k-1}^i+v_k^i-\mathsf{E}[A_i{x}_{k-1}^i+B_i{u}_{k-1}^i+w_{k-1}^i|{\tilde{\mathcal{I}}}_k^i]\hfill \\ & =A_i(x_{k-1}^i-\mathsf{E}\left[x_{k-1}^i\right|{\tilde{\mathcal{I}}}_k^i])+w_{k-1}^i+v_k^i=A_i({\tilde{e}}_{k-1}^i-v_{k-1}^i)+v_k^i+w_{k-1}^i.\hfill \end{array}$$

(12)

Note that we can write $\mathsf{E}\left[x_{k-1}^i\right|{\tilde{\mathcal{I}}}_k^i]=\mathsf{E}\left[x_{k-1}^i\right|{\tilde{\mathcal{I}}}_{k-1}^i\cup \{{\delta }_k^i,{\varphi }_k^i,z_k^i,u_{k-1}^i\}]$. Since the evolution of $x_{k-1}^i$ is independent of the parameters ${\delta }_k^i,{\varphi }_k^i,z_k^i,u_{k-1}^i$, we then have $\mathsf{E}\left[x_{k-1}^i\right|{\tilde{\mathcal{I}}}_k^i]=\mathsf{E}\left[x_{k-1}^i\right|{\tilde{\mathcal{I}}}_{k-1}^i]$, which confirms Equation (12). Assume now that the decision variables ${\delta }_k^i$ and ${\varphi }_k^i$ are generated and $y_{k-{\Delta }_k^i}^i$, for any arbitrary ${\Delta }_k^i\in [{\Delta }_{k-1}^i+1,k]$, is the latest received state measurement by the estimator ${\mathcal{E}}_i$ at time-step k, i.e., ${\varphi }_k^i(k-{\Delta }_k^i)=1$. Note that the realization of ${\Delta }_k^i$ is determined by the sampling and scheduling variables ${\delta }_{[{\Delta }_{k-1}^i,k]}^i$ and ${\varphi }_{[{\Delta }_{k-1}^i,k]}^i$. We can compute the state estimate as:

$$\begin{array}{cc}& \mathsf{E}\left[x_k^i\right|{\tilde{\mathcal{I}}}_k^i]=\mathsf{E}[A_i^{{\Delta }_k^i}{x}_{k-{\Delta }_k^i}^i+A_i^{{\Delta }_k^i-1}{B}_i{u}_{k-{\Delta }_k^i}^i+\dots +A_i{B}_i{u}_{k-2}^i+B_i{u}_{k-1}^i+A_i^{{\Delta }_k^i-1}{w}_{k-{\Delta }_k^i}^i+\dots +w_{k-1}^i|{\tilde{\mathcal{I}}}_k^i]\hfill \\ & =A_i^{{\Delta }_k^i}\mathsf{E}\left[x_{k-{\Delta }_k^i}^i\right|y_{k-{\Delta }_k^i}^i]+A_i^{{\Delta }_k^i-1}{B}_i{u}_{k-{\Delta }_k^i}^i+\dots +A_i{B}_i{u}_{k-2}^i+B_i{u}_{k-1}^i,\hfill \end{array}$$

where $\mathsf{E}\left[x_{k-{\Delta }_k^i}^i\right|y_{k-{\Delta }_k^i}^i]$ is the minimum mean-square estimate (MMSE) computed by a Kalman filter at the estimator side ${\mathcal{E}}_i$ given the received measurement $y_{k-{\Delta }_k^i}^i$, with the standard Kalman filter equations for a time t at which the measurement sample $y_t^i$ is available as:

$$\begin{array}{cc}\hfill \mathsf{E}\left[x_t^i\right|y_t^i]& ={\widehat{x}}_t^{i^{-}}+K_t^i\left(y_t^i-{\widehat{x}}_t^{i^{-}}\right),\hfill \\ \hfill {\widehat{x}}_t^{i^{-}}& =A_i\mathsf{E}\left[x_{t-1}^i\right|{\mathcal{I}}_{t-1}^i]+B_i{u}_{t-1}^i,\hfill \\ \hfill K_t^i& =P_t^{i^{-}}{\left(P_t^{i^{-}}+{\Sigma }_{v^i}\right)}^{-1},\hfill \\ \hfill P_t^{i^{-}}& =\mathsf{E}\left[\left(x_t^i-{\widehat{x}}_t^{i^{-}}\right){\left(x_t^i-{\widehat{x}}_t^{i^{-}}\right)}^{\top }\right]=A_i{P}_{t-1}^{i^{-}}{A}_i^{\top }+{\Sigma }_{w^i},\hfill \\ \hfill P_t^i& =\mathsf{E}\left[\left(x_t^i-\mathsf{E}\left[x_t^i\right|y_t^i]\right){\left(x_t^i-\mathsf{E}\left[x_t^i\right|y_t^i]\right)}^{\top }\right]=P_t^{i^{-}}-K_t^i\left(P_t^{i^{-}}+{\Sigma }_{v^i}\right)K_t^{i^{\top }},\hfill \end{array}$$

where $P_t^{i^{-}}$ and $P_t^i$ denote, respectively, the a priori and the a posteriori estimation error covariances. Therefore, from Equation (11) and using the equivalent expression:

$$y_k^i=A_i^{{\Delta }_k^i}{x}_{k-{\Delta }_k^i}^i+A_i^{{\Delta }_k^i-1}{B}_i{u}_{k-{\Delta }_k^i}^i+\dots +A_i{B}_i{u}_{k-2}^i+B_i{u}_{k-1}^i+A_i^{{\Delta }_k^i-1}{w}_{k-{\Delta }_k^i}^i+\dots +w_{k-1}^i+v_k^i,$$

we conclude that:

$$\begin{array}{cc}\hfill \{{\tilde{e}}_k^i& |{\varphi }_k^i(k-{\Delta }_k^i)=1\}\hfill \\ & =A_i^{{\Delta }_k^i}\left(x_{k-{\Delta }_k^i}^i-\mathsf{E}\left[x_{k-{\Delta }_k^i}^i\right|y_{k-{\Delta }_k^i}^i]\right)+A_i^{{\Delta }_k^i-1}{w}_{k-{\Delta }_k^i}^i+\dots +w_{k-1}^i+v_k^i\hfill \\ & =A_i^{{\Delta }_k^i}\left({\tilde{e}}_{k-{\Delta }_k^i}^i-v_{k-{\Delta }_k^i}^i\right)+v_k^i+\sum _{r=1}^{{\Delta }_k^i}{A}_i^{r-1}{w}_{k-r}^i.\hfill \end{array}$$

(13)

where ${\tilde{e}}_{k-{\Delta }_k^i}^i$ is the MMSE error due to having access to $y_{k-{\Delta }_k^i}^i$. Otherwise, if ${\varphi }_k^i=0$, we use the model-based estimation error as in Equation (12), wherein ${\tilde{e}}_{k-1}^i$ is not necessarily the MMSE error.

2.2. Problem Description

As discussed above, the time for generating a measurement sample and injecting it into the queue is determined by the sampler, while the time for delivering that generated sample, if not discarded due to the arrival of a new sample, to the corresponding controller is determined by the network scheduler. Hence, the source-to-destination delay, i.e., the gap between the current time until the time a generated sample is received by the controller, depends on how the local samplers and the centralized scheduler policies are designed. The problem we tackle in this article is the co-design of sampling, scheduling, and control policies $\{{\xi }_i,\pi ,{\gamma }_i\}$. We discuss the optimal control policy, then consider ET and AoI-based policies for the design of sampling and scheduling policies, and study the effects of the combined architecture on the control performance, which is correlated with the end-to-end delay. Performance comparisons are made according to the LQG index functions as the asymptotic cost metrics for each local sub-system, denoted by $J_i$:

$$J_i=\underset{T\to \infty }{lim}\frac{1}{T}\mathsf{E}\left[x_T^{i^{\top }}{Q}_i^2{x}_T^i+\sum _{k=0}^{T-1}{x}_k^{i^{\top }}{Q}_i^1{x}_k^i+u_k^{i^{\top }}{R}_i{u}_k^i\right],$$

(14)

where $Q_i^1,Q_i^2⪰0$ and $R_i\succ 0$ are, respectively, the state and control input weight matrices of appropriate dimensions, and we assume each pair $(A_i,\sqrt{Q_i^1})$ is detectable, $\forall i\in \mathrm{N}$. The overall asymptotic NCS performance is measured by the average cost:

$$J=\frac{1}{N}\sum _{i=1}^N{J}_i.$$

(15)

3. NCS Design

In this section, we first study the structural properties of the feedback controllers ${\mathcal{C}}_i$, $i\in \mathrm{N}$, and show that the local control law ${\gamma }_i\left({\tilde{\mathcal{I}}}_k^i\right)$ can be designed separately from the local sampling law ${\xi }_i\left({\mathcal{I}}_k^i\right)$ and the scheduling law $\pi \left({\mathcal{I}}_k^s\right)$. Afterwards, we discuss the combined design of the local sampling law and the network scheduling law and discuss which class of ET or AoI-based policies matches the corresponding decision maker.

3.1. CE Control Law

Let us first make a crucial assumption about the sampling policy ${\xi }_i\left({\mathcal{I}}_k^i\right)$:

Assumption 1.

The local sampling policies ${\xi }_i\left({\mathcal{I}}_k^i\right)$’s are selected from the classes of control input-independent sampling policies, i.e., the decision variables ${\delta }_k^i$, $i\in \mathrm{N}$, are computed independently of the sequence of control inputs $\{u_0^i,\dots ,u_{k-1}^i\}$.

Assumption 1 does not result in a loss of generality w.r.t. the introduced information structure at the sampler (see Equation (8)) that indicated that ${\mathcal{I}}_k^i$ does not contain any knowledge of control inputs $\{u_0^i,\dots ,u_{k-1}^i\}$. This is crucial for the derivation of the optimal control policies, as will be discussed in Theorem 1.

Theorem 1.

Consider an NCS as described in Equations (1)–(6), where each control system is steered at every time-step $k\in {\mathbb{N}}_0$ by a local sampler ${\xi }_i\left({\mathcal{I}}_k^i\right)$ and a local plant controller ${\gamma }_i\left({\tilde{\mathcal{I}}}_k^i\right)$ with ${\mathcal{I}}_k^i$ and ${\tilde{\mathcal{I}}}_k^i$ given in Equations (8) and (10), respectively. If the local sampling policies are selected according to Assumption (1), then the optimal control policy in the sense of LQG given in Equation (14) is CE, i.e.,

$${\gamma }_i^{*}\left({\tilde{\mathcal{I}}}_k^i\right)=L_k^i\mathsf{E}\left[x_k^i|{\tilde{\mathcal{I}}}_k^i\right],$$

(16)

where $L_k^i=-{\left(R_i+B_i^{\top }{P}_{k+1}^i{B}_i\right)}^{-1}{B}_i^{\top }{P}_{k+1}^i{A}_i$ is the optimal state feedback control gain.

Proof. 

See Appendix A. □

Remark 2.

Showing that the optimal control law exists over the time horizon $[0,T]$, we can take the limit as $T\to \infty$, which results in having the asymptotic control gain $L_{\infty }^i=-{\left(R_i+B_i^{\top }{P}_{\infty }^i{B}_i\right)}^{-1}{B}_i^{\top }{P}_{\infty }^i{A}_i$, with $P_{\infty }^i={lim}_{k\to \infty }{P}_k^i$ being the asymptotic a posteriori estimation error covariance. We later show in Section 4.2 that, under appropriate sampling/scheduling co-design, $\forall i\in \mathrm{N},P_{\infty }^i$ indeed exists and is not unbounded.

Remark 3.

The result of Theorem 1 is in accordance with the existing results on the separation of control and sampling policies w.r.t. the LQG cost function, if the sampling law is independent of the control inputs. In fact, it was discussed in [22,57] that in the presence of control input-dependent sampling policies, the separation between the sampling and control policies cannot generally be achieved. As is shown in Equations (18) and (19), the estimation error evolves independently of the control inputs; therefore, the sampling policies are allowed to be a function of the estimation error without violating the results of Theorem 1.

Remark 4.

Theorem 1 states that the optimal control law is of the CE form; however, the optimal control inputs $u_k^{i,\ast }$ are still computed based on the state estimate $\mathsf{E}\left[x_k^i|{\tilde{\mathcal{I}}}_k^i\right]$. As shown before, the estimation process depends on the sampling and scheduling policies ${\xi }_i\left({\mathcal{I}}_k^i\right)$ and $\pi \left({\mathcal{I}}_k^s\right)$; hence, the sequence of control inputs $\{u_0^{i,\ast },\dots ,u_k^{i,\ast }\}$, $i\in \mathrm{N}$, is only optimal w.r.t. the given sampling and scheduling policies, and the control inputs are globally optimal only if sampling/scheduling policies are optimal. However, under any sampling policy that satisfies Assumption 1 and any scheduling policy, the optimal control law Equation (16) remains CE.

Now that the control law is characterized, we can derive the dynamics of the estimation error at the sampler, assuming that the local samplers are aware of the control law form in Equation (16). This assumption is essential in the sense that the samplers do not need to have the knowledge of the control inputs $\{u_0^i,\dots ,u_{k-1}^i\}$ to compute the estimation error, and this coincides with the information structure in Equation (8). The estimation error at the sampler is defined as:

$$\begin{array}{cc}\hfill e_k^i& =y_k^i-\mathsf{E}\left[x_k^i\right|{\mathcal{I}}_k^i].\hfill \end{array}$$

(17)

From Equation (8) and at time-step k, the sampler has the knowledge of the latest controller measurement update $z_{k-1}^i$. Let for any arbitrary ${\Delta }_{k-1}^i\in [0,k-1]$, $y_{k-1-{\Delta }_{k-1}^i}^i$ be the latest received state measurement by the estimator ${\mathcal{E}}_i$ at time-step $k-1$, i.e., ${\varphi }_{k-1}^i(k-1-{\Delta }_{k-1}^i)=1$. Then, similar to Equation (13), we can compute the estimation error $e_k^i$ as:

$$\begin{array}{cc}\hfill \{e_k^i& |{\varphi }_{k-1}^i(k-1-{\Delta }_{k-1}^i)=1\}\hfill \\ & =A_i^{{\Delta }_{k-1}^i+1}\left(x_{k-1-{\Delta }_{k-1}^i}^i-\mathsf{E}\left[x_{k-1-{\Delta }_{k-1}^i}^i\right|y_{k-1-{\Delta }_{k-1}^i}^i]\right)+A_i^{{\Delta }_{k-1}^i}{w}_{k-1-{\Delta }_{k-1}^i}^i+\dots +w_{k-1}^i+v_k^i\hfill \\ & =A_i^{{\Delta }_{k-1}^i+1}\left({\tilde{e}}_{k-1-{\Delta }_{k-1}^i}^i-v_{k-1-{\Delta }_{k-1}^i}^i\right)+v_k^i+\sum _{r=1}^{{\Delta }_{k-1}^i+1}{A}_i^{r-1}{w}_{k-r}^i.\hfill \end{array}$$

(18)

If ${\varphi }_{k-1}^i=0$, the estimation error at the sampler is, similar to Equation (12), computed based on the model parameters, i.e.,

$$\left\{e_k^i\right|{\varphi }_{k-1}^i=0\}=A_i\left({\tilde{e}}_{k-1}^i-v_{k-1}^i\right)+v_k^i+w_{k-1}^i.$$

(19)

Note the difference between ${\tilde{e}}_{k-1-{\Delta }_{k-1}^i}^i$ and ${\tilde{e}}_{k-1}^i$ in the Equations (18) and (19), where the former is the MMSE error due to having the measurement sample $y_{k-1-{\Delta }_{k-1}^i}^i$, while the latter is not MMSE as the estimator does not have access to $y_{k-1}^i$ at time-step $k-1$.

Remark 5.

Comparing Equations (12) and (19), we conclude that if the estimator ${\mathcal{E}}_i$ does not receive any state measurement update at time $k-1$, i.e., ${\varphi }_{k-1}^i=0$, then $e_k^i={\tilde{e}}_k^i$. It should, however, be noted that this equality is valid under the assumption that the estimator does not incorporate side-information contained in the sampling variables to compute the state estimate.

3.2. Co-Design of Sampling and Scheduling Laws

As the optimal control policy is shown to be CE, we now propose the sampling/scheduling co-design. We specifically focus on two common classes of policies, the ET and AoI utility functions, and study which class of policies is more suitable for sampling and which fits better to govern the scheduling process. We remind that the sampling is performed locally within each sub-system, while the scheduler resides in the network layer and is performed in a centralized fashion; see Figure 1.

We now introduce the ET and VoI functions used in the rest of this article. For the sampling policy, if the ET threshold-based approach is employed, then a sample of a local sub-system $i\in \mathrm{N}$ is generated and forwarded to the network buffer whenever the square norm of the corresponding sub-system’s estimation error exceeds a positive random threshold $r_k^i$, i.e.,

$${\delta }_k^{i,\mathrm{ET}}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\parallel e_k^i{\parallel }_2^2>r_k^i,\hfill \\ 0,\hfill & \mathrm{if}\parallel e_k^i{\parallel }_2^2\le r_k^i,\hfill \end{array}\right.$$

(20)

where the binary-valued ${\delta }_k^{i,\mathrm{ET}}$ indicates if a sample is forwarded for transmission or not based on the ET policy. The sequence of i.i.d. real-valued random thresholds $r_k^i\sim \exp \left({\mu }_r^i\right)$, $k\in {\mathbb{N}}_0$ is assumed to be exponentially distributed, with ${\mu }_r^i\in {\mathbb{R}}_{\ge 0}$ being the rate parameter of the exponential distribution. The random threshold policy is a more general form of the threshold-based policies; hence, the presented results in this article are easily extendable for the ET deterministic threshold-based approach. Note that the sampling policy in Equation (20) is in accordance with Assumption 1. We remind that $e_k^i$ denotes the estimation error computed at time k at the sampler side ${\mathcal{S}}_i$ (not at the controller side ${\mathcal{C}}_i$).

When the AoI policy is employed for sampling, a state sample of a sub-system i is sent to the communication network for transmission whenever the age of the latest received state information at the controller ${\mathcal{C}}_i$ exceeds a given threshold ${\lambda }^i\in {\mathbb{N}}_0$, i.e.,

$${\delta }_k^{i,\mathrm{AoI}}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{\Delta }_{k-1}^i>{\lambda }^i,\hfill \\ 0,\hfill & \mathrm{if}{\Delta }_{k-1}^i\le {\lambda }^i.\hfill \end{array}\right.$$

(21)

Since age is a discrete variable taking only non-negative integer values, without loss of any generality, the threshold ${\lambda }^i$ is also assumed to be selected from non-negative integers.

As a comparative scenario, we also consider the periodic sampling, in which each sensor sample is sent for transmission at pre-defined instances of time and the inter-transmission time is determined by the constant time period $T_p\in \mathbb{N}$. Therefore, we have:

$${\delta }_k^{i,\mathrm{P}}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}k=n{T}_p+i,n\in \mathbb{N},\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(22)

As noticed in the Equations (20)–(22), we use the superscripts “ET”, “AoI”, and “P” to indicate that the sampling policies are ET, AoI-based, and Periodic, respectively.

For the purpose of illustrations and ease of analysis, let us set the communication channel capacity to $c=1$, i.e., at every time-step k, the scheduler allows only one piece of state information to be forwarded to the corresponding controller (see Equation (4)). We already introduced ${\mathrm{N}}_k^b$ as the set of all sub-systems that have a state sample in the network buffer at time-step k. Note that this state information might belong to the current time k or to a previous time; hence, the buffered state measurements are not necessarily time-synchronized. For the AoI-based scheduling, we introduce the highest AoI policy, which in fact minimizes the average age of all sub-systems in ${\mathrm{N}}_k^b$ by selecting a sub-system with the highest age for transmission. For a sub-system $i\in {\mathrm{N}}_k^b$, this can be expressed as:

$$\mathsf{P}[{\varphi }_k^{i,\mathrm{AoI}}=1]=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{\Delta }_{k-1}^i>{\Delta }_{k-1}^j,\forall j\in {\mathrm{N}}_k^b,j\ne i\hfill \\ \frac{1}{{\eta }_k},\hfill & \mathrm{if}\underset{{\eta }_k\mathrm{sub-systems}}{\underbrace{{\Delta }_{k-1}^i=\dots ={\Delta }_{k-1}^l}}>{\Delta }_{k-1}^j,\forall j\in {\mathrm{N}}_k^b,j\ne i,\dots ,l\hfill \\ 0,\hfill & \mathrm{if}\exists j\in {\mathrm{N}}_k^b,{\Delta }_{k-1}^j>{\Delta }_{k-1}^i\hfill \end{array}\right.$$

(23)

where ${\eta }_k$ denotes the number of sub-systems in ${\mathrm{N}}_k^b$ with the highest age at time-step k. We also express that if $i\notin {\mathrm{N}}_k^b$, then $\mathsf{P}[{\varphi }_k^{i,\mathrm{AoI}}=1]=0$.

For randomized scheduling, we employ the common uniform randomization, and we, therefore, have for all $i\in \mathrm{N}$:

$$\mathsf{P}[{\varphi }_k^{i,\mathrm{R}}=1]=\left\{\begin{array}{cc}\frac{1}{|{\mathrm{N}}_k^b|},\hfill & \mathrm{if}i\in {\mathrm{N}}_k^b\hfill \\ 0,\hfill & \mathrm{if}i\notin {\mathrm{N}}_k^b\hfill \end{array}\right.$$

(24)

where $|·|$ represents the set cardinality operator and the superscript “R” in Equation (24) stands for the random scheduling policy.

In the following, we analytically compare AoI-based vs. ET design for the decentralized sampling and will show (Section 4.3) that the ET threshold-based sampling policy outperforms the AoI-based counterpart if thresholds are appropriately designed. We, moreover, show that the AoI sampling is in fact a more general form of periodic sampling with two differences: first, the transmission pattern may contain more than one fixed period, and second, the period(s) is (are) a function of the number of sub-systems and the AoI thresholds. For the centralized scheduling process, we employ the AoI-based prioritizing policy of highest AoI. In comparison with the randomized scheduling policy (Section 4.3), we show that the highest AoI policy does not necessarily outperform the randomized scheduling, if the heterogeneity of sub-systems is not taken into account in the scheduling process. We then propose the highest-age-first prioritization for the unstable sub-systems and show that this AoI-based policy is indeed capable of coordinating the communication resources more efficiently compared to the random scheduling, in the sense of the lower average sum of estimation errors of all sub-systems. We do not investigate the ET design as an applicable architecture for the scheduling policy since scheduling is a centralized process and decision making based on ET policies requires knowledge of real-time state information from all sub-systems, which might not be desirable. It is, however, conjectured that if for certain small-sized networked control scenarios, the ET policy might be favorable to be employed as the centralized scheduler, then it would even outperform AoI-based prioritizing scheduling due to its powerful capability of real-time prioritization based on the current state of each single sub-system. The sampling/scheduling policy combinations that we address in this article, either analytically or in the simulation results, are summarized in

Table 1. Considered combinations of sampling/scheduling policies. The combinations designated with * are discussed either analytically or in simulations.

		Scheduling
		ET	AoI	R	P
Sampling	ET		*	*
	AoI		*	*
	R
	P		*	*

.

4. Performance Analysis of the Joint Design

In this section, we propose two major co-design methodologies for the sampling and scheduling, where in the first method, the sampling process is governed by an AoI threshold-based policy introduced in Equation (21) and the scheduling is performed based on the highest AoI policy introduced in Equation (23). In the second co-design, the scheduling will be performed similarly based on the highest AoI policy law in Equation (23), while the sampling process is controlled by the ET threshold-based policy shown in Equation (20). We additionally consider the periodic sampling policy and random scheduling, introduced in Equations (22) and (24), respectively, as two conventional models for sampling and scheduling and provide comparisons, theoretically or numerically, with the proposed co-designs. For the purpose of brevity, we use the abbreviations “AoI/AoI”, “ET/AoI”, “ET/R”, “AoI/R”, “P/R”, and “P/AoI” to denote the combined “sampling/scheduling” policy; see Table 1. To avoid confusion, it is worth reminding about the difference between the AoI policies for decentralized sampling and centralized scheduling; see Equations (21) and (23).

4.1. AoI Sampling and Scheduling Co-Design

In the AoI/AoI co-design architecture, the AoI sampling is performed locally at every sub-system’s sensor station according to the threshold-based policy given in Equation (21), while the AoI scheduling is done in a centralized fashion according to the highest AoI prioritizing policy in Equation (23). Assume an NCS is comprised of a set of stable and a set of unstable sub-systems, denoted respectively by ${\mathrm{N}}_s$ and ${\mathrm{N}}_u$, where ${\mathrm{N}}_s\cup {\mathrm{N}}_u=\mathrm{N}$ and $N_s=\left|{\mathrm{N}}_s\right|$ and $N_u=\left|{\mathrm{N}}_u\right|$ indicate the number of stable and unstable sub-systems, respectively. Here, we study the asymptotic sampling and transmission patterns for the AoI/AoI co-design for different values of the deterministic thresholds ${\lambda }^i$, $i\in {\mathrm{N}}_s$ and ${\lambda }^j$, $j\in {\mathrm{N}}_u$.

Let ${\lambda }^i<N$ and ${\lambda }^j<N$. It is straightforward to conclude that each sub-system, either stable or unstable, will be scheduled for transmission once in every N time-steps with a fixed unique pattern. Moreover, the sampler of each stable sub-system will send $N-{\lambda }^i$ number of samples to the buffer in the same cycle of N time-steps, while unstable sub-systems send each $N-{\lambda }^j$ samples. We demonstrate this pattern with an illustrative example below and then summarize the concluding statements in the Proposition 1.

Illustrative example: Assume $N_s=N_u=3$, $c=1$, ${\lambda }^i=5$, and ${\lambda }^j=2$. Figure 2 shows the sampling and transmission patterns of each sub-system, wherein each circle (square) shows that a new measurement sample from a stable (unstable) sub-system is sent to the buffer. The red-bordered ones are the scheduled data packets, and the numbers inside circles and squares denote the age of that corresponding sub-system at that time-step. According to Equation (21), every Unstable System j (denoted by US1, US2, and US3 in Figure 2) sends a fresh sample to the buffer at any time-step k at which ${\Delta }_{k-1}^j>2$. Hence, no data packet is injected into the buffer before time-step $k=4$, at which all three unstable sub-systems will send a measurement sample to the buffer. Note that, at time-step $k=4$, the samplers decide based on ${\Delta }_3^j=3>2$. The same occurs for the Stable Sub-systems (denoted by SS1, SS2, and SS3 in Figure 2); hence, they all send their first measurement samples to the buffer at time-step $k=7$, knowing that ${\Delta }_6^i=6>5$. Since at time-step $k=4$, there are three data packets all with identical highest ages, the AoI scheduler selects one of the three randomly, i.e., ${\eta }_4=3$ (see the second argument of Equation (23)). This randomization is repeated again at the next time-step $k=5$, now with only two data packets with similar ages belonging to US2 and US3 (US1 remains silent for the next two time-steps). At $k=6$, there is only one data packet in the buffer, and it is certainly scheduled as there is no competition for the single transmission resource. At time-step $k=7$, there are four data packets belonging to SS1, SS2, SS3, and US1. The data packet belonging to US1 will not be scheduled for transmission because it has a lower age compared to the other three. For the remaining ones with the same ages, one will be scheduled for transmission randomly (e.g., SS2, as in Figure 2). At $k=8$, random selection is done between only SS1 and SS3 since the existing data packets of US1 and US2 entail lower ages. Finally, at $k=9$, SS1 is certainly scheduled for transmission as it has the highest age among all the data packets in the buffer. From this time-step forward, the same pattern of transmissions is repeated without any randomization.

As is also illustrated by the above example, we state the following proposition for which we omit the lengthy, but straightforward proof:

Proposition 1.

For the sketched heterogeneous NCS scenario, if $c=1$, ${\lambda }^i,{\lambda }^j<N$, $\forall i\in {\mathrm{N}}_s$, and $\forall j\in {\mathrm{N}}_u$, then the following statements hold, asymptotically:

• each sub-system is scheduled a transmission once every N time-steps.
• stable and unstable sub-systems send, respectively, $N-{\lambda }^i$ and $N-{\lambda }^j$ fresh samples to the buffer during every N time-steps.
• if ${\lambda }^i={\lambda }^j=N-1$, then the AoI sampling is equivalent to the time-triggered sampling.

Now, assume that ${\lambda }^i,{\lambda }^j\ge N$. We can express similar statements as in Proposition 1 and conclude that both stable and unstable sub-systems successfully transmit in the asymptotic regime, respectively, every ${\lambda }^i+1$ and ${\lambda }^j+1$ time-steps, and they send only one sample to the buffer per each successful transmission. It is then clear that this scenario is also equivalent to the periodic transmission with periods of ${\lambda }^i+1$ and ${\lambda }^j+1$ for stable and unstable sub-systems, respectively.

If ${\lambda }^i\ge N$ and ${\lambda }^j<N$, the transmission pattern for each sub-system $i\in {\mathrm{N}}_s$ is similarly periodic with a time period of ${\lambda }^i+1$, and only one measurement sample is sent to the buffer per each transmission. For sub-systems $j\in {\mathrm{N}}_u$, however, the transmission pattern is not periodic with a unique period, i.e., the inter-transmission times vary between every two consecutive successful transmissions, if $N\le {\lambda }^i<2N$. In fact, it changes between ${\lambda }^j+1$ and N for each $j\in {\mathrm{N}}_u$. When the inter-transmission time is ${\lambda }^j+1$, no data sample is discarded in between, while when it is N, each sub-system j sends $N-{\lambda }^j$ number of samples per each transmission. In addition, if ${\lambda }^i\to \infty$, then every $j\in {\mathrm{N}}_u$ successfully transmits every ${\lambda }^j+1$ time-steps during which each sub-system j sends $\max (1,N_u-{\lambda }^j)$ number of samples to the buffer. The same can be said for ${\lambda }^j\to \infty$. These statements can be numerically tested by, for example, setting ${\lambda }^i=7$ in the depicted illustrative example in Figure 2.

From the above discussions, we can make two crucial conclusions. First, the AoI/AoI co-design policy governed by the AoI threshold-based sampling law in Equation (21) and AoI-based highest AoI scheduling law in Equation (23) is not equivalent to the unique fixed periodic transmission policy, although, for some specific parameters, e.g., ${\lambda }^i={\lambda }^j=N-1$, they coincide. Second, all the possible transmission patterns are determined by the capacity constraint in Equation (4), the AoI thresholds ${\lambda }^i$ and ${\lambda }^j$, and the number of network sharing sub-systems, which are all constants. Hence, the resulting transmission patterns are insensitive w.r.t. the dynamics of stable or unstable sub-systems. We may design the AoI thresholds ${\lambda }^i$ and ${\lambda }^j$ differently for stable and unstable sets of sub-systems; however, they are assumed to be constant parameters and not adjusted by changing the dynamics (The discussions of Section 4.1 can be extended to cover the scenarios in which the AoI thresholds are not identical within the set of stable or unstable sub-systems, i.e., if $i,l\in {\mathrm{N}}_u$, then ${\lambda }^i\ne {\lambda }^l$. Although this leads to more complex transmission patterns, it does not contradict the crucial conclusions of this section, as summarized in the last part of Section 4.1.).

4.2. ET Sampling and AoI Scheduling Co-Design

In this section, we study the ET/AoI co-design architecture, where the sampling is locally performed according to the ET law in Equation (20) and the scheduling is centrally governed by the highest AoI policy in Equation (23). For the clarity of analysis and illustrative purposes, we first assume that the ET thresholds in Equation (20) are deterministic and constant, i.e., $r_k^i=r^i\in {\mathbb{R}}_{\ge 0}$. We discuss in the next section how to extend the performance results to the ET sampling with stochastic thresholds.

Since the estimation error $e_k^i$ is a random variable (see Equations (18) and (19)), there is generally no fixed pattern for transmission of each sub-system when sampling is controlled by the ET law in Equation (20). Hence, we study the asymptotic transmission rate for which we try to find mathematical expressions or bounds. To do that, we first compute the asymptotic sampling rate for an arbitrary sub-system $i\in \mathrm{N}$, as follows:

$$\underset{k\to \infty }{lim}\mathsf{E}\left[{\delta }_k^i\right]=\underset{k\to \infty }{lim}\mathsf{P}({\delta }_k^i=1)=\underset{k\to \infty }{lim}\mathsf{P}(\parallel e_k^i{\parallel }_2^2>r^i)\le \frac{{lim}_{k\to \infty }\mathsf{E}[\parallel e_k^i{\parallel }_2^2]}{r^i},$$

(25)

where the inequality in Equation (25) is obtained using Markov’s inequality knowing $\parallel e_k^i{\parallel }_2^2$ is a non-negative random variable and $r^i$ is a non-negative constant. To provide a more meaningful bound, we first state the following Lemma, which essentially states that the dynamics of a stable sub-system’s estimation error variance is insensitive to closing the feedback loop, asymptotically.

Lemma 1.

For any stable LTI stochastic control system modeled by Equation (1), the estimation error variance is asymptotically bounded regardless of how often the feedback loop is closed.

Proof. 

From Equation (19), we can express the estimation error at the sampler’s side, assuming that no transmission has taken place from the initial time until the current time k, i.e., ${\varphi }_1^i={\varphi }_2^i=\dots ={\varphi }_{k-1}^i=0$, which ensures ${\Delta }_{k-1}^i=k-1$, as follows:

$$e_k^i=A_i^k\left({\tilde{e}}_0^i-v_0^i\right)+v_k^i+\sum _{r=1}^k{A}_i^{r-1}{w}_{k-r}^i=A_i^k\left(x_0^i-{\mu }_{x_0^i}\right)+v_k^i+\sum _{r=1}^k{A}_i^{r-1}{w}_{k-r}^i,$$

where the second equality holds since ${\tilde{e}}_0^i=y_0^i-\mathsf{E}\left[x_0^i\right]=x_0^i+v_0^i-{\mu }_{x_0^i}$. According to the last expression, $e_k^i$ is zero-mean; hence, we can compute the asymptotic estimation error variance as:

$$\begin{array}{cc}\hfill \underset{k\to \infty }{lim}\mathsf{E}\left[e_k^{i^{\top }}{e}_k^i\right]& =\underset{k\to \infty }{lim}\mathsf{E}\left[\parallel e_k^i{\parallel }_2^2\right]=\underset{k\to \infty }{lim}\mathsf{E}\left[\parallel A_i^k\left(x_0^i-{\mu }_{x_0^i}\right)+v_k^i+\sum _{r=1}^k{A}_i^{r-1}{w}_{k-r}^i{\parallel }_2^2\right]\hfill \\ \hfill & =\underset{k\to \infty }{lim}\mathsf{E}\left[\parallel A_i^k\left(x_0^i-{\mu }_{x_0^i}\right){\parallel }_2^2\right]+\underset{k\to \infty }{lim}\sum _{r=1}^k\mathsf{E}\left[\parallel A_i^{r-1}{w}_{k-r}^i{\parallel }_2^2\right]+{\Sigma }_{v^i}\hfill \end{array}$$

(26)

$$\begin{array}{cc}& \le \underset{k\to \infty }{lim}\sum _{r=1}^k{\parallel A_i\parallel }_2^{2(r-1)}{\Sigma }_{w^i}+{\Sigma }_{v^i}\hfill \\ & =\frac{{\Sigma }_{w^i}}{1-\parallel A_i{\parallel }_2^2}+{\Sigma }_{v^i},\hfill \end{array}$$

(27)

where, to obtain Equation (26), we used the mutual statistical independence of $x_0^i$, $v_k^i$, and $\{w_0^i,w_1^i,\dots ,w_k^i\}$, and the third expression is derived using the sub-multiplicative property of vector norms and also knowing that ${lim}_{k\to \infty }{A}_i^k=0$, since $A_i$ is Hurwitz. The final bound in Equation (27) is obtained knowing that the infinite series ${lim}_{k\to \infty }{\sum }_{r=1}^k{\parallel A_i\parallel }_2^{2(r-1)}{\Sigma }_{w^i}$ is convergent since $\parallel A_i{\parallel }_2^2<1$, which completes the proof. □

Having Lemma 1, we can then re-express Equation (25) for all stable sub-systems $i\in {\mathrm{N}}_s$, as follows:

$$\underset{k\to \infty }{lim}\mathsf{P}({\delta }_k^i=1|i\in {\mathrm{N}}_s)\le \frac{1}{r^i}\left(\frac{{\Sigma }_{w^i}}{1-\parallel A_i{\parallel }_2^2}+{\Sigma }_{v^i}\right).$$

(28)

It should be noted that the Equation (25) holds for both stable and unstable sub-systems, while Equation (28) is valid only for the former one. Moreover, the bound in Equation (28) becomes trivial if the right-hand side of the inequality is greater than one, so this might also be seen as a rule to design the threshold $r^i$. It is clear that the higher $r^i$ is, the lower the transmission rate of stable sub-systems becomes, which is expected.

Lemma 1 does not apply to unstable sub-systems; hence, to derive a similar upper-bound for sub-systems $j\in {\mathrm{N}}_u$, we compute ${lim}_{k\to \infty }\mathsf{E}[\parallel e_k^j{\parallel }_2^2]$ according to the estimation error dynamics in Equation (18), as in the following:

$$\begin{array}{cc}& \underset{k\to \infty }{lim}\mathsf{E}[\parallel e_k^j{\parallel }_2^2|{\Delta }_{k-1}^j,j\in {\mathrm{N}}_u]=\hfill \\ & \underset{k\to \infty }{lim}\mathsf{E}\left[\parallel A_j^{{\Delta }_{k-1}^j+1}\left({\tilde{e}}_{k-1-{\Delta }_{k-1}^j}^j-v_{k-1-{\Delta }_{k-1}^j}^j\right)+v_k^j+\sum _{r=1}^{{\Delta }_{k-1}^j+1}{A}_j^{r-1}{w}_{k-r}^j{\parallel }_2^2|{\Delta }_{k-1}^j,j\in {\mathrm{N}}_u\right]\le \hfill \\ & {\Sigma }_{v^j}+\underset{k\to \infty }{lim}\sum _{r=1}^{1+{\Delta }_{k-1}^j}\parallel A_j^{r-1}{\parallel }_2^2{\Sigma }_{w^j}+\underset{k\to \infty }{lim}{\parallel A_j\parallel }_2^{2({\Delta }_{k-1}^j+1)}\mathsf{E}\left[\parallel {\tilde{e}}_{k-1-{\Delta }_{k-1}^j}^j-v_{k-1-{\Delta }_{k-1}^j}^j{\parallel }_2^2|{\Delta }_{k-1}^j,j\in {\mathrm{N}}_u\right].\hfill \end{array}$$

(29)

Let us denote ${lim}_{k\to \infty }{\Delta }_{k-1}^j={\Delta }_{\infty }^j$. Having the highest AoI scheduler introduced in Equation (23), we can compute an upper-bound for Expression (29) by evaluating the two disjoint cases ${lim}_{k\to \infty }{\parallel e_k^j\parallel }_2^2>r^j$ or ${lim}_{k\to \infty }{\parallel e_k^j\parallel }_2^2\le r^j$, almost surely. If the first case holds, then according to Equation (23), we know that a measurement sample belonging to the sub-system j should exist in the buffer, and according to Proposition 1 and the discussions afterwards, it yields that ${lim}_{k\to \infty }{\Delta }_{k-1}^j\le M^j\triangleq \max \{N,{\lambda }^j+1\}$. Otherwise, if ${lim}_{k\to \infty }{\parallel e_k^j\parallel }_2^2\le r^j$, almost surely, no measurement sample would be sent to the buffer asymptotically according to the ET sampling law in Equation (20), and then, ${lim}_{k\to \infty }{\Delta }_{k-1}^j>\max \{N,{\lambda }^j+1\}$. Finally, knowing that ${\tilde{e}}_{k-1-{\Delta }_{k-1}^j}^j$ is the MMSE error computed by the Kalman filter having access to the measurement $y_{k-1-{\Delta }_{k-1}^j}^j$, we can rewrite Equation (29) as:

$$\begin{array}{cc}\hfill \underset{k\to \infty }{lim}\mathsf{E}[\parallel e_k^j{\parallel }_2^2|\underset{k\to \infty }{lim}{\Delta }_{k-1}^j\le M^j,j\in {\mathrm{N}}_u]& \le {\Sigma }_{v^j}+\sum _{r=1}^{M^j+1}\parallel A_j^{r-1}{\parallel }_2^2{\Sigma }_{w^j}+\underset{k\to \infty }{lim}{\parallel A_j\parallel }_2^{2(M^j+1)}{P}_{k-1-{\Delta }_{k-1}^j}^j\hfill \\ & ={\Sigma }_{v^j}+\sum _{r=1}^{M^j+1}\parallel A_j^{r-1}{\parallel }_2^2{\Sigma }_{w^j}+{\parallel A_j\parallel }_2^{2(M^j+1)}{P}_{\infty }^j,\hfill \end{array}$$

(30)

where $P_{\infty }^j={lim}_{k\to \infty }{P}_{k-1-{\Delta }_{k-1}^j}^j$ is the asymptotic estimation error covariance and will be obtained from the following algebraic Riccati equation:

$$P_{\infty }^j=A_j\left(P_{\infty }^j-P_{\infty }^j{(P_{\infty }^j+{\Sigma }_{v^j})}^{-1}{P}_{\infty }^j\right)A_j^{\top }+{\Sigma }_{w^j}.$$

Finally, having Expression (30), we can express the upper-bound in Equation (25) for the unstable sub-systems $j\in {\mathrm{N}}_u$ as:

$$\underset{k\to \infty }{lim}\mathsf{P}({\delta }_k^j=1|j\in {\mathrm{N}}_u)\le \frac{{\Sigma }_{v^j}+{\sum }_{r=1}^{M^j+1}\parallel A_j^{r-1}{\parallel }_2^2{\Sigma }_{w^j}+{\parallel A_j\parallel }_2^{2(M^j+1)}{P}_{\infty }^j}{r^j}.$$

(31)

Note that, if ${lim}_{k\to \infty }{\Delta }_{k-1}^j\le M^j$ holds for finite $M^j$, $P_{\infty }^j$ will also be finite as the Kalman filter receives state measurements asymptotically to compute the MMSE error. Even though the received measurements might not be fresh, the delay is finite, and the Kalman filter algorithm converges.

4.3. Performance Comparisons

To conduct asymptotic performance analysis, we consider a heterogeneous NCS comprised of multiple stable and unstable sub-systems. Note that, for an NCS including all stable sub-systems, asymptotic performance becomes independent of the sampling, scheduling, and control policies due to the natural convergence of states. In fact, for a stochastic system of the form given in Equation (1), the system states are expected to converge asymptotically to a bounded set around the origin where the set boundary is characterized by the moments of the primitive random variables, as we will show later in this section. Hence, to study the properties of the co-design, we focus on the heterogeneous NCS scenario.

The overall asymptotic performance of the NCS is measured by the average cost functions J introduced in Equation (15). It can be seen from (A3) that the local LQG cost function $J_i$ can be minimized by the control law ${\gamma }^i$ in an inner optimization problem, and then, the residual cost becomes a function of the sampling law ${\xi }_i$. From (A8), we know that the residual local cost is a function of $\mathsf{E}\left[x_k^i\right|{\tilde{\mathcal{I}}}_k^i]$ and ${\psi }_k^i$ where the latter is shown in (A15) to be a function of the estimation error ${\tilde{e}}_t^i$ and its variance $P_t^i$, $t\ge k$. It should be remembered, however, that the resource constraint in Equation (4) does not allow the transmissions to be solely determined by the sampling law ${\xi }_i$, and therefore, the local cost $J_i$, from the overall perspective of control and network layers, becomes dependent also on the scheduling law $\pi$. The dependency appears in ${\tilde{\mathcal{I}}}_k^i$, which eventually affects both the estimate $\mathsf{E}\left[x_k^i\right|{\tilde{\mathcal{I}}}_k^i]$ and ${\psi }_k^i$. Hence, the optimal asymptotic cost function in Equation (15) should be minimized by both the sampling and scheduling policies, which can be written as:

$$\begin{array}{c}\hfill J=\underset{T\to \infty }{lim}\frac{1}{N}\sum _{i=1}^N\frac{1}{T}\mathsf{E}\left[\underset{{\delta }_{[0,T-1]}^i,{\varphi }_{[0,T-1]}^i}{\min }\mathsf{E}\left[\mathsf{E}{\left[x_0^i\right]}^{\top }{P}_0^i\mathsf{E}\left[x_0^i\right]+{\psi }_0^i\right]\right]\end{array}$$

(32)

where in $\mathsf{E}\left[x_0^i\right]$ is known a priori and ${\psi }_0^i$ is a function of the estimation errors ${\tilde{e}}_k^i$ and their variances $P_k^i$, $k\in \{0,1,2,\dots \}$, according to Expression (A15).

Solving the optimization problem in Equation (32) is very challenging due to the coupling of the decision variables with ${\psi }_0^i$ through the end-to-end delay ${\Delta }_t^i$ and also the non-linear nature of the ET and AoI functions. The aim of this article is, therefore, to identify the appropriate class of policies for the sampling and scheduling that jointly result in an improved overall performance. Since the overall performance is a convex function of the estimation error according to Equation (32), it is easier to study the asymptotic behavior of the estimation error of all sub-systems. In fact, if a certain co-design of sampling and scheduling policies results in a lower asymptotic average sum of estimation errors of all sub-systems, compared to another co-design, then it certainly results in a lower asymptotic overall cost as well. Hence, for the performance analysis and to make comparisons between different sampling/scheduling co-designs, we consider the following performance metric:

$$J_{\infty }^e=\underset{k\to \infty }{lim}\frac{1}{N}\sum _{i=1}^N\mathsf{E}\left[{\tilde{e}}_k^{i^{\top }}{\tilde{e}}_k^i\right]=\underset{k\to \infty }{lim}\frac{1}{N}\left[\sum _{i=1}^{N_s}\mathsf{E}\left[{\tilde{e}}_k^{i^{\top }}{\tilde{e}}_k^i\right]+\sum _{j=1}^{N_u}\mathsf{E}\left[{\tilde{e}}_k^{j^{\top }}{\tilde{e}}_k^j\right]\right].$$

(33)

Theorem 2 summarizes the second main result of this article on the appropriate sampling/scheduling co-design architecture. First, we define “non-trivial threshold-based functions” as follows:

Definition 2.

A threshold-based function with stochastic thresholds of the form given in Equation (20) is said to be non-trivial if $\mathsf{P}[r_k^i\ne 0]>0$, $\forall k$, and almost surely, $\mathrm{P}[r_k^i=\infty ]=0$, $\forall k$. For deterministic threshold-based functions of the form given in Equation (21), we call the threshold-based function non-trivial if ${\lambda }^i\ne \{0,\infty \}$.

Theorem 2.

Consider a heterogeneous NCS comprised of N LTI stochastic sub-systems modeled as in Equations (1) and (2) from which $N_s$ sub-systems are stable and $N_u$ sub-systems are unstable. Let the network scheduler select only one sub-system per time-step to transmit its freshest state measurement in the buffer to the controller, i.e., $c=1$ in Equation (4). Then, for any non-trivial AoI sampling policy given in Equation (21), there exists a non-trivial constant threshold ET sampling policy in the form of Equation (20) that asymptotically outperforms, in terms of Equation (33), the AoI sampling policy, given that the network scheduling policy obeys the AoI-based highest AoI law in Equation (24).

Proof. 

See Appendix B. □

Remark 6.

Reminding about the evolution of $e_k^j$ in Equations (18) and (19), it is clear that the estimation error at the sampler has a zero-mean, but not normal distribution. For general square matrix $A_j\in {\mathbb{R}}^{n^j\times n^j}$, the asymptotic CDF of $\parallel e_k^j{\parallel }_2^2$ might not have an analytical form, but can be efficiently computed numerically. Indeed, the distribution of $\parallel e_k^j{\parallel }_2^2$ for general $A_j$ is determined by the distribution of its elements, which are statistically dependent via the off-diagonal elements of $A_j$. For specific forms of $A_j$, however, the CDF has indeed an analytical form. If $A_j$ is a diagonal matrix, then the distribution of $\parallel e_k^j{\parallel }_2^2$ follows the sum of $n_j$ independent Gamma distributions, which has an analytical CDF. For scalar systems, the distribution of $\parallel e_k^j{\parallel }_2^2=e_k^{j^2}$ follows a single Gamma distribution.

Remark 7.

Theorem 2 can be extended to the case that the thresholds $r_k^i$ and $r_k^j$ are stochastic, as in Equation (20). This would result in Markov’s inequality in Equation (25), and the expression ${lim}_{k\to \infty }\mathrm{P}(\parallel e_k^j{\parallel }_2^2>r^j)=1-F_{\parallel e^j\parallel }^j\left(r^j\right)$ to be no longer valid due to the random nature of the thresholds. For stochastic thresholds, instead of Markov’s inequality, which holds for non-negative random variables, we can employ the Chernoff bound, which is a generalization of Markov’s inequality for real-valued random variables. In fact if the thresholds are stochastic, we can construct the new real-valued random variable $\parallel e_k^j{\parallel }_2^2-r_k^j$ and find the upper-bound for it by applying the Chernoff bound. Further, we can write ${lim}_{k\to \infty }\mathsf{P}(\parallel e_k^j{\parallel }_2^2-r_k^j>0)=1-F_{\parallel e^j\parallel -r^j}^j\left(0\right)$, where $F_{\parallel e^j\parallel -r^j}^j$ is now the asymptotic CDF of the constructed random variable $\parallel e_k^j{\parallel }_2^2-r_k^j$. Note that, $F_{\parallel e^j\parallel }^j\left(0\right)=0$ since $\parallel e_k^j{\parallel }_2^2$ is a non-negative random variable; however, $F_{\parallel e^j\parallel -r^j}^j\left(0\right)>0$ since $\parallel e_k^j{\parallel }_2^2-r_k^j$ is real-valued. The CDF $F_{\parallel e^j\parallel -r^j}^j$ may not have an analytical form, depending on the distributions of the random variables $\parallel e_k^j{\parallel }_2^2$ and $r_k^j$.

In the following, we discuss how the highest AoI scheduling policy in Equation (23) may outperform the randomized transmission policy in Equation (24), but not always. In fact, we discuss how if the scheduler’s “highest-age-first” prioritizing feature is applied first to the set of unstable sub-systems, then it certainly outperforms the randomized transmission policy. We then define the prioritizing unstable (PU) highest AoI policy similar to Equation (23), with the exception that the law is applied asymptotically, first on the set of unstable sub-systems, and the resource is assigned to the unstable sub-system (i.e., the resource is assigned to the unstable sub-system with the highest age, even if there are stable ones with higher age).

Corollary 1.

For a fixed sampling policy, the highest AoI scheduling law in Equation (23) does not necessarily outperform the randomized scheduling policy in Equation (24), asymptotically, in an NCS of heterogeneous stable and unstable control sub-systems sharing a capacity-limited communication network. The PU highest AoI, however, asymptotically outperforms the randomized scheduling policy in Equation (24).

Proof. 

As discussed before, the average sum of the estimation error variance of the set of stable sub-systems does not asymptotically change sampling and scheduling decisions. Reminding about Equation (13), we see that the higher the age ${\Delta }_k^i$ is for unstable sub-systems, the larger the estimation error becomes. This is also true for the variance of the estimation error. Therefore, if a scheduling policy results in a higher transmission probability for the unstable sub-systems with the highest age, then the average sum of the estimation error variance will also be more reduced. According to Equation (24), all sub-systems that have a data packet in the buffer are assigned identical probabilities of transmission $\frac{1}{|{\mathrm{N}}_k^b|}$, irrespective of their age or stability properties. For the same set of sub-systems, each with a packet in the buffer, the probability that the sub-system with the highest age (stable or unstable) successfully transmits is, according to Equation (23), $\frac{1}{{\eta }_k}$, if there are ${\eta }_k\le \left|{\mathrm{N}}_k^b\right|$ number of sub-systems all with the identical highest age, which leads to $\frac{1}{{\eta }_k}\ge \frac{1}{|{\mathrm{N}}_k^b|}$. The equality occurs only if all sub-systems in the buffer have a similar age, which is also the highest age. It should, however, be noted that if all those sub-systems that have the highest age are stable, then the unstable sub-systems in the buffer that may have a relatively large age, but not the highest, would be assigned with a probability of zero for successful transmissions, while this probability is $\frac{1}{|{\mathrm{N}}_k^b|}$ for the randomized scheduling policy that leads to a lower average sum of estimation error variance. With the modified PU highest AoI, however, the described problem can be easily considered in scheduling, and therefore, this policy always either outperforms or results in a similar performance as for the randomized scheduling, given a similar sampling policy. □

5. Numerical Evaluations

We consider different NCS setups with different numbers of stable and unstable sub-systems to numerically test the co-design architectures and compare with the common approaches. The number of sub-systems N is chosen from the set $\{2,4,6,8,10\}$ with an equal number of stable and unstable sub-systems, that is, $\frac{N}{2}$ each. For the ease of interpretation, we choose scalar LTI sub-systems. The system matrices for stable and unstable sub-systems are selected to be $0.5$ and $1.05$, respectively, and $B_i=1$ for all $i\in \mathrm{N}$. The system disturbance is modeled as $w_k^i\sim \mathcal{N}(0,1)$, for all $i\in \mathrm{N}$, and $k\in {\mathbb{N}}_0$, and for the ease of illustrations, we assume that measurements are noiseless. Each data point in the plots is generated by running the simulative setup for ${10}^6$ iterations. In the following, we define the sampling and network scheduling strategies and the selected parameter values for each scenario.

Sampling Strategies

• Event triggering: The sampler (sensor) samples the plant in each time-step, and if the value of the estimation error is greater than a threshold, then the sample is forwarded to the queue. The threshold is generated from an exponential distribution, and the mean of the distribution is chosen from the set $\{0,0.1,0.5,1,2:2:30\}$, where 2:2:30 are integer values in $[2,30]$ that are divisible by two. We use a default setting where each sampler uses the same mean threshold.
• Period-n sampling: Each sampler samples the plant periodically with period n.
• AoI sampling (n): Each sampler samples the plant whenever the AoI at the sampler exceeds n. The AoI at the sampler is equal to the AoI at the respective estimator from the previous time-step plus one. The default value for n is $N-1$, and for brevity we use AoI sampling to refer to this case.

Network Scheduling

• Highest AoI: Under this scheduling policy, the network chooses the plant that has a packet in the queue and the maximum AoI at the estimator, which is the “highest-age-first” prioritization.
• Randomized: The network scheduler chooses a packet from the queue in the uniform random fashion.
• Prioritizing unstable (PU) highest AoI: The scheduler selects the unstable plant which has a packet in the queue and has maximum AoI. If no packet belonging to an unstable plant exists in the queue, then the scheduler selects the stable plant with a packet in the queue with highest AoI.

In Figure 3, we plot the average estimation error variance (across all sub-systems) by varying the ET mean threshold. We observe that the estimation error variance is minimized for certain mean threshold values, which increase with the number of sub-systems N. To understand this, note that when the thresholds are small, all the samplers will place a packet in the queue in almost every time-step. In this case, highest AoI behavior is close to round-robin scheduling for the plants. Thus, plants with high or low estimation errors are treated rather indifferently, leading to relatively high estimation error variance. The threshold values that attain minimum estimation error variances are such that the sub-systems with low estimation errors (usually the stable ones) do not contend for the network frequently as they do not exceed the thresholds frequently. This results in more often transmission of packets from sub-systems with high estimation errors, thus lowering the overall estimation error variance. In the following figures, we present the statistics of the ET sampling at mean thresholds that provide the lowest estimation error variance, which are marked by black circles in Figure 3.

In Figure 4, we compare the estimation error variance achieved under different schemes. For ET sampling, we plot the minimum estimation error variance achieved over different thresholds for each N. We observe that the ET sampling strategy obtains a lower estimation error variance, 30–40% lower than that of AoI sampling when $N=10$. While AoI sampling and Period-1 sampling result in the same variance, the former’s sampling frequency is much lower and equals $\frac{1}{N-1}$. Furthermore, it can be observed that, in general, using highest AoI scheduling results in a lower estimation error variance especially as the number of sub-systems grows. In Figure 5, we observe the estimation error variance of various sampling policies with the PU highest AoI scheduling policy, and as expected, we see that the ET sampling outperforms both periodic and AoI sampling policies. Periodic sampling yields better performance than the AoI sampling basically due to the sampling threshold of the latter, which is $\frac{N}{2}$. However, comparing the AoI sampling with PU highest AoI and AoI sampling with highest AoI (the solid red plot in Figure 4), we observe that prioritizing in favor of unstable plants leads to performance improvements.

In Figure 6, we compare the normalized total number of network transmissions that occur under different schemes. While Period-1 sampling and AoI sampling result in a transmission in each time-step, ET sampling results in transmissions $80\%$ of the time for a varying number of sub-systems. This is because the queue remains empty $20\%$ (on average) under ET sampling since only sub-systems with estimation errors greater than the threshold are allowed to place a packet in the queue. Therefore, ET sampling not only provides lower estimation error variance, but also reduces the number of network transmissions.

In Figure 7, we compare the average AoI (averaged over all the estimators), achieved under different schemes. Since AoI sampling samples a plant based on its AoI at the estimator, this strategy results in the lowest average AoI. ET sampling, however, results in higher average AoI, as it samples based on estimation error, which increases non-linearly with AoI. Furthermore, since the highest AoI scheduling picks the plant with the highest AoI and transmits its packet, this strategy results in lower average AoI across different sampling strategies. For the PU highest AoI, we again observe relatively higher values of AoI, which is expected because under this policy stable plants have transmission chance only when there is no packet belonging to an unstable plant in the queue. The main conclusion is that, although the ET sampling policy does not result in the lowest average AoI across the NCS, it results in the lowest achieved estimation error variance. In addition, PU highest AoI results in noticeably higher average AoI compared to the randomized and highest AoI policies, but leads to lower average estimation error variance (see Figure 5 and Figure 7).

6. Conclusions

In this article, the major goal is to propose a co-design networked control architecture of sampling, scheduling, and control for NCSs comprised of multiple heterogeneous LTI stochastic control systems that close their sensor-to-controller loops over a shared capacity-limited communication network. We first show that under mild assumptions on the information structure of each policy maker, the optimal control law is of the certainty equivalence form. We then investigate various combinations of decentralized sampling and centralized scheduling architectures employing the applicable concepts of event-triggered and AoI utility functions. We analytically show that the event-triggered sampling is capable of asymptotically outperforming the AoI sampling policy when the communication resources are limited, while we demonstrate that AoI-based prioritizing scheduling may outperform the randomized scheduling policy under the appropriate prioritization metric. To discuss the effectiveness of each co-design, we measure the overall NCS performance by the average sum of local LQG cost functions. Our theoretical analyses are successfully validated for the proposed co-designs, and comparisons are made with conventional periodic and randomized network access approaches through simulations on different NCS scenarios.