Robust Stability of Scaled-Four-Channel Teleoperation with Internet Time-Varying Delays

Abstract

We describe the application of a generic stability framework for a teleoperation system under time-varying delay conditions, as addressed in a previous work, to a scaled-four-channel (γ-4C) control scheme. Described is how varying delays are dealt with by means of dynamic encapsulation, giving rise to mu-test conditions for robust stability and offering an appealing frequency technique to deal with the stability robustness of the architecture. We discuss ideal transparency problems and we adapt classical solutions so that controllers are proper, without single or double differentiators, and thus avoid the negative effects of noise. The control scheme was fine-tuned and tested for complete stability to zero of the whole state, while seeking a practical solution to the trade-off between stability and transparency in the Internet-based teleoperation. These ideas were tested on an Internet-based application with two Omni devices at remote laboratory locations via simulations and real remote experiments that achieved robust stability, while performing well in terms of position synchronization and force transparency.

1. Introduction

A teleoperation system consists of master and slave mechanical systems where the master is directly manipulated by a human operator and the slave, operating in a remote environment, is designed to track the master closely. The main issues in the analysis and synthesis of these systems are stability and transparency. Transparency refers to the level of achievement of an ideal situation of the operator acting on the remote environment. In practice, there is a compromise between these two goals mainly due to the presence of time delays generated by the communication channel [1].

Given this compromise, many control schemes for teleoperation have been proposed, broadly classified in terms of two main frameworks, namely, intrinsically stable schemes (passivity-based), and delay-dependent stability schemes [2]. In early works on constant delay, stability was addressed by means of frequency, Laplace or passivity techniques applied to linear time-invariant (LTI) master-slave two-port systems [1,2,3,4,5].

In regard to transparency properties, the most successful control scheme in achieving full transparency under ideal conditions is the four-channel (4C) control scheme [1,2,4,6]. The specialized literature contains very detailed analyses of transparency and control architectures based on four, three and two channels, that, in general assume constant delays (see especially [4] and references therein).

However, in recent years a new line of research on Internet-based teleoperation [6,7,8,9,10,11] exposes the system control loop to the time-varying delay of a packet-switched network—the subject of a tutorial presented by Hokayem and Spong [12].

Control schemes designed within the passivity framework using scattering and two-port network theory concepts have been improved—by Chopra, Spong and Lozano [13]—by means of an adaptive coordination architecture that provides robust stability against constant delay in network and position tracking. Reset control techniques have also been used to enhance performance [14] and, more recently, passivity-based controllers have also been described that include scattering-based, damping injection and adaptive controllers [15].

Other research lines refer to time-delay compensation to improve teleoperation performance and stability, such as a passivity-based approach to constant delay [16] or methods based on the concept of network disturbance and the communication disturbance observer [9,17,18]. All these theoretical approaches assume a global delay given by τ = τ_m + τ_s (the sum of the delays between master and slave and vice versa) and a Laplace delay block e^−τs representing the global delay, which can commute with the other LTI blocks in the control loop. Therefore, although experimental results for simulated time-varying delay based on these techniques have been described [9,17,18], the stability condition for constant delay cannot apply. The loop must maintain all the time-varying delay channels of the control scheme because these cannot be combined in a single delay: time-varying delays do not commute with LTI systems.

Focusing on Internet-based teleoperation based on delay-dependent control schemes which can provide better transparency properties, the real problem is to establish delay-dependent conditions for systems with two internal control loops (master and slave) ”connected” by (time-varying) delayed signals. This problem can be solved—to name just two possible approaches—by applying delay-dependent stability tools developed for time-domain techniques that are based on Lyapunov-Krasovskii functionals, with a formulation of stability obtained via easily computable LMI conditions (see [6,10,19,20] and references therein), or by using approaches based on small-gain type theorems in the input-to-state stability framework [21], which can be used even when passivity is lost.

More specifically, referring to previous research by us, focusing on the stability of the delay-dependent teleoperation control schemes, we defined a generic approach to modelling any teleoperation setup as a negative single-feedback loop containing an LTI block and an uncertain time-varying delay [11]. The main contribution of this approach was the possibility of deriving frequency-domain conditions for robust stability in the presence of time-varying delays and parametric uncertainties. As a case study, the two-channel position-error (PE) control scheme was modelled, implemented and experimentally tested for the Internet-based haptic teleoperation of a laboratory 3D crane in normal operation conditions, that is, without contact with the environment.

Using this generic approach, we presented preliminary results [22] on the 4C control scheme for teleoperating simulated manipulators, as more flexible than 3C [23] or 2C [24] versions. Further work [25] addressed the robust stability of simulated teleoperated systems, with the 4C architecture affected by time-varying communication delays, by using disturbance observers that are relevant when only position (but not force) measurements are available.

The main differences between our approach [6,22,25] and other delay-dependent proposals are that we look for: (a) complete stability to zero of the whole state; and (b) robustness against continuous-time time-varying delay. The first goal implies that, in the absence of external inputs, the whole state should converge to zero. In particular, the nominal, non-delayed closed-loop system should have well-posed dynamics dx/dt = Ax, with stable closed-loop eigenvalues, Real(Eig(A)) < 0. This excludes schemes where master and slave positions mutually converge x_m → x_s but do not tend to zero. It also excludes cases where the master or the slave ports should be closed by dissipative port terminations to ensure stability. The first goal is important in applications where master or slave devices could be released and we want the resulting free motion to tend to zero (a fixed safe position in the workspace).

It is well-known [4] that 4C schemes with perfect transparency do not have well-posedness and complete stability to zero of the state vector. This is the reason why, starting out with the 4C architecture [6,22,25], we introduced a scaling parameter γ to achieve complete stability to zero of the whole state.

The second goal is to ensure robust stability in Internet-based teleoperation, that is, under time-varying delay. We address the problem of uncertain delays from a continuous-time perspective (not in discrete-time [26,27]) and for time-varying delays (more realistic than uncertain but constant delays [28]). The induced problem is more difficult but, since useful delay encapsulations have been described in the literature [29], we show how to adapt these delay bounds to achieve our second goal of robustness against continuous-time time-varying delay.

The main differences between our previous work and the present work—in other words, the novelty of the present paper—is that here we explicitly discuss ideal transparency issues and rearrange previous solutions [22,25] so that now the resulting controllers are proper, that is, they do not contain single or double differentiators and so avoid problems of noise. We also validated the ideas with simulations and real-life experiments on an Internet teleoperation setup that coordinated two Omni haptic devices between remote laboratory setups in Spain (University of Vigo, Vigo, Spain) and the UK (University of Manchester, Manchester, UK).

The paper is organized as follows: in Section 2 the model for robust stability in 4C-based teleoperation is introduced and in Section 3 a robust stability condition for this control scheme is defined. Section 4 describes the haptic teleoperation case study. Section 5 presents the analysis and simulation results and Section 6 presents the experimental results. Finally, Section 7 concludes the paper with a discussion.

2. Model for Robust Stability in 4C Based Teleoperation

Consider the generic teleoperation setup depicted in Figure 1, consisting of an exchange of signals through a communication channel between the master side (left) and the slave side (right). The master side is formed of the plant 1/P_m and the controller K_m. The plant is modelled as a mechanical system with coordinates (positions) given by the n × 1 vectors x_m, x_s. In the Laplace domain, if $x_m(0)={\dot{x}}_m(0)=0,$ we have:

$$P_m(s)X_m(s):=F_m(s),P_s(s)X_s(s):=F_s(s)$$

(1)

Note that we use lowercase and uppercase letters for time-domain and Laplace domain signals, respectively.

The input of the master plant is the master force $f_m=f_h-f_{mc},$ where f_h is the force applied by the human operator and f_mc is the force from the controller. The plant has two outputs, namely, the n_y signals that send information to the slave (y_m) and the n_z signals used by the local controller (z_m).

On the slave side, f_e is the force from the environment. The controller force f_sc enters additively in $f_s=f_e+f_{sc}.$

Master and slave exchange signals through the communication channel. Taking into account our theoretical [10], and experimental [11] discussions on delay bounds using the Internet User Datagram Protocol (UDP), the delay τ(t) can be assumed in a non-restrictive way from teleoperation point of view, to be an unknown time-varying function for which the upper bounds on the magnitude and the variation satisfy $\forall t\ge 0$:

$$0\le \tau (t)=h+\eta (t)\le h_{\max },\begin{array}{cc}& \left|\eta (t)\right|\end{array}\le \kappa \le h$$

(2)

$$\left|\dot{\tau }(t)\right|\le d<1$$

(3)

The time-varying delay can be described by the operator $𝒟$ which is a (nonlinear) system used (in the time domain) in any of the equivalent notations: $y_{jd}={\mathcal{D}}_j(y_j)={\mathcal{D}}_j\circ y_j={\mathcal{D}}_j y_j, (j=m,s)$. This means that:

$$y_{m{d}_i}(t)={\mathcal{D}}_m{y}_{m_i}(t):=y_m{}_i(t-{\tau }_m(t)),\begin{array}{cc}& y_{s{d}_i}(t)={\mathcal{D}}_s{y}_{s_i}(t):=y_s{}_i(t-{\tau }_s(t))\end{array}$$

(4)

where $𝒟$ _m, $𝒟$ _s are the corresponding master and slave time-delay operators. To simplify matters we also extend the operator $𝒟$ to the Laplace domain in the equivalent forms $Y_{jd}={\mathcal{D}}_j(Y_j)={\mathcal{D}}_j\circ Y_j={\mathcal{D}}_j Y_j$, which simply means that: $Y_{jd}(s)={\mathcal{D}}_j Y_j(s)\iff Y_{jd}(s)=\mathcal{L}\left(y_{jd}(t)\right)$ and $y_{jd}(t)={\mathcal{D}}_j y_j(t)$.

We now consider a 4C control scheme in the most general and representative case of delay-dependent schemes. Here, the master and slave exchange, through the communication channel, velocities or positions and forces in both directions. The local controllers are defined in terms of the applied external force, their own velocity/position (master/slave) and the delayed velocity/position and force (slave/master) from the other side. When the system exchanges positions and forces (see Figure 2), the controllers K_m, K_s can be defined as:

$$f_{mc}=C_m{x}_m+C_2{f}_{sd}+C_4{x}_{sd}-C_6{f}_h$$

(5)

$$f_{sc}=-C_s{x}_s+C_3{f}_{md}+C_1{x}_{md}+C_5{f}_e$$

(6)

Following the procedure described in a previous work [11], we remodeled this whole system to obtain a single feedback loop containing an LTI block and an uncertain time-varying delay block with the same dimension as the number of the signals exchanged between the master and slave. For this purpose, we state the following Proposition 1 to obtain the model for 4C-based teleoperation:

Proposition 1. 

A teleoperation setup as described in Figure 1 and Figure 2, given by Equations (1)–(5), can be formulated as a compact model for robust stability analysis using a generic approach (as developed in [11]) in the form given by equations Equation (7), Equation (8) and the block diagram in Figure 3:

$$\Pi (s)X(s)=-C(s)\mathcal{D}D(s)X(s)+\Phi (s)F(s)$$

(7)

with:

$$\begin{array}{l}\Pi =\left(\begin{array}{cc}{P}_m+C_m& 0\\ 0& P_s+C_s\end{array}\right), X=\left(\begin{array}{c}{X}_m\\ X_s\end{array}\right), C= \left(\begin{array}{cc}{0}_{1\times 2}& \begin{array}{cc}{C}_4& C_2\end{array}\\ \begin{array}{cc}{-\text{C}}_1& {-\text{C}}_3\end{array}& 0_{1\times 2}\end{array}\right),\mathcal{D}=\left(\begin{array}{cc}{I}_2\cdot {\mathcal{D}}_m& 0_{2\times 2}\\ 0_{2\times 2}& I_2\cdot {\mathcal{D}}_s\end{array}\right), \\ \\ \\ D=\left(\begin{array}{cc}{\left(\begin{array}{cc}1& P_m\end{array}\right)}^T& 0_{2\times 1}\\ 0_{2\times 1}& {\left(\begin{array}{cc}1& P_s\end{array}\right)}^T\end{array}\right), \Phi =\left(\begin{array}{cc}1+C_6& 0\\ 0& 1+C_5\end{array}\right), F=\left(\begin{array}{c}{F}_h\\ F_e\end{array}\right)\end{array}$$

(8)

Proof: 

The proof of this proposition is developed in Appendix A. ☐

Remark 1. 

The model described by Equation (7) is directly represented by Figure 3, where four time-varying delays channels inherited from Figure 1 are maintained, bearing in mind that they cannot be combined in a single delay because time-varying delays do not commute with LTI systems.

Ideal Transparency Conditions

Note that, from the teleoperation loop in Figure 3, we can also obtain the zero-delay (D = I ⇒ Y_d = Y) closed loop dynamics for studying ideal behavior and transparency properties, considering:

$$X(s)={\left(\Pi (s)+C(s)D(s)\right)}^{-1}\Phi (s)F(s)=:\Lambda (s)F(s)$$

(9)

That is:

$$\left(\begin{array}{c}{x}_m\\ x_s\end{array}\right)=\Lambda \cdot \left(\begin{array}{c}{f}_h\\ f_e\end{array}\right)\Rightarrow \{\begin{array}{l}{x}_m={\Delta }_{\Lambda }{}^{-1}\cdot ({\Lambda }_{11}{f}_h+{\Lambda }_{12}{f}_e)\\ x_s={\Delta }_{\Lambda }{}^{-1}\cdot ({\Lambda }_{21}{f}_h+{\Lambda }_{22}{f}_e)\end{array}$$

(10)

with the following values:

$$\begin{gathered} {\Lambda }_{11}=\left(P_s+C_s\right)\cdot \left(1+C_6\right), {\Lambda }_{12}=-\left(C_4+C_2{P}_s\right)\cdot \left(1+C_5\right) \\ {\Lambda }_{21}=\left(C_1+C_3{P}_m\right)\cdot \left(1+C_6\right), {\Lambda }_{22}=\left(P_m+C_m\right)\cdot \left(1+C_5\right) \\ {\Delta }_{\Lambda }=\left(P_m+C_m\right)\cdot \left(P_s+C_s\right)+\left(C_1+C_3{P}_m\right)\left(C_4+C_2{P}_s\right) \end{gathered}$$

(11)

The previous admittance matrix definition can be associated with the impedance 𝒵 or hybrid $\mathscr{H}$ models:

$$\left(\begin{array}{c}{f}_h\\ f_e\end{array}\right)=\mathcal{Z}\cdot \left(\begin{array}{c}{x}_m\\ x_s\end{array}\right)\Rightarrow \{\begin{array}{l}{f}_h={\Delta }_Z{}^{-1}\cdot \left({\Lambda }_{22}{x}_m-{\Lambda }_{12}{x}_s\right)\\ f_e={\Delta }_Z{}^{-1}\cdot \left(-{\Lambda }_{21}{x}_m+{\Lambda }_{11}{x}_s\right)\end{array}, {\Delta }_Z=\left(1+C_5\right)\cdot \left(1+C_6\right)$$

(12)

$$\left(\begin{array}{c}{f}_h\\ x_s\end{array}\right)=\mathcal{H}\cdot \left(\begin{array}{c}{x}_m\\ f_e\end{array}\right)\Rightarrow \{\begin{array}{l}{f}_h=h_{11}{x}_m+h_{12}{f}_e\\ x_s=h_{21}{x}_m+h_{22}{f}_e\end{array}$$

(13)

The ideal transparency properties are usually defined on the $\mathscr{H}$ matrix, regarding which:

$$h_{11}=h_{22}=0 transparency, h_{12}\cdot h_{21}=-1 condition set$$

(14)

Comparing Equations (13) and (14) with Equation (9), we obtain:

$$\begin{gathered} h_{11}\to 0 \Rightarrow {\Delta }_{\Lambda }\to 0 , h_{12}=-\frac{{\Lambda }_{12}}{{\Lambda }_{11}}=-1\Rightarrow {\Lambda }_{12}={\Lambda }_{11} \\ h_{21}=\frac{{\Lambda }_{21}}{{\Lambda }_{11}}=1\Rightarrow {\Lambda }_{21}={\Lambda }_{11} , h_{22}=0\Rightarrow {\Lambda }_{11}{\Lambda }_{22}-{\Lambda }_{12}{\Lambda }_{21}=0\Rightarrow {\Lambda }_{22}={\Lambda }_{11} \end{gathered}$$

(15)

Therefore, these properties can also be analyzed through Λ and 𝒵 matrix in the following form:

$$Transparency and Condition set\equiv {\mathcal{Z}}_{ideal}=\frac{1}{{\Delta }_Z}\left(\begin{array}{cc}{\Lambda }_{11}& -{\Lambda }_{11}\\ -{\Lambda }_{11}& {\Lambda }_{11}\end{array}\right), {\Lambda }_{ideal}=\frac{1}{{\Delta }_{\Lambda }}\left(\begin{array}{cc}{\Lambda }_{11}& {\Lambda }_{11}\\ {\Lambda }_{11}& {\Lambda }_{11}\end{array}\right) with {\Delta }_{\Lambda }\to 0$$

(16)

3. Robust Stability for 4C Based Teleoperation

Below we apply our previously developed approach to stability under time-varying delay in teleoperation [11,22] to the 4C control scheme. The time-varying delay is an uncertainty that may give rise to instability. To ensure robust stability, of necessity we have to impose some limits on maximum delay and maximum delay variation. Our treatment of time-varying delay is based on an encapsulation described in the literature [29] and adapted by us to a 4C teleoperation setup. This delay encapsulation [29]—based on the concept of the integral quadratic constraint (IQC), which is a powerful framework for robust stability—is combined with input-output stability criteria and μ-analysis and synthesis techniques in order to achieve a final robust stability condition (Theorem 1).

Proposition 2. 

A teleoperation setup, as stated in Proposition 1 and depicted in Figure 3, can be modeled as a negative single-feedback loop containing an LTI block G(s) and an uncertain time-varying delay block $𝒟$ , shown in Figure 4a, with:

$$G(s)=D(s){\Pi }^{-1}(s)C(s)$$

(17)

$$G=\left(\begin{array}{cc}{0}_{2\times 2}& \begin{array}{cc}{\left(P_m+C_m\right)}^{-1}{C}_4& {\left(P_m+C_m\right)}^{-1}{C}_2\\ P_m{\left(P_m+C_m\right)}^{-1}{C}_4& P_m{\left(P_m+C_m\right)}^{-1}{C}_2\end{array}\\ \begin{array}{cc}-{\left(P_s+C_s\right)}^{-1}{C}_1& -{\left(P_s+C_s\right)}^{-1}{C}_3\\ -P_s{\left(P_s+C_s\right)}^{-1}{C}_1& -P_s{\left(P_s+C_s\right)}^{-1}{C}_3\end{array}& 0_{2\times 2}\end{array}\right)$$

(18)

Theorem 1. 

Consider a 4C-based teleoperation system, modeled by Equations (7) and (8), directly represented by Figure 3 and transformed into Figure 4a by Proposition 2. Given G(s) as defined in Equation (15), let φ(s) [29]:

$$\mathsf{\phi }(s)=k\cdot \frac{h_{\max }^2{s}^2+c\cdot h_{\max }s}{h_{\max }^2{s}^2+a{h}_{\max }s+k\cdot c}$$

(19)

with $k=1+1/\sqrt{1-d},a=\sqrt{2kc}$ where c is any positive real number and delay bounds h_max and where d is as defined in Equations (2) and (3). If necessary, [29] provides extensions of the delay encapsulations by another multipliers φ in Equation (19) that deal with great variability. A sufficient condition for stability of the delayed system as described in Figure 4a, with a complex diagonal structured uncertainty, is given by:

$$\Gamma =\underset{\mathsf{\omega }\in \left[0,\infty \right]}{\max }\overline{\mathsf{\mu }}\left[H(j\mathsf{\omega })\right]=\underset{\mathsf{\omega }\in \left[0,\infty \right]}{\max }\mathsf{\rho }\left[H(j\mathsf{\omega })\right]<1$$

(20)

where:

$$H(s)=:\mathsf{\phi }(s)\cdot {\left(I+G(s)\right)}^{-1}G(s)$$

(21)

and where $\overline{\mathsf{\mu }}(\cdot )$ is the structured singular value (SSV) with respect to a repeated complex scalar uncertainty, which is, in turn, equal to $\mathsf{\rho }(\cdot )$, the spectral radius of a matrix (the maximum of the norms of the eigenvalues).

Proof. 

This result is obtained in two ways. First, in applying the loop transformation theorem (the loop depicted in Figure 4a is stable if the loop depicted in Figure 4b is stable) and the small gain theorem to Figure 4b, whereby${\Vert H(s)\Vert }_{L_2}\cdot {\Vert \Delta \Vert }_{L_2}<1$, since ${\Vert \Delta \Vert }_{L_2}\le 1$ by [29], a sufficient condition is ${\Vert H(s)\Vert }_{L_2}<1$. Second, using the μ-techniques for robust stability [30] we can interpret Figure 4b as a nominal system H(s) connected to a dynamic or complex uncertainty Δ that is actually diagonal (more details are given in [11]). ☐

Remark 2. 

As a prerequisite for robust stability, we must first satisfy nominal stability. Hence, H(s) must be Hurwitz-stable, that is, all the λ_i eigenvalues must verify $real\left({\mathsf{\lambda }}_i\right)<0\begin{array}{cc}& \forall i,i=1\dots n\end{array}$.

This frequency approach based on IQCs combined with μ-analysis and synthesis techniques is a powerful framework for robust stability. Therefore, using the stability condition of Equation (20) based on the system in Equation (21), for any delay satisfying Equations (2) and (3) and for some designed closed-loop (non-delayed) dynamic in G, we can ensure robust stability under time-varying delays.

4. Haptic Teleoperation Case Study

The previous 4C architectural framework was applied to remote teleoperation between master and slave haptic devices located in laboratories in the University of Vigo and the University of Manchester.

Below we describe the details of the setup and of master and slave device identification. We then describe a scaled control scheme γ-4C in which the stability factor γ lets us obtain a practical and internally stable solution to Internet-based teleoperation, where all the controllers are proper (without differentiators), as is usual in classical solutions. Further sections will describe simulations and experimental results.

The master and slave sides consist of two Phantom Omni haptic devices manufactured by SensAble Technologies Inc. (Wilmington, MA, USA). In terms of degree of freedom (DoF), these devices have 6-DoF position sensing and 3-DoF force actuation. To focus on the robust stability issue, we limited movements to 1-DoF—rather than multi-DoF—using the first rotational coordinate (the base rotation) as the free DoF. In this way, the two Omnis rotated around their vertical axes, while the shoulder, elbow and stylus were blocked.

Thus, the master and slave positions were (x_m, x_s), the sensed Omni base angles in radians (rad). The actuations are (f_m, f_s), which are the torque inputs to the motors for vertical axis rotation in machine units (we did not have access to physical units).

We concentrated on 1-DoF movements around a fixed position (the “zero” of the Omnis). Although the Omnis are robotic nonlinear systems, given the linearization principle, it was expected that the local dynamics from forces to positions could be captured as is usual by models in the form G_i(s) = 1/(s(Ms + B)) [31]. Note that here, for simplicity sake, we use the terms “forces” and “positions” to denote torques and angles. Note also that, since the forces are in machine units, the inertia-mass M and friction B coefficients were given in machine or virtual units.

Before identifying the models G_i(s), our experiments showed that there was a deadzone or static friction at the force inputs. This meant that the effective force f_ef was smaller than the applied or written force f_i. This effect can be modelled by a deadzone function f_ef = DZ(f_i; D_pos, D_neg) in the form:

$$f_{ef}=\underset{}{\max \left\{\begin{array}{cc}0,& f_i-D_{pos}\end{array}\right\}} \begin{array}{cc}if& f_i>0,\end{array}{f}_{ef}=\underset{}{\min \left\{\begin{array}{cc}0,& f_i+D_{neg}\end{array}\right\}} \begin{array}{cc}if& f_i<0\end{array}$$

(22)

Although this static friction slightly damaged linearity, producing small steady-state errors, this was easily mitigated by pre-compensation at input by an inverse function in the form f_i = DZ(f; −D_pos, −D_neg) with negative values −D_pos, −D_neg. If the deadzone (asymmetric) widths D_pos, D_neg were estimated well, the composition of these two functions would give rise to the identity, f_ef = f, thus linearizing the system.

After linearizing the devices by deadzone compensation, the linear models G_i(s) = 1/(s(Ms + B)), were obtained, by a standard procedure [24] applied in closed -loop (proportional controller) square and triangular references, recording force inputs and position outputs and performing standard least-squares identifications of M, B. The results (in machine units, m.u.) were:

M = 0.00118 (m.u.) B = 0.0199 (m.u.)

(23)

Therefore, for the teleoperation setup described in Figure 1 formed of two identical OMNI Phantoms as the master and slave systems, for each DoF, the linearized dynamic models were:

$$P_m=M{s}^2+Bs=P_s=P$$

(24)

Seeking a practical solution to Internet-based teleoperation, we fine-tuned a scaled 4C control scheme γ-4C. We also imposed internal stability, that is, the closed-loop system had to be Hurwitz-stable. The operator and environment were modelled as external forces.

These requirements were met in the scheme depicted in Figure 1 and Figure 2 and given by Equations (1)–(5), with master and slave systems as in Equation (24) and with the controllers in Equation (5) defined in the following form:

$$\begin{gathered} For C_m=C_s=K>0 \mathrm{and} \mathsf{\gamma }>1: \\ {C}_4(s)=-\frac{C_s}{\gamma },\begin{array}{cc}& C_1(s)=C_m,\end{array} \begin{array}{cccc}{C}_2=-\frac{1}{\gamma },& C_3=1,& C_5=C_6=K\prime & \end{array} \end{gathered}$$

(25)

The Λ and 𝒵 matrix in this γ-4C scheme with controllers given by Equation (25) take the values:

$$\begin{array}{l}\Lambda (s)=\frac{\left(1+K\prime \right)}{{\Delta }_{\Lambda }}\left(\begin{array}{cc}\left(P+K\right)& \frac{1}{\gamma }\cdot \left(P+K\right)\\ \left(P+K\right)& \left(P+K\right)\end{array}\right),\begin{array}{cc}& {\Delta }_{\Lambda }={\left(P+K\right)}^2\cdot \left(1-\frac{1}{\gamma }\right)\end{array}\\ \\ Z=\frac{1}{{\left(1+K\prime \right)}^2}\left(\begin{array}{cc}\left(P+K\right)& -\frac{1}{\gamma }\left(P+K\right)\\ -\left(P+K\right)& \left(P+K\right)\end{array}\right)\end{array}$$

(26)

Note that the control parameters in Equation (5) with the values selected in Equation (25) were not eight independent controllers. Due to the transparency restriction requirements, there were ultimately only a minimum of two DoFs in the design: the gain K and the coefficient γ (K’ can be zero). The gain K was the only parameter that directly influenced the closed-loop poles (Δ_Λ = 0) in the non-delayed case, as seen in Equation (26). The value k > 0 was necessary to ensure non-delayed system stability. We could thus tune k > 0 from pole-placement principles, bearing in mind that the feedback loop would be affected by a (time-varying) delay. As for the coefficient γ, this ideally should be set to γ = 1 to ensure perfect transparency, but to avoid singularity in the feedback loop impedance (admittance) matrix, it was set to γ > 1. For greater transparency, γ should be as close as 1 as possible, but has to be greater than 1 to ensure robust stability and loop well-posedness.

Remark 3. 

If γ = 1 + ε, with ε → 0, the teleoperation system with the controllers defined by Equation (25) verify the transparency-optimized conditions for zero delay Equation (16). But in the ideal case ε = 0, the impedance matrix Equation (12) loses rank and thus the admittance model Equation (10) is thus not well-defined (ill-posed).

Remark 4. 

To avoid the previous issue, the γ factor in Equation (25) can be selected to verify well-posedness and robust stability conditions under Internet time-varying delay, while maintaining the transparency properties of the system as much as possible. This control scheme therefore gives internal stability and a practical solution to the trade-off between performance and stability under time-varying delay.

Remark 5. 

The proposed control scheme in Figure 1 and Figure 2 as given by Equation (5) allows proper controllers in Equation (25) to be selected based on ideally optimized transparency, that is, they do not contain single or double differentiators like classical 4C solutions and so avoid negative effects with noise. See the explanation in Appendix B.

Remark 6. 

The controllers defined in Equation (25) also provide robust stability and performance against parametric uncertainty. The closed-loop poles are now given by (M_ms² + B_ms + K) (M_ss² + B_ss + K) = 0 and, consequently, for zero delay, the system will be stable iff K, M_m,s = M ± Δ_M, B_m,s = B ± Δ_B >0.

These remarks are further analyzed in the following sections.

5. Analysis and Simulation Results

We first describe a simulation case for use with Simulink and Matlab, considering linear dynamic models for the master and slave, the rounded identified values given by M = 0.001, B = 0.02 and the controllers as stated in Equation (25), with K = 0.1 and K’ = 0.

To justify choosing this control scheme based on performance criteria, using this simulation case and applying zero delay, we compared our scheme with some frequently used two- and three-channel (2C and 3C) control schemes. These schemes can be obtained by canceling certain controller parameters in Equation (25), as shown in

Table 1. Control Schemes (P_m = P_s = P = 0.001s² + 0.02s, C_m = C_s = K = 0.1, C₅ = C₆ = K’ = 0).

	C1	C2	C3	C4	${\mathbf{\Delta }}_{\mathbf{\Lambda }}$	${\mathbf{\Delta }}_{\mathbf{\Lambda }}\mathbf{\Lambda }$
PE	K	0	0	−K	$P^2+2PK$ si = 0, −20, −10 ± 10i	$\left(\begin{array}{cc}P+K& K\\ K& P+K\end{array}\right)$
FP	0	0	1	−K	$P^2+PK+K^2$ si = −18.7 ± 5i, −1.3 ± 5i	$\left(\begin{array}{cc}P+K& K\\ P& P+K\end{array}\right)$
PF	K	−1	0	0	$P^2+PK+K^2$ si = −18.7 ± 5i, −1.3 ± 5i	$\left(\begin{array}{cc}P+K& P\\ K& P+K\end{array}\right)$
3C-i	K	−1	0	−K	$P^2+PK$ si = 0, −20, −10, −10	$\left(\begin{array}{cc}P+K& P+K\\ K& P+K\end{array}\right)$
3C-ii	K	0	1	−K	$P^2+PK$ si = 0, −20, −10, −10	$\left(\begin{array}{cc}P+K& K\\ P+K& P+K\end{array}\right)$
4C (γ = 1.07)	K	−1/γ	1	−K/γ	${\left(P+K\right)}^2\cdot \left(1-\frac{1}{\gamma }\right)$ si = −10, −10, −10, −10	$\left(\begin{array}{cc}P+K& \frac{1}{\gamma }(P+K)\\ P+K& P+K\end{array}\right)$

.

Note that the 4C scheme results were obtained for γ = 1.07 ≈ 1 to avoid singularities. The selected value in K provided an adequate time response by fixing equal and real closed-loop poles, that is, the linear model of the system had critical damping and so did not overshoot.

Performance can be studied by simulations that apply a simulated sine reference force (f_h)—like the force imposed by a human operator—to move the master. The slave will follow the master in free motion in the interval t = (0, 10) and (18, 30) s and with an environment step force f_e simulated from t = 10 s and t = 18 s Note that the operator and environment are not assumed to be in contact or to be passive.

The best results for tracking properties were obtained for the schemes which approximated the ideal transparency conditions (γ-4C) or, at least, the schemes (PE, 3Ci, 3Cii) in which conditions were achieved in the steady state (s → 0). The simulation results, for brevity, are shown only for these schemes (Figure 5, Figure 6, Figure 7 and Figure 8). Only the γ-4C scheme provided internal stability to the system. Because the other schemes have a closed-loop pole at the origin, without external forces, some states of the system tended to the same value (in this case x_m = x_s) but not to zero, for example, PE in Figure 9a. However, the γ-4C scheme in Figure 9b, without external forces, drove the master and slave to their zero states.

Furthermore, it may happen that, for some input forces as used in remote teleoperation, the master and slave positions are not bounded for systems without internal stability, although position tracking is accomplished—for instance, one of the schemes in Figure 10 and the solution given for the 4C scheme in Figure 11.

Finally, referring back to Remark 6, we used the γ-4C controllers defined in Equation (25) with master plant as in Equation (24), except that now the slave model was defined as $P_s=\frac{M}{3}{s}^2+2Bs$. The results can be seen in Figure 12. Due to the fact that the models were different, the master and slave forces—obtained from the external forces and local proportional controllers—were also different in order to accomplish position tracking.

Once the performance of the different schemes was analyzed, we studied stability when the time-varying delay was considered. The idea was to maintain as much as possible the ideal tracking properties obtained by the standard controllers in the 4C control scheme by increasing, by means of the γ factor, the stability margin in practical conditions, that is, when there was time-varying delay in signals transmission.

Once the non-delayed part of the master and slave dynamics were fixed (including the gain K), only γ remained to be chosen. All the delay uncertainty was characterized by two measures: the maximum delay h_max and the maximum delay rate d.

The stability condition in Equation (20) is based on the $H(s)=:\varphi (s)\cdot {\left(I+G(s)\right)}^{-1}G(s)$ system in Equation (21). The closed-loop (non-delayed) system G(s) only depended on γ, and the “multiplier” φ in Equation (19) depended on h_max, d. Thus the condition in Equation (20) took the form $\Gamma \left(\gamma ,h_{\max },d\right)<1$. It is important to emphasize that Equation (20) involves the spectral radius of the frequency response H(jω) of some H(s) depending on (γ, h_max, d) in a complex way. Thus, Equation (20) had to be treated numerically (not analytically).

There was an easy way to deal with Equation (20) numerically: the delay uncertainty parameters (h_max, d) could be discretized in the ranges of interest, and for each discretized value, the minimum γ could be computed. These computations can be represented in the form of 3D surfaces and the designer can use the numerical results to decide, depending on the expected delay uncertainty, the adequate tuning of γ to be included in the controllers.

The stability analysis could be assessed applying Theorem 1 and using Matlab. In all cases it had to be proved, firstly, that the system verified the nominal stability condition given in Equation (22). Here we show, briefly as Figure 13, the effect on stability (from the results on SSV) of different bounds on the delay, fixing the stability factor at γ = 12, which was the value selected for the experiments described in the next section. Note that the delay variation was the most critical aspect of stability. The results depicted in Figure 14 show the minimum value of the stability factor γ (z axis) needed to ensure the stability of the system given the bounds on the delay (h_max, d).

6. Experimental Results

In order to test the above simulation results, real experiments were conducted from the haptic teleoperation case study as described above. The Omni devices were connected to the computer through a FireWire connection. The computers contained the local controllers implemented under Matlab/Simulink and Quarc real-time control software as a real-time machine for the experiments. The Quarc software managed the interface with the haptic device using Quarc Phantom Omni software and also collected and saved all exchanged data. The sampling period in the experiments was set to 1 millisecond. The two devices were remotely coordinated under programmed force profiles like the human and environment external forces, and the experimental data validated the stability and performance results in the theoretical part.

We describe results for three experiments. The first experiment (Figure 16) reproduced real local teleoperation of the Omni devices at the University of Vigo, artificially increasing a time-varying delay between master and slave. The simulation case depicted in Figure 15 is compared with this first experiment as depicted in Figure 16.

The results were very similar to the simulation results, although the difference between the linear identified models used in the simulation case and the OMNI nonlinear dynamic with a pre-compensated deadzone Equation (22) can be appreciated.

The second experiment (Figure 17) reproduced real remote teleoperation between the University of Vigo and the University of Manchester using linear models of master and slave like the corresponding simulation model described in Section 5. Here we could analyze the impact of time delays separately from model uncertainty and nonlinearities issues. The final experiment (Figure 18) tested real remote teleoperation between the master haptic device located at the University of Vigo and the slave haptic device located at the University of Manchester. Note that, for implementation over the haptic devices, the deadzone was pre-compensated at the input Equation (22) by an inverse function in the form explained in Section 4. In general, this function caused the system to be more oscillating than the linear model and to have a small steady-state error.

The data (forces and positions) were exchanged between the master and the slave computers by a transmission of UDP packets. Each packet was composed of time stamps and the transmitted value with a fixed size of 68 bytes. Figure 19 shows some experimental data for connections via Internet between Vigo and Manchester. The delay was measured in terms of the round trip time (RTT) communication. The maximum delay was near h_max = 0.028 s, with very low variation tested with a transmission rate of 1000 packets per second (Figure 19a). The test over the communication channel also revealed that variations in time delay never exceeds d = 0.45, as can be seen in Figure 19b. It should be noted that the communication network used in the experiment belongs to the GÉANT pan-European research and education network, which has very good quality linkage between facilities.

7. Conclusions

In this paper we have extended our generic framework to stability under time-varying delay [11] to scaled 4C or γ-4C teleoperation. The application of the theoretical framework to the four channel case, with time-varying delays, gave rise to an encapsulated delay uncertainty, size 4 × 4 and diagonal. Assuming symmetric delays, the related structured singular value or μ-value was equal to the spectral radius of the linear subsystem. The approach represents an appealing frequency technique to deal with the stability robustness of the architecture.

The main novelty of the present research compared to previous related research is that we have explicitly discussed the problems associated with ideal transparency and that we have rearrange previous 4C solutions [22,25] so that the resulting controllers are proper, that is, they do not contain single or double differentiators and so avoid problems of noise. We have also validated this research through new simulations and real-life experiments using an Internet teleoperation setup that coordinates two Omni haptic devices between remote laboratory setups in Spain (University of Vigo) and the UK (University of Manchester).

The experiments achieved remote device coordination under programmed force profiles. This required resolving a large number of practical issues related to hardware, software and communications. Some important practical problems were related to linearized model identification and to deadzone or static friction in the Omni devices, which had to be pre-compensated. Finally, the architecture was fine-tuned and successfully tested, obtaining complete stability to zero of the whole state and robustness against continuous-time time-varying delay for Internet-based teleoperation. The real and simulated experiments give similar results, thus validating the models. Stable remote teleoperation also performed adequately in terms of position synchronization and force transparency.

This research has several possible extensions to future work. First, time-varying delay uncertainty was treated here by means of encapsulation [29], but exploring other novel approaches to uncertain time delays could improve the robustness analysis, for example, Lyapunov-Krasovskii functionals [32]. The generic framework based on IQCs [33] is also a potentially powerful tool in addressing teleoperation robustness. Second, one of the main difficulties encountered was the problem of deadzones as, although the coarse values of the (asymmetrical) deadzone widths can easily be estimated, the related static friction does not repeat well from one experiment to another. Since deadzone is a static monotone bounded uncertainty, it could potentially be addressed by means of the IQC approach that uses Zames-Falb multipliers to deal with robust stability [34].