Image-Based Visual Servoing for Three Degree-of-Freedom Robotic Arm with Actuator Faults

Abstract

This study presents a novel image-based visual servoing fault-tolerant control strategy aimed at ensuring the successful completion of visual servoing tasks despite the presence of robotic arm actuator faults. Initially, a depth-independent image-based visual servoing model is established to mitigate the effects of inaccurate camera parameters and missing depth information on the system. Additionally, a robotic arm dynamic model is constructed, which simultaneously considers both multiplicative and additive actuator faults. Subsequently, model uncertainties, unknown disturbances, and coupled actuator faults are consolidated as centralized uncertainties, and an iterative learning fault observer is designed to estimate them. Based on this, suitable sliding surfaces and control laws are developed within the super-twisting sliding mode visual servo controller to rapidly reduce control deviation to near zero and circumvent the chattering phenomenon typically observed in traditional sliding mode control. Finally, through comparative simulation between different control strategies, the proposed method is shown to effectively counteract the effect of actuator faults and exhibit robust performance.

1. Introduction

Visual servoing is a vision-based robotic control method that allows the robot to interact with its environment by capturing images using a camera and extracting image feature information to achieve closed-loop control. Visual servoing technology enables robots to fully perceive their task environment and has been widely researched and applied in areas such as target grasping and tracking [1,2,3]. Depending on the number and types of cameras employed, visual servoing can be categorized into monocular visual servoing [4], binocular visual servoing [5], and multi-camera visual servoing systems [6]. According to the different image features used by robots, visual servoing can be divided into position-based visual servoing (PBVS) [7,8,9], image-based visual servoing (IBVS) [10,11,12], and hybrid visual servoing (HVS) [13,14,15]. PBVS employs 3-DOF position information to achieve visual servo control of a robot. In contrast, IBVS utilizes 2-DOF coordinates of the image feature points to achieve closed-loop control. HVS, on the other hand, integrates feedback information from both 2-DOF and 3-DOF. Compared to PBVS, IBVS can circumvent the issue of the target object leaving the camera’s field of view. When contrasted with HVS, IBVS can avoid complex calculations, thereby enhancing the control system’s real-time performance. Moreover, IBVS exhibits greater tolerance for robot modelling errors, camera calibration errors, and image noise. As a result, IBVS demonstrates superior control performance in non-structured control scenarios [16]. This study proposes a novel fault-tolerant control approach for IBVS.

In classical IBVS controllers, accurate camera intrinsic and extrinsic matrices are crucial for achieving precise control. However, obtaining accurate calibration parameters is often infeasible in most scenarios. To overcome this limitation, the uncalibrated visual servo technique has been proposed, which does not require accurate calibration parameters and uses image feature motion during movement to estimate the robot’s motion in real-time, providing good control performance in unknown and complex environments [17,18]. Numerous studies have been conducted to enhance the servoing efficiency and robustness of uncalibrated visual servoing [19,20,21,22,23,24,25]. Norouzi-Ghazbi et al. proposed an IBVS method based on model predictive control for visual servoing of continuum robots constrained by visual and actuator constraints, achieving endpoint control in the presence of constraints and improving the effectiveness and robustness of the system [19]. A constraint visual servoing method based on a sliding mode observer and model predictive controller was proposed in [20], where the sliding mode observer was designed to observe model uncertainty and joint velocities of the manipulator, and the model predictive controller converted the IBVS task into a nonlinear optimization problem and fully considered torque and visibility constraints, demonstrating the control performance of the proposed method in 2-DOF and 6-DOF IBVS systems. A homography-based unmanned aerial vehicle (UAV) visual servoing method was proposed in [21], where a saturation model-generated page control input was introduced into the robust controller to ensure that image feature points remained in the camera’s region of interest, and the asymptotic stability of the closed-loop system was proven. The authors of [22,23] investigated the estimation problem of the image interaction matrix in IBVS. In [22], a robust Kalman filter visual servoing controller based on long short-term memory was proposed, which utilized the long short-term memory state estimation of the image interaction matrix and improved the initialization accuracy of the Jacobian matrix. In [23], the bidirectional extreme learning machine algorithm was adopted to approximate the image interaction matrix, exhibiting strong robustness to camera calibration errors and image noise. In [24,25], the authors adopted the reinforcement learning algorithm to adaptively tune visual servoing gains, effectively improving visual servoing efficiency compared to the traditional method with a fixed gain. Aghili proposed a novel hierarchical control visual servoing method based on adaptive Kalman filtering, considering the scenario where the visual system is temporarily occluded and completely fails. This method addresses both physical and operational constraints, achieving the capture of free-floating objects [26].

Although the aforementioned studies have made progress in achieving stable control of visual servoing, they all assume that the robot maintains normal operation during movement and overlook the possibility of robot failures. Due to long-term and repetitive work, the robot’s actuators may experience degradation or even damage. To enhance the reliability and stability of visual servoing, it is necessary to consider the impact of actuator faults on visual servoing performance [27]. Fault-tolerant control can be broadly classified into two categories, which are active fault-tolerant control and passive fault-tolerant control. Active fault-tolerant control compensates for the robot’s faults by detecting the fault information and providing the corresponding control signal through a fault-tolerant controller, making it suitable for high-precision control scenarios [28]. In contrast, passive fault-tolerant control considers the entire system and introduces fault-tolerance mechanisms to bring reliability [29]. Vision-based control of robotic arms is often applied in scenarios such as sorting and target capturing, which demands high precision of the control system. Therefore, the active fault-tolerant control method is more suitable for addressing the visual servoing problem of a robotic arm with actuator faults. Regarding fault detection methods for time-varying actuator faults, refs. [30,31] adopted the adaptive Radial Basis Function (RBF) neural network-based state observer to estimate the fault information. Le and Kang unified model uncertainties, disturbance torques, and fault information as centralized uncertainties and established an extended state observer to estimate the uncertainties [32]. A high-order sliding mode observer that improves convergence speed by adding a linear term was proposed in [33] to estimate joint fault status, and Lyapunov theory was used to demonstrate the system’s finite-time stability. In designing fault-tolerant controllers for robotic arms, fuzzy logic controller [34], neural network controller [35], and sliding mode controller [36,37,38,39] have been adopted to input appropriate joint torques. Among them, the sliding mode control method has been widely studied due to its advantages of a simple design process and strong robustness. To overcome the disadvantages of the traditional sliding mode controller, such as low convergence speed and control signal oscillation, terminal sliding mode, fast terminal sliding mode, and non-singular fast terminal sliding mode methods have been developed [36,37]. In sliding mode control, problems such as load disturbance and slow system response due to discontinuity of control input often occur at the beginning of the system. To address these issues, refs. [38,39] introduced the super-twisting method to provide additional control force, improving the system’s response speed and control accuracy.

Guided by existing achievements, this study proposes an active fault-tolerant control method for a vision-based robotic arm, which is capable of handling actuator faults. The main contributions of this study are as follows:

(1)

Compared to [30], this study simultaneously considers the impact of both multiplicative and additive actuator faults on the system and reconstructs model uncertainties, load disturbances, and actuator faults as centralized uncertainties. The iterative learning fault observer (ILFO) is proposed to accurately and quickly estimate time-varying uncertainties. The dynamic relationship between the image feature point and the robotic arm is established, and super-twisting sliding mode visual servo controller (STSMVSC) with suitable sliding surfaces and control laws is designed to provide appropriate input torques to complete the visual servoing trajectory tracking task. Moreover, the stability analysis of the proposed observer and controller is provided using the Lyapunov theory.

(2)

In the simulation, the effectiveness and robustness of the proposed fault-tolerant control method are demonstrated under different coupled actuator fault severities. Compared to the fault-tolerant control method based on the extended state observer and STSMVSC, the proposed method can more quickly and accurately approach centralized uncertainties. Compared to the fault-tolerant control method based on ILFO and a traditional sliding mode visual servo controller, the proposed method can effectively reduce the chattering phenomenon and provide superior tracking performance.

The subsequent structure of the study is as follows: Section 1 presents the kinematic and dynamic models of visual servoing with simultaneous multiplicative and additive actuator faults. Section 2 introduces the iterative learning observer and the super-twisting sliding mode controller, along with the analysis of stability. Comparative simulations are presented in Section 3 to verify the effectiveness, generalization capability, and robustness of the proposed method. Finally, Section 4 summarizes the work presented in this study.

2. Problem Formulation

An IBVS system that experiences actuator faults can be described in Figure 1. The camera perceives the pose of image feature points in real-time and feeds them back to the controller. Based on the current pose of the feature points and the desired pose, the controller generates control signals that drive the motion of the robotic arm, allowing the control error to converge to an acceptable range. In this section, an IBVS model is developed, along with a dynamics model for a robotic arm experiencing actuator faults.

2.1. Image-Based Visual Servoing

In IBVS, the control error occurs in the 2D image plane, while the 2D coordinates of feature points are obtained from their Cartesian coordinates through a perspective projection model. The image coordinates of the feature point $s_i={\left(\begin{array}{cc}{m}_i& n_i\end{array}\right)}^T$ can be represented as follows:

$$s_i=\left(\begin{array}{c}\hfill m_i\\ \hfill n_i\end{array}\right)=\frac{1}{D_z}\left(\begin{array}{c}\hfill u_1^T\\ \hfill u_2^T\end{array}\right)T\left(\begin{array}{c}\hfill p_i\\ \hfill 1\end{array}\right),$$

(1)

where $m_i$ and $n_i$ represent the coordinates of the feature point projected onto the m- and n-axes of the image plane, respectively; $D_z$ represents the depth value of the feature points; $u_i^T$ denotes the ith row vector of the unknown camera intrinsic and extrinsic matrix $U\in {\Re }^{3\times 4}$; $T\in {\Re }^{4\times 4}$ denotes the homogeneous transformation matrix, which is up to the forward kinematics of the robotic arm. T is given by $T=\left(\begin{array}{cc}{R}_T& P_T\\ 0_{1\times 3}& 1\end{array}\right),$ where $R_T\in {\Re }^{3\times 3}$ and $P_T\in {\Re }^{3\times 1}$ represent the rotation matrix and translation matrix, respectively; $p_i={\left(\begin{array}{ccc}{X}_i& Y_i& Z_i\end{array}\right)}^T$ denotes the Cartesian coordinates of the feature point [40]. By taking the time derivative of both sides of Equation (1), the dynamic relationship between the velocity of the feature point and the robotic arm can be obtained:

$$\begin{array}{cc}\hfill {\dot{s}}_i=& \frac{1}{{D_z}^2}\left(\left(\begin{array}{c}\hfill u_1^T\\ \hfill u_2^T\end{array}\right)T\left(\begin{array}{c}\hfill {\dot{p}}_i\\ \hfill 1\end{array}\right)\dot{\upsilon }{D}_z-{\dot{D}}_z\left(\begin{array}{c}\hfill u_1^T\\ \hfill u_2^T\end{array}\right)T\left(\begin{array}{c}\hfill p_i\\ \hfill 1\end{array}\right)\right)\hfill \\ \hfill =& \frac{1}{D_z}\left(\begin{array}{c}\hfill u_1^T\\ \hfill u_2^T\end{array}\right)T\left(\begin{array}{c}\hfill {\dot{p}}_i\\ \hfill 1\end{array}\right)\dot{\upsilon }-\frac{{\dot{D}}_z}{D_z}\left(\begin{array}{c}\hfill u_1^T\\ \hfill u_2^T\end{array}\right)T\left(\begin{array}{c}\hfill p_i\\ \hfill 1\end{array}\right),\hfill \end{array}$$

(2)

where $\dot{\upsilon }$ denotes joint velocities of the robotic arm. The depth $D_z$ can be represented as follows:

$$D_z=u_3^{T}T\left(\begin{array}{c}\hfill p_i\\ \hfill 1\end{array}\right).$$

(3)

Taking the time derivative of Equation (3) yields

$${\dot{D}}_z=u_3^{T}T\left(\begin{array}{c}\hfill {\dot{p}}_i\\ \hfill 1\end{array}\right)\dot{\upsilon }.$$

(4)

By substituting Equation (4) into Equation (2), Equation (2) can be simplified to

$$\begin{array}{cc}\hfill {\dot{s}}_i=& \frac{1}{D_z}\left(\begin{array}{c}\hfill m_i-u_1^T\\ \hfill n_i-u_2^T\end{array}\right)T\left(\begin{array}{c}\hfill {\dot{p}}_i\\ \hfill 1\end{array}\right)\dot{\upsilon }\hfill \\ \hfill =& \frac{1}{D_z}J\dot{\upsilon },\hfill \end{array}$$

(5)

where matrix J represents the depth-independent image interaction matrix that does not contain depth information.

The depth-independent image interaction matrix J and the depth of feature point $D_z$ are nonlinear. However, they can be represented linearly with regressor matrices and unknown parameter vectors by the following property.

Property 1. 

For any vector χ, the products $J\chi$ and $D_z\chi$ can be parameterized in a linear form as

$$J\chi =A(\chi ,\upsilon ,s)\vartheta ,$$

(6)

$$D_z\chi =B(\chi ,s)\vartheta ,$$

(7)

where $A(\chi ,\upsilon ,s)$ and $B(\chi ,s)$ are the regressor matrices. ϑ is the corresponding parameter vector determined by the products of the camera parameters and the robot kinematic parameters.

2.2. Faulty Robotic Arm Dynamic Model

The dynamic equation of a rigid robotic arm with n degrees of freedom is typically described as follows:

$$W\left(\upsilon \right)\ddot{\upsilon }+C\left(\upsilon ,\dot{\upsilon }\right)\dot{\upsilon }+K\left(\upsilon \right)+F_r\left(\upsilon ,\dot{\upsilon }\right)={\tau }_r-{\tau }_{dt},$$

(8)

where $\upsilon \in {\Re }^n,\dot{\upsilon }\in {\Re }^n,\ddot{\upsilon }\in {\Re }^n$ represent the joint angle, angular velocity, and angular acceleration of the robotic arm, $W\left(\upsilon \right)\in {\Re }^{n\times n}$ denotes the symmetric and positive definite inertia matrix, $C\left(\upsilon ,\dot{\upsilon }\right)\in {\Re }^{n\times n}$ denotes the skew-symmetric matrix of the Coriolis and centripetal forces, $K\left(\upsilon \right)\in {\Re }^{n\times 1}$ denotes the gravity matrix, $F_r\left(\upsilon ,\dot{\upsilon }\right)\in {\Re }^{n\times 1}$ denotes the friction matrix, ${\tau }_r\in {\Re }^{n\times 1}$ denotes the joints torque, and ${\tau }_{dt}\in {\Re }^{n\times 1}$ denotes the disturbance matrix. When there exist uncertainties in the dynamics of the robotic arm, Equation (8) is expressed as follows:

$$\begin{array}{cc}\hfill & \left(W_n\left(\upsilon \right)+\Delta W\left(\upsilon \right)\right)\ddot{\upsilon }+\left(C_n\left(\upsilon ,\dot{\upsilon }\right)+\Delta C\left(\upsilon ,\dot{\upsilon }\right)\right)\dot{\upsilon }\hfill \\ \hfill & +\left(K_n\left(\upsilon \right)+\Delta K\left(\upsilon \right)\right)+F_r\left(\upsilon ,\dot{\upsilon }\right)={\tau }_r-{\tau }_{dt},\hfill \end{array}$$

(9)

where $W_n\left(\upsilon \right),C_n\left(\upsilon ,\dot{\upsilon }\right),K_n\left(\upsilon \right)$ denotes the known dynamics, $\Delta W\left(\upsilon \right),\Delta C\left(\upsilon ,\dot{\upsilon }\right),\Delta K\left(\upsilon \right)$ denotes the unknown dynamics. Harsh environments and long-term operations can cause robotic arms to experience actuator multiplicative and additive faults. However, this paper does not focus on a specific type of actuator fault in robotic arms but adopts a more general actuator fault model. The paper considers the coupled actuator faults that include both multiplicative and additive faults, and models the fault torque as [41]

$${\tau }_r^{*}=\left(I_n-\omega \right){\tau }_r+{\tau }_a,$$

(10)

where ${{\tau }_r}^{*}\in {\Re }^{n\times 1}$ denotes the degraded torque, $\omega \in diag\left({\omega }_1,{\omega }_2,\cdots ,{\omega }_n\right)$ denotes the remaining control factor, and ${\tau }_a\in {\Re }^{n\times 1}$ denotes the additive fault torque. By substituting Equation (10) into Equation (9), the dynamic equation of robot manipulation with actuator fault can be expressed as follows:

$$\begin{array}{cc}\hfill & \left(W_n\left(\upsilon \right)+\Delta W\left(\upsilon \right)\right)\ddot{\upsilon }+\left(C_n\left(\upsilon ,\dot{\upsilon }\right)+\Delta C\left(\upsilon ,\dot{\upsilon }\right)\right)\dot{\upsilon }\hfill \\ \hfill & +\left(K_n\left(\upsilon \right)+\Delta K\left(\upsilon \right)\right)+F_r\left(\upsilon ,\dot{\upsilon }\right)=\left(I_n-\omega \right){\tau }_r+{\tau }_a-{\tau }_{dt}.\hfill \end{array}$$

(11)

The above equation can be simplified as follows:

$$\begin{array}{cc}\hfill & W_n\left(\upsilon \right)\ddot{\upsilon }+C_n\left(\upsilon ,\dot{\upsilon }\right)\dot{\upsilon }+K_n\left(\upsilon \right)\hfill \\ \hfill =& {\tau }_r-\omega {\tau }_r+{\tau }_a-{\tau }_{dt}+\Delta W\left(\upsilon \right)\ddot{\upsilon }-\Delta C\left(\upsilon ,\dot{\upsilon }\right)\dot{\upsilon }\hfill \\ \hfill & -\Delta K\left(\upsilon \right)-F_r\left(\upsilon ,\dot{\upsilon }\right)\hfill \\ \hfill =& {\tau }_r-\Phi \left(\upsilon ,\dot{\upsilon }\right),\hfill \end{array}$$

(12)

where $\Phi \left(\upsilon ,\dot{\upsilon }\right)$ denotes the concentrated uncertainty term containing model uncertainty, frictional torque, and fault torque. Then, the dynamics of the robot manipulation can be expressed as follows:

$$\ddot{\upsilon }={W_n}^{-1}\left(\upsilon \right)\left({\tau }_r-\Phi \left(\upsilon ,\dot{\upsilon }\right)-C_n\left(\upsilon ,\dot{\upsilon }\right)\dot{\upsilon }-K_n\left(\upsilon \right)\right).$$

(13)

3. Fault-Tolerant Visual Servo Control

3.1. Iterative Learning Observer

To facilitate the design of state observers and fault-tolerant visual servo controllers, the following state variables are defined:

$$x\left(t\right)=\left[\begin{array}{c}\hfill x_1\left(t\right)\\ \hfill x_2\left(t\right)\end{array}\right]=\left[\begin{array}{c}\hfill \upsilon \left(t\right)\\ \hfill \dot{\upsilon }\left(t\right)\end{array}\right],$$

(14)

$$y\left(t\right)=Cx\left(t\right),$$

(15)

$$u\left(t\right)={\tau }_r\left(t\right).$$

(16)

Then, Equation (13) can be rewritten as the following state-space equation:

$$\left\{\begin{array}{c}\hfill \begin{array}{c}\hfill \dot{x}\left(t\right)=Ax\left(t\right)+B\left(x\left(t\right),u\left(t\right)\right)+D\left(x\left(t\right)\right)+E\left(x\left(t\right)\right)\Phi \left(t\right)\end{array}\\ \hfill y\left(t\right)=Cx\left(t\right),\end{array}\right.$$

(17)

where $A=\left[\begin{array}{cc}\hfill 0_n& I_n\hfill \\ \hfill 0_n& 0_n\hfill \end{array}\right]$,

$B\left(x\left(t\right),u\left(t\right)\right)=\left[\begin{array}{c}\hfill 0_n\\ \hfill {W_n}^{-1}\left(x_1\left(t\right)\right)u\left(t\right)\end{array}\right]$,

$C=I_{2n}$,

$D\left(x\left(t\right)\right)=\left[\begin{array}{c}\hfill 0_n\\ \hfill {W_n}^{-1}\left(x_1\left(t\right)\right)\left(-C_n\left(\upsilon ,\dot{\upsilon }\right)\dot{\upsilon }-K_n\left(\upsilon \right)\right)\end{array}\right]$,

$E\left(x\left(t\right)\right)=\left[\begin{array}{c}\hfill 0_n\\ \hfill -{W_n}^{-1}\left(x_1\left(t\right)\right)\end{array}\right]$,

Where $0_n$ represents an $n\times n$ zero matrix, where all elements are zeros. $I_n$ represents an $n\times n$ identity matrix, which has ones on the diagonal and zeros elsewhere, $I_{2n}$ represents an $2n\times 2n$ identity matrix, which has ones on the diagonal and zeros elsewhere.

To accurately estimate the aggregated uncertainties, design the following iterative learning fault observer:

$$\left\{\begin{array}{c}\begin{array}{c}\dot{\widehat{x}}\left(t\right)=A\widehat{x}\left(t\right)+B(\widehat{x}\left(t\right),u\left(t\right))+D\left(\widehat{x}\left(t\right)\right)+G(y\left(t\right)-\widehat{y}\left(t\right))+E\zeta \left(t\right)\end{array}\\ \zeta \left(t\right)=r_1\zeta (t-\kappa )+r_2[y\left(t\right)-\widehat{y}\left(t\right)]\widehat{y}\left(t\right)=C\widehat{x}\left(t\right)\end{array},\right.$$

(18)

where $\widehat{x}\left(t\right)$ denotes the state estimation, $\widehat{y}\left(t\right)$ denotes the output estimation, $\kappa$ denotes the sampling time, $\zeta \left(t\right)$ denotes the iterative observer input at time t, $\zeta \left(t-\kappa \right)$ denotes the iterative observer input from the previous sampling period, and $G,r_1,r_2$ are the observer gain matrices.

Remark 1. 

In practical applications, the velocity state of a robotic arm often cannot be directly measured; typically, only joint angle information is available. In this study, the ILFO is designed to accurately and quickly estimate unmeasurable velocity states, as well as other time-varying uncertainties and actuator faults. By employing an iterative learning mechanism, this observer can continuously update its estimates in a continuous-time model, thereby improving the dynamic estimation accuracy and response speed of the system.

Remark 2. 

The system dynamics are inherently continuous, as described by the state estimation $\widehat{x}\left(t\right)$ and the output estimation $\widehat{y}\left(t\right)$ in Equation (18). These continuous-time expressions form the core of the system’s behavior. The variable κ denotes this sampling interval, representing the discrete time steps at which the iterative learning updates occur. $\zeta \left(t-\kappa \right)$ refers to the iterative observer input from the previous sampling period. This discrete update mechanism is integrated into the continuous-time model to leverage the benefits of both continuous-time system dynamics and discrete-time learning adjustments. Thus, the update of $\zeta \left(t\right)$ occurs at discrete intervals, while the overall system operates in continuous time.

Define the estimation error as follows:

$$\left\{\begin{array}{c}\hfill e_x\left(t\right)=x\left(t\right)-\widehat{x}\left(t\right)\\ \hfill e_y\left(t\right)=y\left(t\right)-\widehat{y}\left(t\right)\\ \hfill \tilde{\zeta }\left(t\right)=\Phi \left(t\right)-\zeta \left(t\right)\end{array}.\right.$$

(19)

Substituting Equations (17) and (18) into Equation (19), the dynamic equation for the state observation error can be expressed as follows:

$$\begin{array}{cc}\hfill {\dot{e}}_x\left(t\right)=& \dot{x}\left(t\right)-\dot{\widehat{x}}\left(t\right)\hfill \\ \hfill =& A\left(x\left(t\right)-\widehat{x}\left(t\right)\right)\hfill \\ \hfill & +\left(B\left(x\left(t\right),u\left(t\right)\right)-B\left(\widehat{x}\left(t\right),u\left(t\right)\right)\right)\hfill \\ \hfill & +\left(D\left(x\left(t\right)\right)-D\left(\widehat{x}\left(t\right)\right)\right)+E\left(x\left(t\right)\right)\Phi \left(t\right)\hfill \\ \hfill & -G\left(y\left(t\right)-\widehat{y}\left(t\right)\right)-E\left(x\left(t\right)\right)\zeta \left(t\right)\hfill \\ \hfill =& \left(A-GC\right)e_x\left(t\right)\hfill \\ \hfill & +\left(B\left(x\left(t\right),u\left(t\right)\right)-B\left(\widehat{x}\left(t\right),u\left(t\right)\right)\right)\hfill \\ \hfill & +\left(D\left(x\left(t\right)\right)-D\left(\widehat{x}\left(t\right)\right)\right)+E\left(x\left(t\right)\right)\left(\Phi \left(t\right)-\zeta \left(t\right)\right)\hfill \\ \hfill =& \left(A-GC\right)e_x\left(t\right)\hfill \\ \hfill & +\left(B\left(x\left(t\right),u\left(t\right)\right)-B\left(\widehat{x}\left(t\right),u\left(t\right)\right)\right)\hfill \\ \hfill & +\left(D\left(x\left(t\right)\right)-D\left(\widehat{x}\left(t\right)\right)\right)+E\left(x\left(t\right)\right)\tilde{\zeta }\left(t\right).\hfill \end{array}$$

(20)

First, the assumption and theorem for the proposed iterative learning sliding mode observer are as follows:

Assumption 1. 

Assume that $B\left(x\left(t\right),u\left(t\right)\right)$ and $D\left(x\left(t\right)\right)$ are bounded, and there exists Lipschitz constants $o_1,o_2$ such that the following inequality holds:

$$\begin{array}{c}\hfill ∥B\left(x\left(t\right),u\left(t\right)\right)-B\left(\widehat{x}\left(t\right),u\left(t\right)\right)∥\le o_1∥x\left(t\right)-\widehat{x}\left(t\right)∥,\\ \hfill ∥D\left(x\left(t\right)\right)-D\left(\widehat{x}\left(t\right)\right)∥\le o_2∥x\left(t\right)-\widehat{x}\left(t\right)∥.\end{array}$$

(21)

Theorem 1. 

For the robot arm dynamics Equation (17) and the iterative learning observer Equation (18), there exist symmetric matrices $P\in {\Re }^{2n\times 2n},Q\in {\Re }^{2n\times 2n}$ that satisfy the following equation:

$${\left(A-GC\right)}^{T}P+P\left(A-GC\right)+2o_{1}P+2o_{2}P=-Q,$$

(22)

and satisfying Assumption 1 can make the errors of the state and concentrated uncertainty estimation consistently asymptotically stable.

Proof. 

Define the Lyapunov equation as follows:

$$V={e_x}^T\left(t\right)P{e}_x\left(t\right)+{\int }_{t-\kappa }^t{\tilde{\zeta }}^T\left(s\right)\tilde{\zeta }\left(s\right)ds.$$

(23)

Taking the derivative of the above equation and substituting Equation (20) yields

$$\begin{array}{cc}\hfill \dot{V}=& {e_x}^T\left(t\right)\left({\left(A-GC\right)}^{T}P+P\left(A-GC\right)\right)e_x\left(t\right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)P\left[D\left(x\left(t\right)\right)-D\left(\widehat{x}\left(t\right)\right)\right]\hfill \\ \hfill & +2{e_x}^T\left(t\right)P\left[B\left(x\left(t\right),u\left(t\right)\right)-B\left(\widehat{x}\left(t\right),u\left(t\right)\right)\right]\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE\tilde{\zeta }\left(t\right)\hfill \\ \hfill & +{\tilde{\zeta }}^T\left(t\right)\tilde{\zeta }\left(t\right)-{\tilde{\zeta }}^T\left(t-\kappa \right)\tilde{\zeta }\left(t-\kappa \right).\hfill \end{array}$$

(24)

Based on Equation (21), it can be obtained that

$$\begin{array}{cc}\hfill \dot{V}\le & {e_x}^T\left(t\right)\left(\begin{array}{c}\hfill {\left(A-GC\right)}^{T}P+P\left(A-GC\right)\\ \hfill +2P{o}_1+2P{o}_2\end{array}\right)e_x\left(t\right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE\tilde{\zeta }\left(t\right)+{\tilde{\zeta }}^T\left(t\right)\tilde{\zeta }\left(t\right)\hfill \\ \hfill & -{\tilde{\zeta }}^T\left(t-\kappa \right)\tilde{\zeta }\left(t-\kappa \right).\hfill \end{array}$$

(25)

Construct the auxiliary variable as follows:

$${\zeta }_d\left(t\right)=\Phi \left(t\right)-r_1\Phi \left(t-\kappa \right).$$

(26)

Then,

$$\tilde{\zeta }\left(t\right)=r_1\tilde{\zeta }\left(t-\kappa \right)-r_{2}C\widehat{x}\left(t\right)+{\zeta }_d\left(t\right),$$

(27)

$$\begin{array}{cc}\hfill \dot{V}\le & {e_x}^T\left(t\right)\left(\begin{array}{c}\hfill {\left(A-GC\right)}^{T}P+P\left(A-GC\right)\\ \hfill +2P{o}_1+2P{o}_2\end{array}\right)e_x\left(t\right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE{r}_1\tilde{\zeta }\left(t-\kappa \right)-2{e_x}^T\left(t\right)PE{r}_{2}C\widehat{x}\left(t\right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE{\zeta }_d\left(t\right)\hfill \\ \hfill & +{\tilde{\zeta }}^T\left(t\right)\tilde{\zeta }\left(t\right)-{\tilde{\zeta }}^T\left(t-\kappa \right)\tilde{\zeta }\left(t-\kappa \right).\hfill \end{array}$$

(28)

By introducing positive constants $\alpha ,\beta$ that satisfy $\alpha -\beta =1$, the above equation can be expressed as follows:

$$\begin{array}{cc}\hfill \dot{V}\le & {e_x}^T\left(t\right)\left(\begin{array}{c}\hfill {\left(A-GC\right)}^{T}P+P\left(A-GC\right)\\ \hfill +2P{o}_1+2P{o}_2\end{array}\right)e_x\left(t\right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE{r}_1\tilde{\zeta }\left(t-\kappa \right)-2{e_x}^T\left(t\right)PE{r}_{2}C\widehat{x}\left(t\right)\hfill \\ \hfill & +\alpha {\tilde{\zeta }}^T\left(t\right)\tilde{\zeta }\left(t\right)-\beta {\tilde{\zeta }}^T\left(t\right)\tilde{\zeta }\left(t\right)-{\tilde{\zeta }}^T\left(t-\kappa \right)\tilde{\zeta }\left(t-\kappa \right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE{\zeta }_d\left(t\right).\hfill \end{array}$$

(29)

Substituting Equation (27) into the above equation yields

$$\begin{array}{cc}\hfill \dot{V}\le & {e_x}^T\left(t\right)\left(\begin{array}{c}\hfill {\left(A-GC\right)}^{T}P+P\left(A-GC\right)\\ \hfill +2P{o}_1+2P{o}_2\end{array}\right)e_x\left(t\right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE{r}_1\tilde{\zeta }\left(t-\kappa \right)-2{e_x}^T\left(t\right)PE{r}_{2}C\widehat{x}\left(t\right)\hfill \\ \hfill & +2{e_x}^T\left(t\right)PE{\zeta }_d\left(t\right)\hfill \\ \hfill & -\beta {\tilde{\zeta }}^T\left(t\right)\tilde{\zeta }\left(t\right)-{\tilde{\zeta }}^T\left(t-\kappa \right)\tilde{\zeta }\left(t-\kappa \right)\hfill \\ \hfill & +\alpha {{\zeta }_d}^T\left(t\right){\zeta }_d\left(t\right)+2\alpha {\tilde{\zeta }}^T\left(t-\kappa \right){r_1}^T{\zeta }_d\left(t\right)\hfill \\ \hfill & -2\alpha {e_x}^T\left(t\right){\left(r_{2}C\right)}^T{\zeta }_d\left(t\right)\hfill \\ \hfill & +\alpha {\tilde{\zeta }}^T\left(t-\kappa \right){r_1}^T{r}_1\tilde{\zeta }\left(t-\kappa \right).\hfill \end{array}$$

(30)

If both Equation (22) and the following relationship are satisfied

$$PE=\alpha {\left(r_{2}C\right)}^T,$$

(31)

it yields

$$\begin{array}{cc}\hfill \dot{V}\le & -{\lambda }_{min}\left(Q\right){∥e_x\left(t\right)∥}^2-\beta {∥\tilde{\zeta }\left(t\right)∥}^2\hfill \\ \hfill & -{\tilde{\zeta }}^T\left(t-\kappa \right)\tilde{\zeta }\left(t-\kappa \right)\hfill \\ \hfill & +\alpha {\tilde{\zeta }}^T\left(t-\kappa \right){r_1}^T{r}_1\tilde{\zeta }\left(t-\kappa \right)\hfill \\ \hfill & +2\alpha {\tilde{\zeta }}^T\left(t-\kappa \right){r_1}^T{\zeta }_d\left(t\right)\hfill \\ \hfill & +\alpha {{\overline{\zeta }}_d}^2+2\mu ∥P∥{∥e_x\left(t\right)∥}^2.\hfill \end{array}$$

(32)

Using the following inequality property:

$$\begin{array}{c}\hfill 2\alpha {\tilde{\zeta }}^T\left(t-\kappa \right){r_1}^T{\zeta }_d\left(t\right)\le l^2{\tilde{\zeta }}^T\left(t-\kappa \right){r_1}^T{r}_1{\zeta }_d\left(t\right)\\ \hfill +\frac{{\alpha }^2}{l^2}{{\zeta }_d}^T\left(t\right){\zeta }_d\left(t\right).\end{array}$$

(33)

Then,

$$\begin{array}{cc}\hfill \dot{V}\le & -{\lambda }_{min}\left(Q\right){∥e_x\left(t\right)∥}^2-\beta {∥\tilde{\zeta }\left(t\right)∥}^2\hfill \\ \hfill & +{\tilde{\zeta }}^T\left(t-\kappa \right)\left[\left(\alpha +l^2\right){r_1}^{T}r-I\right]\tilde{\zeta }\left(t-\kappa \right)\hfill \\ \hfill & +\left(\alpha +\frac{{\alpha }^2}{l^2}\right){{\overline{\zeta }}_d}^2+2\mu {\lambda }_{max}\left(P\right){∥e_x\left(t\right)∥}^2.\hfill \end{array}$$

(34)

By defining $\nu ={\lambda }_{min}\left(Q\right)-2\mu {\lambda }_{max}\left(P\right)>0$, it yields

$$\begin{array}{cc}\hfill \dot{V}\le & -\nu {∥e_x\left(t\right)∥}^2-\beta {∥\tilde{\zeta }\left(t\right)∥}^2+\left(\alpha +\frac{{\alpha }^2}{l^2}\right){{\overline{\zeta }}_d}^2\hfill \\ \hfill & +{\tilde{\zeta }}^T\left(t-\kappa \right)\left[\left(\alpha +l^2\right){r_1}^T{r}_1-I\right]\tilde{\zeta }\left(t-\kappa \right).\hfill \end{array}$$

(35)

If condition $I>\left(\alpha +l^2\right){r_1}^T{r}_1>0$ holds, then the iterative observer is uniformly ultimately stable, and the proof is complete. □

3.2. Fault-Tolerant Sliding Mode Controller

The control system diagram is shown in Figure 2. The objective of this section is to design a fault-tolerant IBVS controller that drives a robotic arm to converge the image feature points to zero within a finite time, given the initial position and the desired trajectory of the image feature points, after receiving the estimated fault values from the observer. To overcome the drawbacks of traditional sliding mode control, such as control signal chattering and low control accuracy, a super-twisting sliding mode controller is designed in this section. In sliding mode control, when the system state moves along the sliding surface $\sigma =0$, it indicates that the actual state variables have completely tracked the target state variables. Before designing the sliding surface, we first define the system state variables.

The tracking error is defined as

$$e_s=s_{dt}-s_{ct},$$

(36)

where $s_{dt}$ represents the desired trajectory of the feature points, $s_{ct}$ represents the current coordinates of the feature points, as given by Equation (1). Based on Equations (1), (5) and (13), the first and second derivatives of the tracking error concerning time can be obtained

$${\dot{e}}_s={\dot{s}}_{dt}-{\dot{s}}_{ct},$$

(37)

$${\ddot{e}}_s={\ddot{s}}_{dt}-{\ddot{s}}_{ct}=\Gamma \ddot{\upsilon }+\dot{\Gamma }\dot{\upsilon }-{\ddot{s}}_{ct},$$

(38)

where $\Gamma =\frac{1}{D_z}J$.

Define the sliding surface as follows:

$$\sigma ={\dot{e}}_s+\delta e_s,$$

(39)

where $\delta$ is a positive constant that needs to be designed. After completing the design of the sliding surface, the next step is to design a suitable control law to make $\dot{\sigma }=0$, which means making the system trajectory converge to the sliding surface. According to the state-space equation, Equations (17) and (37)–(39), the derivative of the sliding surface is expressed as

$$\begin{array}{cc}\hfill \dot{\sigma }=& {\ddot{e}}_s+\delta {\dot{e}}_s\hfill \\ \hfill =& {\ddot{s}}_{dt}-{\ddot{s}}_{ct}+\delta {\dot{e}}_s\hfill \\ \hfill =& \Gamma \ddot{\upsilon }+\dot{\Gamma }\dot{\upsilon }-{\ddot{s}}_{ct}+\delta {\dot{e}}_s\hfill \\ \hfill =& \Gamma {W_n}^{-1}\left(\upsilon \right)\left({\tau }_r-\Phi \left(\upsilon ,\dot{\upsilon }\right)-C_n\left(\upsilon ,\dot{\upsilon }\right)\dot{\upsilon }-K_n\left(\upsilon \right)\right)\hfill \\ \hfill & +\dot{\Gamma }\dot{\upsilon }-{\ddot{s}}_{ct}+\delta {\dot{e}}_s\hfill \\ \hfill =& \Gamma {W_n}^{-1}\left(x_1\left(t\right)\right)u-\Gamma {W_n}^{-1}\left(x_1\left(t\right)\right)\Phi \left(\upsilon ,\dot{\upsilon }\right)\hfill \\ \hfill & -\Gamma {W_n}^{-1}\left(x_1\left(t\right)\right)\left(C_n\left(\upsilon ,\dot{\upsilon }\right)\dot{\upsilon }+K_n\left(\upsilon \right)\right)\hfill \\ \hfill & +\dot{\Gamma }\dot{\upsilon }-{\ddot{s}}_{ct}+\delta {\dot{e}}_s.\hfill \end{array}$$

(40)

In this study, we employ the use of the sign function, defined as follows:

$$\mathrm{sign}\left(\sigma \right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\sigma >0\\ 0\hfill & \mathrm{if}\sigma =0\\ -1\hfill & \mathrm{if}\sigma <0\end{array}.\right.$$

(41)

The visual servoing control law is designed as follows:

$$u=u_1+u_2,$$

(42)

where $u_1$ is used to control the nominal term, expressed as

$$\begin{array}{c}\hfill u_1=C_n\left(\upsilon ,\widehat{\dot{\upsilon }}\right)\widehat{\dot{\upsilon }}+K_n\left(\upsilon \right)-W_n\left(\upsilon \right){\Gamma }^{-1}\left(\dot{\Gamma }\widehat{\dot{\upsilon }}-{\ddot{s}}_{ct}+\delta {\dot{e}}_s\right),\end{array}$$

(43)

where $\widehat{\dot{\upsilon }}$ represents the joint angular velocities estimated by ILFO. $u_2$ is used to control the effects caused by model uncertainties, unknown disturbances and faults, expressed as

$$\begin{array}{cc}\hfill & u_2=W_n\left(\upsilon \right){\Gamma }^{-1}\left({\mu }_1{∥\sigma ∥}^{1/2}\mathrm{sign}\left(\sigma \right)-\beta \right)+\zeta \left(t\right)\hfill \\ \hfill & \dot{\beta }=-{\mu }_2\mathrm{sign}\left(\sigma \right).\hfill \end{array}$$

(44)

Substituting Equations (42)–(44) into Equation (40), we obtain

$$\begin{array}{cc}\hfill & \dot{\sigma }=-{\mu }_1{∥\sigma ∥}^{1/2}\mathrm{sign}\left(\sigma \right)+\beta +\lambda \left(t\right)\hfill \\ \hfill & \dot{\beta }=-{\mu }_2\mathrm{sign}\left(\sigma \right),\hfill \end{array}$$

(45)

where $\lambda \left(t\right)\le \overline{\lambda }$ is a bounded element related to unknown disturbances, $\overline{\lambda }$ is the upper bound. When the robot model given by Equation (13) experiences a fault, by selecting an appropriate control gain, the visual servoing tracking error can be made to converge to zero in a finite time when the sliding mode surface is determined by Equation (39) and the control rate is determined by Equations (42)–(44). A stability-proof method for a trajectory tracking controller for a robotic arm was proposed in [42]. Although it is not directly related to the problem discussed in this paper, it can be applied to the stability proof of the visual servoing controller proposed in this paper. Therefore, the stability proof of the controller is not separately provided in this paper.

4. Numerical Simulations

In this section, an IBVS trajectory tracking task is set up to validate the effectiveness, robustness, and generalization capability of the proposed method when the robotic arm experiences actuator faults. The 3-DOF robotic arm used in the simulation has parameters as shown in [43], and only the first three joints are used. The camera is fixed to the end-actuator of the robotic arm. The real values and roughly estimated values of the camera parameters are shown in

Table 1. Camera parameters.

Parameters	Real Value	Rough Estimated Value
$m_0$ (pixels)	640	600
$n_0$ (pixels)	512	500
${\alpha }_m$ (pixels/m)	265,000	260,000
${\alpha }_n$ (pixels/m)	266,500	260,000

, where ${\alpha }_m$ and ${\alpha }_n$ represent the scale factors along the m- and n- axes, respectively. The focal length of the camera is set to 0.005 m. To ensure system synchronization, the camera’s image capture speed was set to 25 frames per second, and the control system’s sampling time was set to 40 ms.

In the simulation, friction forces and disturbance torques are taken into consideration and are set as follows:

$$\begin{array}{c}\hfill F_r=\left[\begin{array}{c}\hfill F_{r1}\\ \hfill F_{r2}\\ \hfill F_{r3}\end{array}\right]=\left[\begin{array}{c}\hfill 0.5{\dot{\upsilon }}_1\\ \hfill 0.8{\dot{\upsilon }}_2\\ \hfill 1.2{\dot{\upsilon }}_3\end{array}\right],\\ \hfill {\tau }_{dt}=\left[\begin{array}{c}\hfill {\tau }_{dt1}\\ \hfill {\tau }_{dt2}\\ \hfill {\tau }_{dt3}\end{array}\right]=\left[\begin{array}{c}\hfill 0.1sin\left({\upsilon }_1\right)\\ \hfill 0.15sin\left({\upsilon }_2\right)\\ \hfill 0.2sin\left({\upsilon }_3\right)\end{array}\right].\end{array}$$

(46)

To demonstrate that the proposed method is not limited to specific actuator fault parameters, two sets of robotic arm actuator fault parameters are given in the simulation to validate the stability and generalization capability of the proposed method.

Case 1 

The simulation time is set to 50 s, with actuator faults occurring at the 20th, 30th and 40th seconds, respectively. The input torque with faults, as shown in Equation (10), is described as

$${\tau }_r^{*}=\left[\begin{array}{c}\hfill {\tau }_{r1}^{*}\\ \hfill {\tau }_{r2}^{*}\\ \hfill {\tau }_{r3}^{*}\end{array}\right]=\left[\begin{array}{c}\hfill 0.8{\tau }_{r1}-1\\ \hfill 0.7{\tau }_{r2}\\ \hfill {\tau }_{r3}-1\end{array}\right],t\in \left(20,30,40\right).$$

(47)

Case 2 

The simulation time is set to 50 s, with a time-varying actuator fault persisting after the 20th second. The input torque with faults, as shown in Equation (10), is described as

$${\tau }_r^{*}=\left[\begin{array}{c}\hfill {\tau }_{r1}^{*}\\ \hfill {\tau }_{r2}^{*}\\ \hfill {\tau }_{r3}^{*}\end{array}\right]=\left\{\begin{array}{c}\hfill \left[\begin{array}{c}\hfill {\tau }_{r1}\\ \hfill {\tau }_{r2}\\ \hfill {\tau }_{r3}\end{array}\right],t\in \left[0,20\right)\\ \hfill \left[\begin{array}{c}\hfill 0.8{\tau }_{r1}\\ \hfill \begin{array}{c}\hfill \left(0.75-0.1sin\left(t\right)\right){\tau }_{r2}\\ \hfill -0.5sin\left(t\right)\end{array}\\ \hfill \left(0.8+0.1cos\left(t\right)\right){\tau }_{r3}-1\end{array}\right],t\in \left[20,50\right].\end{array}\right.$$

(48)

Furthermore, to demonstrate the effectiveness and superiority of the proposed method, the comparative simulation between the proposed method and the control strategies based on the extended state observer (ESO) [44] with super-twisting sliding mode visual servo controller (ESO-STSMVSC), and the control strategies based on the iterative learning fault observer with traditional sliding mode visual servo controller [38] (ILFO-SMVSC). The control objective is to make the image feature point track the desired trajectory.

$s_d={\left[400+90sin\left(0.3t\right),400+90cos\left(0.3t\right)\right]}^T$ from the initial coordinates $s_{initial}={\left[400,260\right]}^T$.

The parameters of ILFO corresponding to each degree of freedom are designed as $G=\left[\begin{array}{cc}20.38& 12.53\\ -8.3& 22.51\end{array}\right],$ $r_1=diag\left[\begin{array}{cc}0.85& 0.9\end{array}\right],r_2=diag\left[\begin{array}{cc}20.5& 30\end{array}\right].$ The parameters of STSMVSC are designed as ${\mu }_1=2.15$ and ${\mu }_2=2.5.$

The comparison results of trajectory tracking are illustrated in Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10. Figure 3 shows the trajectories of the image feature point under Case 1, with the blue pentagram representing the initial position of the feature point, and red and green curves representing the desired and real trajectories, respectively. It can be observed that, under all three control strategies, the feature points can track the desired trajectories from their initial positions. Figure 4 demonstrates the changes in tracking errors along the m- and n- axes. It is evident that, in the initial stage of tracking, compared to ILFO-SMVSC, both ILFO-STSMVSC, and ESO-STSMVSC enable faster convergence of image deviations. When actuator faults occur, compared to ESO-STSMVSC, both ILFO-STSMVSC and ILFO-SMVSC can more rapidly mitigate the impact of actuator faults on the system, driving the tracking error to converge into a small residual around zero. This indicates that the proposed method outperforms the other two control strategies concerning both error convergence speed and mitigating actuator fault disturbances.

Figure 5 presents a comparison of concentrated uncertainty estimation performance for different observers under Case 1. The ILFO can dynamically update estimation errors based on the sample time, offering higher estimation accuracy and faster convergence of estimation errors compared to ESO. Due to the presence of the iterative learning factor $\zeta \left(t\right)$, the observed values of the fault state can be adjusted in real-time during the observation process, enhancing the dynamic estimation accuracy of the observer, which helps improve the system’s response speed and robustness. At the same time, by iteratively updating the historical data, the observer improves the estimation accuracy of the system state and output. The input $\zeta \left(t\right)$ in the iterative process is adjusted according to the iterative observer input $\zeta \left(t-\kappa \right)$ from the previous sampling period, gradually optimizing the observer’s performance. Moreover, the observer can accurately estimate the state and output, enhancing the ability to detect system uncertainties and faults. Compared to ESO, ILFO requires relatively fewer parameters to be adjusted during implementation and can automatically optimize parameters through the iterative process, making the deployment and debugging of ILFO less challenging.

Figure 6 illustrates the comparison of input torques for the robotic arm under different controllers in Case 1. It can be observed that the control laws given by Equations (42)–(44) reduce the impact of switching gain on system performance, effectively mitigating the chattering phenomenon associated with the traditional sliding mode. Consequently, this leads to smoother control signals and an overall improvement in control quality. Figure 3, Figure 4, Figure 5 and Figure 6 confirm that, under Case 1, the proposed controller demonstrates better control performance compared to ESO-STSMVSC and ILFO-SMVSC. The proposed fault observer and visual servo controller also exhibit superior performance.

Figure 7, Figure 8, Figure 9 and Figure 10 presents the comparative simulation results under Case 2. Figure 7 and Figure 8 indicate that, under time-varying actuator faults, the proposed control strategy offers superior tracking effects compared to ESO-STSMVSC and ILFO-SMVSC. In Figure 9, it can be observed that the estimation accuracy and speed of the proposed ILFO surpass those of ESO. Figure 10 displays the input torques under different control strategies, and similar to Case 1, the proposed controller effectively avoids the chattering phenomenon of the controller based on traditional sliding mode control.

Figure 11 and Figure 12 show the velocities of the robotic arm’s end-effector under the proposed control scheme in Case 1 and Case 2. It can be observed that the velocity of the end-effector changes smoothly. After being disturbed by a fault, the end-effector quickly moves to bring the system back to a stable tracking state. The above comparative simulations demonstrate the effectiveness, stability, and generalization capabilities of the proposed method under different time-varying actuator faults. The proposed method can effectively accomplish visual servo trajectory tracking tasks in the presence of actuator faults.

5. Conclusions

This paper proposes an IBVS control strategy that utilizes an iterative learning observer and a super-twisting sliding mode controller for a robotic arm’s visual servo system in the presence of actuator faults. In contrast to existing research, the impact of actuator faults on robust visual servoing control is taken into account. The model uncertainty, load disturbance, and coupling actuator fault are reconstructed as centralized uncertainty, and the ILFO is designed to estimate it. The dynamic relationship between the image feature point and the robotic arm is established, and the STSMVSC is designed to mitigate the chattering phenomenon commonly observed in traditional sliding mode control and enhance convergence speed. Employing the Lyapunov theory, the stability of both the observer and controller is ensured. The stability and generalization capability of the proposed method under various actuator fault conditions are validated through simulation. The comparative simulation results indicate that, when faced with various actuator faults, the proposed method demonstrates superior servo efficiency and tracking accuracy compared to other IBVS approaches. Moreover, it effectively mitigates the impact of concentrated uncertainty on the system. In summary, the proposed method can accomplish uncalibrated visual servo trajectory tracking tasks even in the presence of actuator faults.