Optimal Shortcuts to Adiabatic Control by Lagrange Mechanics

Abstract

We combined an inverse engineering technique based on Lagrange mechanics and optimal control theory to design an optimal trajectory that can transport a cartpole in a fast and stable way. For classical control, we used the relative displacement between the ball and the trolley as the controller to study the anharmonic effect of the cartpole. Under this constraint, we used the time minimization principle in optimal control theory to find the optimal trajectory, and the solution of time minimization is the bang-bang form, which ensures that the pendulum is in a vertical upward position at the initial and the final moments and oscillates in a small angle range.

1. Introduction

In the past few decades, with the development of experimental technology, the unstable control of nonlinear unstable systems [1,2,3,4,5] by a moving controller has attracted more and more attention. As far as we know, a sufficiently slow adiabatic process is a very simple way to achieve the ideal transfer, i.e., without excitations or losses [6,7,8,9]. However, a sufficiently slow process is problematic, firstly a sufficiently slow process, is impractically long and, secondly, over a long period of time, the ideal dynamics may be subject to the accumulation of random and/or uncontrollable perturbations that can undermine the expected results. In some experiments and theories, it is necessary to speed up this transport process but also to ensure that there is no excitation or loss at the initial and final moments [10,11,12,13,14,15,16,17]. To solve these problems, “Shortcuts To Adiabaticity” (STA) technology was proposed decades ago [18]. It achieves the same results as the adiabatic process in a short time [19]. At present, STA technology has been successfully applied to atoms, molecules, optical physics, solid state physics, chemistry, engineering, quantum systems, and classical systems [12,20,21,22,23,24,25]. With the development of STA technology, the STA technique includes many different schemes, such as the quantum invariant-based inverse control method [26,27,28,29], the reverse thermal transfer compensation method [30,31,32], the no-jump quantum drive method [33,34,35], etc. These methods have been validated in a number of experiments.

Cartpole is a typical multivariate, higher-order, nonlinear, strongly coupled, naturally unstable system. The stable control of the cartpole system is a typical problem in control theory. Here, we used the inverse engineering method, based on Lagrange mechanics, for a fast but stable control of the cartpole. By fixing the boundary conditions at the initial and final times, we aimed to find the trajectories that meet the boundary conditions [19,36] but that also achieves transition-free evolution ideally in a shorter time. Later, this approach can be complemented by optimal control theory [37,38], where we used the time minimization criterion in the Pontryagin maximization principle. Pontryagin’s maximum (or minimum) principle [39] is used in optimal control theory to find the best possible control for taking a dynamical system from one state to another, especially in the presence of constraints for the state or input controls. It was formulated in 1956 by the Russian mathematician Lev Semenovich Pontryagin and his students. The main thing is to find a controller so that the control Hamiltonian of the system reaches the maximum (or minimum) and then determines the optimal trajectory of the system. In this article, we used the relative displacement of the ball and the trolley horizontally as a controller, and then used the principle to find the optimal trajectory.

2. Physical Model

Here, we studied an ideal straight type cartpole system model; its schematic diagram and related parameters are shown in Figure 1. The trolley moves from $x_0=0$ and $x_0=d$, swinging the pole as far as possible in a vertically upward position at the initial and final moments. Never a perfect harmonic in actual transmission, similar to the transport potential of approximate optical tweezers, the quartic anharmonic plays a very important role, as shown in Figure 1b. We aimed to combine the inverse engineering method [26,27,28,29] with the optimal control theory in the Pontryagin maximization principle [37,38]. By setting boundary conditions to suppress the final excitation, we used the relative distance between the ball and the trolley as a controller throughout the process, to achieve the time minimization.

The physical model and related parameters are shown in Figure 1, where M indicates the mass of the trolley, l is the length of the non-elongated rod, m is the mass of the ball, $\theta$ is the angle between the pole and the vertical direction, $x_0$ represents the position of the trolley over time, and x represents the position of the ball. We assumed that the model is an ideal model by setting the following conditions: (i) the quality and friction of the rod can be ignored; (ii) point masses; (iii) constant rod length l; (iv) the trolley position is treated as a control parameter rather than a dynamical variable.

3. Hamiltonian of the System and Inverse Engineering

The solution of the cartpole system can be solved by Lagrange mechanics. According to the parameters provided in Figure 1, the Cartesian coordinate of mass in a rest frame are given by:

$$\begin{array}{c}x=x_0-lsin\theta ,\\ y=lcos\theta ,\end{array}$$

(1)

where $x_0$ represents the position of the trolley. The kinetic energy and potential energy of the system can be expressed as follows:

$$\begin{array}{c}T=\frac{1}{2}M{\dot{x}}_0^2+\frac{1}{2}m\left({\dot{x}}^2+{\dot{y}}^2\right),\hfill \\ V=mgy,\hfill \end{array}$$

(2)

where the dots represent time derivatives. It can be obtained by substituting Equation (1) into Equation (2),

$$\begin{array}{c}T=\frac{1}{2}(M+m){\dot{x}}_0^2+\frac{1}{2}m{l}^2{\dot{\theta }}^2-m{\dot{x}}_{0}l\dot{\theta }cos\theta ,\hfill \\ V=mglcos\theta .\hfill \end{array}$$

(3)

Thus, the Lagrangian $L=T-V$ of the system can be written as:

$$L=\frac{1}{2}(M+m){\dot{x}}_0^2+\frac{1}{2}m{l}^2{\dot{\theta }}^2-m{\dot{x}}_{0}l\dot{\theta }cos\theta -mglcos\theta .$$

(4)

Based on the Euler–Lagrange equation $\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{\theta }}\right)-\frac{\partial L}{\partial \theta }=0$, we can obtain the corresponding dynamics equation as follows:

$$\ddot{\theta }=\frac{g}{l}sin\theta +\frac{{\ddot{x}}_0}{l}cos\theta .$$

(5)

In order to reverse engineer the trajectory of the trolley, we need to calculate the Hamiltonian of the system, according to $p_{\theta }=\frac{\partial L}{\partial \dot{\theta }}$, where $p_{\theta }$ is the momentum with $\theta$ as the generalized coordinate. So, we need the conjugate momentum of $\theta$,

$$p_{\theta }=m{l}^2{\dot{\theta }}^{}-m{\dot{x}}_{0}lcos\theta .$$

(6)

Thus, the Hamiltonian $H_{\theta }=\dot{\theta }{p}_{\theta }-L$ can be obtained from the Lagrangian:

$$\begin{array}{c}{H}_{\theta }=\frac{1}{2}m{l}^2{\dot{\theta }}^2-\frac{1}{2}(M+m){\dot{x}}_0^2+mglcos\theta .\hfill \end{array}$$

(7)

Using the canonical transformation approach, change the system from $\theta$ to q as the coordinate, and the Hamiltonian is rewritten as (please refer to Appendix A for details):

$$H=\frac{p^2}{2m}+mgl-\frac{1}{2}m{\omega }^2{q}^2-\frac{1}{8l^2}m{\omega }^2{q}^4-\frac{1}{2}(M+m){\dot{x}}_0^2-m{\ddot{x}}_{0}q,$$

(8)

where $\omega =\frac{g}{l}$ is the natural frequency of the pendulum, p represents the momentum with q as the generalized coordinate, $q=x_0-x$. According to the canonical transformation [40], we can obtain:

$$\begin{array}{c}\dot{q}=\frac{p}{m},\hfill \\ \dot{p}=m{\omega }^{2}q+\frac{m{\omega }^2}{2l^2}{q}^3+m{\ddot{x}}_0,\hfill \end{array}$$

(9)

from which the dynamic equation of the system becomes

$$\ddot{q}-{\omega }^{2}q-\frac{{\omega }^2}{2l^2}{q}^3={\ddot{x}}_0.$$

(10)

This can be written as:

$$\ddot{x}+{\omega }^2(x_0-x)+\frac{{\omega }^2}{2l^2}{(x_0-x)}^3=0.$$

(11)

To fix the distance, we assumed that the trolley is transported from $x_0\left(0\right)=0$ to $x_0\left(t_f\right)=d$. We can set suitable boundary conditions for $x\left(t\right)$ according to our expectations, and based on these boundary conditions the trolley trajectory $x_0\left(t\right)$ can be inverse engineered [26,27,28,29] to see how the position of the mass changes over time. We imposed the conditions as follows:

$$\begin{array}{c}{x}_{}\left(0\right)=0\begin{array}{cc},& \end{array}{x}_{}\left(t_f\right)=d,\hfill \\ {\dot{x}}_{}\left(0\right)=0\begin{array}{cc},& \end{array}{\dot{x}}_{}\left(t_f\right)=0,\hfill \\ {\ddot{x}}_{}\left(0\right)=0\begin{array}{cc},& \end{array}{\ddot{x}}_{}\left(t_f\right)=0.\hfill \end{array}$$

(12)

In general, there are different options from which to choose the trajectories fulfilling the boundary conditions. For instance, one can choose the polynomial ansatz to interpolate the function. In the following section, we would like to complement it with time-optimal control for this problem.

4. Optimal Control for Time Minimization

In this section, we try to find the time minimization by using the Pontryagin maximum principle [37,38]. To be consistent, we set new notations for the Equation (11) of motion of the system:

$$y_1=x\begin{array}{cc},& \end{array}{y}_2={\dot{y}}_1=\dot{x}\begin{array}{cc},& \end{array}u\left(t\right)=x_0-x,$$

(13)

such that (where $u\left(t\right)$ is the controller, representing the relative displacement of the ball to the trolley in the horizontal direction; x is the dynamic variable, representing the position of the ball)

$$y_2={\dot{y}}_1,$$

(14)

$${\dot{y}}_2=-{\omega }^{2}u-\frac{{\omega }^2}{2l^2}{u}^3.$$

(15)

For this system, the boundary conditions for $y_1$, $y_2$ and $u\left(t\right)$ can be given by Equations (11) and (12), and $u\left(t\right)$ is required to satisfy $u\left(t\right)=0$ at $t\le 0$ and $t\ge t_f$. From the optimal control theory, one can find $\left|u\left(t\right)\right|\le \delta$ for a fixed bound $\delta$ and minimize the cost function J. The Pontryagin maximum principle finally provides the necessary conditions for optimal control [37,38].

For the relative displacement between the ball and the trolley, we utilized optimal control theory to find the time-optimal trajectory [38], so we defined the cost function as: $J={\int }_0^{t_f}dt=t_f$. In this manner, we defined the control Hamiltonian as:

$$H_c(\mathit{p}\left(t\right),\mathit{y}\left(t\right),u)=p_{0}g(\mathit{y}\left(t\right),u)+{\mathit{p}}^T·\mathit{f}(\mathit{y}\left(t\right),u).$$

(16)

For almost all $0\le t\le t_f$, the function $H_c(\mathit{p}\left(t\right),\mathit{y}\left(t\right),u)$ attains its maximum at $u=u^{*}$ and $H_c(\mathit{p}\left(t\right),\mathit{y}\left(t\right),u^{*})=c$, where c is constant, and $\dot{\mathit{y}}=\mathit{f}(\mathit{y}\left(t\right),u)$. According to the cost function, the control Hamiltonian can be written as:

$$\begin{array}{c}{H}_c=p_0+p_1{y}_2+p_2[-{\omega }^{2}u-\frac{{\omega }^2}{2l^2}{u}^3],\hfill \end{array}$$

(17)

from which the canonical equations are

$${\dot{p}}_i=-\frac{\partial H_c}{\partial y_i},$$

(18)

$${\dot{y}}_i=\frac{\partial H_c}{\partial p_i}.$$

(19)

Accordingly, we can obtain the following costate equations by way of Equation (18):

$$\begin{array}{c}{\dot{p}}_1=0,\hfill \\ {\dot{p}}_2=-p_1.\hfill \end{array}$$

(20)

We can easily get $p_1=c_1$ and $p_2=-c_{1}t+c_2$, where $c_1$, $c_2$ are constant. In order to maximize the control of the Hamiltonian, we can obtain the optimal $u\left(t\right)$ as a form of bang-bang control,

$$u\left(t\right)=\left\{\begin{array}{cc}\begin{array}{c}0,\\ -\delta ,\\ \delta ,\\ 0,\end{array}& \begin{array}{c}t\le 0\\ 0<t<t_1\\ t_1<t<t_f\\ t\ge t_f\end{array}\end{array}\right.,$$

(21)

as shown in Figure 2, with only one intermediate switching time at $t_1$. Since the controller $u\left(t\right)$ is in the form of a switch, the trajectory of the trolley $x_0\left(t\right)$ changes abruptly at $t=0^{+}$. By substituting $u\left(t\right)$ into Equation (11) and combining the boundary conditions in Equation (12), we eventually obtain the trajectory of $x\left(t\right)$,

$$x\left(t\right)=\left\{\begin{array}{cc}\begin{array}{c}0,\\ \frac{1}{2}{\omega }^2\delta t^2+\frac{1}{4l^2}{\omega }^2{\delta }^3{t}^2,\\ d-\frac{1}{2}{\omega }^2\delta {(t-t_f)}^2-\frac{1}{4l^2}{\omega }^2{\delta }^3{(t-t_f)}^2,\\ d,\end{array}& \begin{array}{c}t\le 0\\ 0<t<t_1\\ t_1<t<t_f\\ t\ge t_f\end{array}\end{array}\right.$$

(22)

According to $u\left(t\right)=x_0-x$, the trajectory of the trolley $x_0\left(t\right)$ can be represented as:

$$x_0\left(t\right)=\left\{\begin{array}{cc}\begin{array}{c}0,\\ \frac{1}{2}{\omega }^2\delta t^2+\frac{1}{4l^2}{\omega }^2{\delta }^3{t}^2-\delta ,\\ d-\frac{1}{2}{\omega }^2\delta {(t-t_f)}^2-\frac{1}{4l^2}{\omega }^2{\delta }^3{(t-t_f)}^2+\delta ,\\ d,\end{array}& \begin{array}{c}t\le 0\\ 0<t<t_1\\ t_1<t<t_f\\ t\ge t_f\end{array}\end{array}\right.$$

(23)

Solving the system of Equations (14) and (15), we find the switching time $t_1$ and final time $t_f$ as follows:

$$t_1=\frac{t_f}{2},$$

(24)

$$t_f=\frac{2l}{\omega }\sqrt{\frac{2d}{2l^2\delta +{\delta }^3}}.$$

(25)

Again, the trajectory of the trolley and the trajectory of the swing ball are shown in Figure 3. Through the above control, the ball can be oscillated in a small angle range.

After finding the optimal trajectory, we brought it into the exact Equation (5) and approximate dynamic equations. Clearly, short times imply larger transient angles, as shown in Figure 4. In addition, for the long time, we can see that there is basically no big difference in the change of angle over time, but the difference is noticeable only for the smaller time, $t_f=3\mathrm{s}$. The main reason is that if we ignore the higher-order terms, the approximation deviates from the reality. At the same time, because the $\delta$ is the relative displacement between the ball and the trolley, the maximum value of which is the length of the pendulum, when the $\delta$ has a fixed value, the $t_f$ will have a lower bound. In other words, when $\delta$ reaches a certain value, the transmission time of the system is relatively small, the speed is relatively large, and the system may break down so that the control is not very good. According to the analysis of physical meaning, when $\delta =l$, in this case, the pendulum is in a “lying flat” state, that is, in horizontal direction, the system basically breaks down completely. In short, we can determine whether the system breaks down based on the change in relative displacement with $\delta$.

5. Conclusions

In conclusion, we applied reverse engineering based on Lagrange mechanics and time minimization in the Pontriagin maximization principle for the fast but stable control of nonlinear unstable systems, e.g., the cartpole. In such control, we took an anharmonic effect and analyzed the stability of control. By fixing the relative displacement between the position of the ball and the trolley as constraints, we applied optimal control theory to design a time-optimal trajectory, as compared to the case of harmonic approximation. In the future, we will work out the classical control of the inverted double pendulum, and compare it with the control of the cartpole. Of course, the machine learning, fed by the results of STA, can be an interesting area for exploration as well.