Next Article in Journal
An Isomorphic Interactive Device for the Interventional Surgical Robot after In Vivo Study
Next Article in Special Issue
Dynamic Modeling and Experimental Validation of a Water Hydraulic Soft Manipulator Based on an Improved Newton—Euler Iterative Method
Previous Article in Journal
Removing Gas from a Closed-End Small Hole by Irradiating Acoustic Waves with Two Frequencies
Previous Article in Special Issue
Bio-Inspired Take-Off Maneuver and Control in Vertical Jumping for Quadruped Robot with Manipulator
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Micromachines
Volume 13
Issue 1
10.3390/mi13010110
micromachines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Underwater Soft Robotics: A Review of Bioinspiration in Design, Actuation, Modeling, and Control

by
Samuel M. Youssef
1,*,
MennaAllah Soliman
2,
Mahmood A. Saleh
2,
Mostafa A. Mousa
3,
Mahmoud Elsamanty
1,4 and
Ahmed G. Radwan
2,5
1
Smart Engineering Systems Research Center (SESC), Nile University, Sheikh Zayed City 12588, Egypt
2
School of Engineering and Applied Sciences, Nile University, Sheikh Zayed City 12588, Egypt
3
Nile University’s Innovation Hub, Nile University, Sheikh Zayed City 12588, Egypt
4
Mechanical Department, Faculty of Engineering at Shoubra, Benha University, Cairo 11672, Egypt
5
Department of Engineering Mathematics and Physics, Cairo University, Giza 12613, Egypt
*
Author to whom correspondence should be addressed.
Micromachines 2022, 13(1), 110; https://doi.org/10.3390/mi13010110
Submission received: 4 December 2021 / Revised: 31 December 2021 / Accepted: 2 January 2022 / Published: 10 January 2022
(This article belongs to the Special Issue Soft Robotics: Design, Fabrication, Modeling, Control and Applications)
Browse Figures
Versions Notes

Abstract

:
Nature and biological creatures are some of the main sources of inspiration for humans. Engineers have aspired to emulate these natural systems. As rigid systems become increasingly limited in their capabilities to perform complex tasks and adapt to their environment like living creatures, the need for soft systems has become more prominent due to the similar complex, compliant, and flexible characteristics they share with intelligent natural systems. This review provides an overview of the recent developments in the soft robotics field, with a focus on the underwater application frontier.

1. Introduction

Underwater exploration, much like space exploration, has been at the frontier of science and engineering ventures. As with the many Mars missions, where rovers and mobile robots are deployed instead of humans, deep underwater missions are mostly carried out using underwater robots. However, to this day, delving deep within the oceans of our planet still poses many challenges for these robotic systems. Some of the early robotic systems sent by humans to explore marine life are known as remotely operated vehicles (ROVs) [1]. ROVs are underwater robots, manually operated by a pilot, using tethered communication. They mainly have a rigid body hull and are actuated using electric thrusters. Autonomous underwater vehicles (AUVs) are similar to ROVs but differ in that they are untethered and do not require a pilot or an operator, as they are programmed to autonomously perform specific tasks. Both ROVs and AUVs vary in size, depending on the type of tasks they are manufactured to perform.
These underwater robotic systems are used to execute a wide range of underwater applications such as maintenance and monitoring applications. Such applications include underwater pipe inspection, offshore infrastructure repairs, and condition monitoring. Biological applications include seabed and abyssal exploration, sample gathering from marine environments such as coral reefs, and ecological aquatic phenomena monitoring and data collection [2]. More specifically, repairing and sampling tasks are carried out using underwater vehicle manipulator systems (UVMSs). UVMSs are unmanned underwater vehicles (UUVs) such as ROVs and AUVs that are equipped with different types of underwater manipulators that are suitable for the mentioned tasks [3]. The majority of manipulators used for underwater applications are actuated using hydraulic or electric systems. They can be used for the installation and maintenance 0f infrastructure such as pipes and cables [4], salvaging debris and sunken objects, mineral exploration [5], and biological samples gathering [6].
Even with the technological advances in UUVs and UVMS, they still face several limitations that prevent deep sea exploration. For instance, AUVs are limited to long-distance maneuvering at middle depths, while the use of ROVs is challenging in surface waters with strong waves and disturbances caused by currents. ROVs and UVMS also require constant communication with the operator, which becomes harder during seabed tasks due to the limitations of tethered communication. Some proposed solutions involve remote satellite communication [7]. In addition, the use of rigid manipulators is undesired for gathering delicate marine samples such as coral reefs, as they are hard to handle and can be damaged by the manipulators.
All these rigid robots have limited flexibility and adaptability to their environment. By contrast, soft robots offer more adaptability due to their compliant nature [8]. Such adaptability is exhibited in their ability to deform and change their shape according to the surrounding environment with which they are interacting. For example, soft robots made from compliant materials can achieve shrinking and bending motion that allow them to navigate within narrow areas. They also offer the possibility to grasp and manipulate various objects by adapting to their different shapes and structures. The evolution of soft robots led to advances in the pursuit of biomimicry of living creatures. The ability of soft robots to deform, change their shapes, exhibit infinite degrees of freedom, and perform complex motion, makes them a suitable candidate for the basis of biological emulation, especially that of underwater creatures, which are one of the sources of biomimetic inspiration for robotic and engineering systems. The exploitation of theses advantages offered by soft robots would help solve some of the challenges facing deep sea exploration. For instance, the use of soft manipulators allows safe handling of delicate coral reef samples [9,10], whereas imitating fishes’ and marine creatures’ propulsion methods can offer more efficient solutions for exploration and maneuvering at depth or in perturbed surface waters [11,12,13,14].
Alongside the great potential of soft robots, many challenges need to be overcome in order to reach the full functionality of bioinspired soft intelligent systems. Key challenges vary from design and fabrication to modeling and control. These challenges and the current state of research in solving them are discussed in this review.

2. Underwater Locomotion

Marine environments can seem extraterrestrial for humans at times. Hence, the study of the locomotion techniques and the morphology of aquatic creatures is essential. These types of biological studies offer insights providing keys toward the successful mimicry of these marine creatures. The aquatic environment plays a large role in defining the types of underwater locomotion, as governed by the four main forces acting on bodies underwater [15]: vertical weight and buoyancy alongside hydrodynamic lift, and horizontal thrust and drag (Figure 1a). Fish are able to generate lift and thrust in order to swim. They can achieve swimming using their fins or swimming propulsors (Figure 1b). According to the motion of these fins, fish swimming methods can be classified into several categories.
The two main categorizations of fish motion are based on which fins are performing the bending motion and the frequency at which the fins move. In terms of the first category, fish use their body and/or caudal fin to generate thrust (BCF). Examples include carangiform and anguilliform such as tuna and eel. Other types of fish use their median and/or paired fins (MPF). Examples include rajiform and labriform such as batoids. The frequency of movement of the fish’s body and fins indicates whether the motion is undulatory or oscillatory. During undulatory motion, the fish’s body performs a wave-shaped pattern, whereas oscillatory swimming uses only swivel-like motion.
Additional underwater locomotion modes fall outside the previous categorizations [16]. One example is the jet propulsion performed by jellyfish, octopus, and squid. Drag-induced swimming is exhibited by turtles as they generate thrust by moving their flippers in the opposing direction of motion. Friction-based crawling is performed by crustaceans, and echinoderms such as starfish use adhesive-based crawling.
In terms of assessing swimming performance, one of the most important metrics is the swimming speed of fish and, in particular, the critical swimming speed ( U c r i t ), which is commonly measured in centimeters per second (cm/s) or body lengths per second (BL/s) [17,18]. One of the main factors that affects fish swimming speed is the tail beat frequency in Hertz (Hz). It relates to the fish’s velocity through the stride length, which is the distance traveled by the fish per tail beat, expressed as ratio of the body length (L) [19,20]. The Reynolds number (Re) and Strouhal number (St) are also important factors to assess the hydrodynamic performance of the fish’s swimming. Several robotic fish platforms inspired from actual fish morphology and swimming, such as tuna, use the same metrics to assess their robots’ performance [13,21,22,23]. Another important factor is to analyze the efficiency of fish propulsion. However, it is hard to establish an accurate measure of propulsive efficiency for real biological fish. In general, efficiency is defined as the ratio of useful output to total input. For a self-propelled body, the measure of such work depends on the drag the body needs to overcome to move, which is hard to quantify as it differs with the shape of the body, as well as the body-propulsor hydrodynamics [24]. It is also challenging to determine input power in fish, which relates to muscle shaft power and the fish’s metabolism and oxygen (fuel) consumption [25]. A common metric used to quantify the fitness of fish and their efficiency is the cost of transport (COT), defined as the energy expended per traveled distance. The COT is a good indication of the fish’s swimming efficiency, and there have been several attempts to define and normalize COT for fish propulsive efficiency [24,26,27,28].

3. Challenges and Potentials of Soft Robots

3.1. Design

3.1.1. Bioinspiration

Since its inception, the field of robotics has drawn inspiration from nature. The main aspect of nature imitation in robotics is apparent in the design and structure of the robots’ bodies that aim to mimic biological systems. By looking at the knowledge gained through biomechanics studies, living creatures with mobile abilities are mainly classified into two groups based on their body structure: vertebrates and invertebrates. Vertebrates include fish, mammals, birds, amphibians, and reptiles; invertebrates include crustaceans (crab, lobster), echinoderms (starfish, sea urchin), coelenterates (jellyfish), arachnids, molluscs (octopus, squid), insects, and worms, among others [29].
The challenge of building robotic systems with motion capabilities similar to those of these creatures lies in their body construction, which exhibits compliance ranging from only a few parts such as an elephant’s trunk or mammals’ organs, to completely soft and deformable bodies in the case of some invertebrates such as jellyfish. The main contributor to this compliance is the elastic nature of the building blocks of these bodies such as muscles, tendons, skin, tissues, and cartilages, as they are known for having low Young’s modulus (less than one gigapascal) [8].
Some attempts have been made to mimic some of these animals using hard materials. However, due to the limited degrees of freedom offered by conventional rigid robots compared to the infinite degrees and redundancy of soft bodies, different structures with continuum deformations had to be implemented. In contrast to conventional non-redundant rigid robots, discrete hyper-redundant and hard continuum robots offer large to infinite degrees of freedom, which brings them closer to mimicking vertebrates’ motion [30,31]. Common examples include tendon-driven continuum manipulators [32,33,34]. One of the first underwater robots to employ a structure of discrete multiple rigid-link sections actuated by tendons is the RoboTuna robotic fish [35]; The VCUUV prototype, inspired by RoboTuna, uses hydraulic actuation to drive an articulated tail [36]. Other serial multi-joint biomimetic fish robots have been developed to imitate carangiform swimming [37,38].
Despite providing more degrees of freedom than rigid robots, hard continuum robots still lack the shape adaptability offered by soft robots, which would help bring robots closer to their bioinspired creatures. The Compliant Robotic Tuna (CRT) [39] is an example of a biomimetic fish robot having a servo-actuated compliant body and tail and is able to perform swimming maneuvers. The Soft Robotic Fish (SoFi) [40] is a marine exploration robot capable of 3D swimming that imitates fish motion. It is driven by a soft fluidic actuator and has a buoyancy control unit for depth adjustment. Other marine creatures such as batoids were also mimicked, as in the case of the stingray robot with a soft silicone outer body and pectoral fins [41].

3.1.2. Design Optimization

Even when taking inspiration from nature, designing soft robots with the desired mechanical behaviors that allow them to perform specific tasks presents another challenge. The complexity of such robotic systems, due to their unconventional components from materials to actuation, makes it hard to use currently known design and simulation tools to build soft robots [42,43]. Optimization techniques have been proposed to help automate the design process, and bridge the gap between simulation, fabrication, and the actual performance of soft robots. The general optimization framework can be summarized as choosing the design behavior to be optimized, such as crawling or grasping; identifying the design variables to be optimized, such as the material and the actuation; and defining the constraints of the system. The optimization process iteratively evaluates the design candidates using analysis tools and searches for the optimal design.
One approach uses evolutionary optimization algorithms to automate the design and manufacturing of freeform soft robots. This approach uses voxel-based dynamic simulation to evaluate the morphology and locomotion of the robot [44]. Voxels are soft cubic blocks with specific parameters, such as stiffness and Poisson’s ratio, that undergo volumetric change when forces are applied to them. Another voxel-based method aims to optimize the morphology to achieve adaptability using the property of criticality, which allows the robot to perform more diverse tasks [45].
Another conceptual design approach provides a spatial grammar to build soft robots and optimize their design for locomotion and actuation [46]. The spatial grammar generates sub-assemblies of interconnected balls based on a set of defined rules. The generated models are then evaluated and optimized in terms of locomotion abilities.
Performing design optimization for underwater soft robots is an even more challenging problem, as the effect of the environment on the robot’s morphology needs to be taken into account. DiffAqua [47], a computational design pipeline, relies on differentiable simulation to perform gradient-based optimization for the geometry and control of soft underwater swimmers. The benefits of exploiting the morphology of soft robots and optimizing it to simplify the control are further discussed in the upcoming modeling and control sections.
Fabricating and assessing these designs are also challenging processes due to the traditional manufacturing methods being unsuitable for these unconventional soft material. Additive manufacturing (AM) is one of the impactful technologies that helped enable this process [48,49]. One approach is to use AM to only fabricate the mold that would be used to pour the soft material in them. A more hybrid approach takes advantage of AM techniques, such as the fused filament fabrication (FFF) method, in addition to molding techniques to fabricate and assemble complex soft robotic systems. The third approach is the total additive manufacturing (TAM) approach. It exploits all the benefits of AM to fabricate soft robots, whether by 3D printing multiple soft parts and assemble them, or manufacturing the complete soft robot as a whole part. Such advance in 3D printing techniques for soft materials increased the ability to produce and test different designs of soft robots and optimize their morphological and material parameters.

3.2. Actuation

The actuation of soft robots poses several challenges due to the large number of degrees of freedom resulting from the large deformation of the soft materials that constitute them, making them underactuated systems that are harder to control. In addition, most conventional robotics actuators, such as DC motors, are bulky and rigid, which contradicts the main reason for developing soft robots with high compliance. Nonetheless, some soft robots use servo motors and gear pumps for fluidic actuation, while others use more unconventional actuators such as smart actuators, chemical reactions, and stiffness modulation [50].
One common actuation method is the use of tendon wires that are anchored at several points in the body of the soft material. These cables are driven by applying tension to them using electric motors such as servos, causing the connected soft material to deform, resulting in different motions or shape changes of the soft body. One example is the bioinspired octopus’s arm [51] made of silicone that is driven using cables. It can perform crawling motion and grasping similar to actual octopus tentacles. The use of traditional motors provides a large actuation force, especially in underwater applications where a powerful enough thrust is needed for locomotion. The shape deformation can be approximately determined through the displacement of the anchoring points of the cables.
Fluidic Elastomer Actuators (FEAs) is another type of soft actuators that rely mainly on fluid pressure [52]. The actuators are made from hyperelastic materials with embedded channels that expand due to the applied pressure. One of the early implementations is the Pneumatic Artificial Muscles (PAMs), most notably the McKibben artificial muscle actuator [53,54], which is made from a flexible elastomer tube constrained by a reinforced fiber to limit its extension but allow it to expand when pressurized, providing considerable force. Other types of fluidic elastomers use various means of pressurization, including pneumatic sources using compressed air [55,56], pressurized gas such as CO2 [57,58,59], or chemical pressure generation [60,61], as well as hydraulic sources [62,63,64]. The multigait crawling robot [55,57] has pneumatic actuators with a Pneu-Net (PN) architecture. The PNs are composed of a series of extensible chambers that inflate when pressurized and an inextensible layer that constrains the expansion of the chambers, causing the elastomer to bend. The geometrical parameters of the chambers and the constraining layer guide the deformation of the elastomer, affecting its bending and twisting motion. Underwater applications using fluidic elastomers include a biomimetic autonomous fish with a bidirectional pneumatic elastomer [59], an extended version of the former fish using a hydraulically pressurized elastomer instead [63], and an underwater crawling robot having bellow fluidic actuators as legs [64]. The completely soft Octobot [61] relies on totally soft microfluidic logic to control gas generation through chemical fuel decomposition, causing actuation. The use of fluidic actuators is advantageous for obtaining high material deformation and the ability to arrange actuators in an agonist-antagonist form, similar to muscle pairs. However, they are slow and have delayed response, and their pressurization units can be hard to embed inside soft robots.
Another actuation approach is the use of different types of smart materials. Smart materials are distinct in their response to external thermal or electric stimuli, causing deformation or stiffness change to the material. Electroactive polymers (EAPs) use electric stimuli to deform. Dielectric elastomer actuators (DEAs) are a type of EAPs that comprise two compliant electrodes that are compressed when high voltage is applied to them [65]. Compression force can be used to induce motion [66,67]. Another type of EAPs used for soft robots’ actuation is ionic polymer metal composite (IPMC). It is composed of Nafion polymer and electrodes. Applying voltage to the electrodes causes the polymer to deform due to the ionization process and the motion of ions between the two electrodes [68,69,70]. Shape memory alloys (SMAs) are smart materials that react to heat stimuli. When applying high temperature to the SMA, it deforms into a certain shape and is restored to its original shape after heat is removed. The heat is usually provided through electrical heating using high voltage. SMAs are used as actuators in soft robotics, as they can be embedded to drive a soft material such as polydimethylsiloxane (PDMS) [71,72].
The use of smart actuators is prominent in underwater robotics [73] due to the favorable operating conditions for smart materials in water. In addition, smart materials can be directly embedded within the elastically deformable body of the robots, making them a good option for biomimetic applications. For example, biomimicry of jellyfish was implemented using DEAs [67] and using SMAs in the case of Robojelly [74]. Manta ray biomimetic robots were actuated using IPMCs [68] as well as SMAs [75]. A biomimetic crawling starfish used actuated legs made from embedded SMA wires cast in PDMS [71]. Another group developed a soft robotic arm inspired by octopus tentacles using cables and SMA springs [72]. The SMA springs help mimic the muscular hydrostat of the octopus’s arm by providing transversal contraction. Smart actuators provide an advantage in terms of their compact size and weight, and high actuation biomimicry resembling real fish swimming modes. However, they require high-voltage sources and are hard to control. The various soft robotic platforms are shown in Table 1, classifying their biomimetic inspiration, actuation types, swimming modes, and level of compliance.

3.3. Modeling

The modeling and control phase is the most challenging part of building functional soft robotic systems capable of performing complex tasks and intelligently interacting with their own environment. All the well-established modeling techniques for rigid robots cannot be applied to soft robots due to their continuum property and their complex non-linear dynamics inherent from their elastic behavior. As conventional kinematic and dynamic modeling methods are inapplicable, new approaches for modeling and control of soft robots are being developed (Figure 2).
These modeling approaches can be separated into two main categories: model-based approaches and model-free data-driven methods. While the former relies on formulating either an exactly accurate or a simplified approximate model of a soft robot, the latter tries to implicitly learn the behavior of the robot using input and output data collected directly from the actual system.
The main goal of the modeling process is to map the soft robot’s actuation space to the configuration or task space. Since continuum soft robots are infinite-dimensional systems, formulating such models becomes highly difficult. Instead, the modeling methods rely on approximations and assumptions to reduce the system to a finite-dimensional one. The most commonly used simplification for kinematic modeling is the constant curvature (CC) method [84]. It assumes that the soft robot has constant strain along its whole length [85]. The piecewise constant curvature (PCC) method is an extended version of the CC that assumes the strain to be piecewise constant, with each segment of the soft structure having a constant strain [86]. A further extension is the variable curvature (VC) [87,88], which models each section as a CC. The PCC approximation is mainly valid for cable-driven soft manipulators. However, it cannot capture all the complex dynamics of soft structures, as it is a steady-state model. Nonetheless, it proves to be an accurate model for control for a lot of tendon-driven manipulators [89,90,91,92]. One group also used a PCC kinematic model of a multi-segment pneumatic actuator to develop a dynamic motion controller [93].
Numerical techniques have also been used for modeling soft robots. The Cosserat rod theory is one of these numerical methods [94,95], accurately representing the tension, shearing, bending, and torsion of rods. A different numerical approach for modeling soft robots is the finite element method (FEM), which yields more accurate results but at the cost of high computational requirements [96]. The method relies on discretizing the structure into a large number of nodes, called mesh, and iteratively solving the differential equations governing the behavior of these nodes, until the model converges. However, the use of FEM for real-time control is difficult, so control approaches based on real-time FEM have been proposed [97,98,99]. An approach combining kinematic modeling using PCC and Denavit–Hartenberg (DH) parameters with FEM analysis was also used to model a soft pneumatic actuator (SPA) [100]. The problem with the FEM is the high dimensional space of the obtained model. A common solution is to reduce the domain of the model in order to achieve high computational efficiency, without sacrificing accuracy [101,102]. The reduced-order model can help with the development of low-order controllers and observers based on a linearized model of the system [103,104]. One group developed a dynamic simulation tool for articulated soft robots based on numerical simulation methods for slender structures [105]. Another method uses genetic algorithms for dynamic parameters estimation of an octopus-inspired robot [106].
Controllers developed from the static kinematic models obtained using the described methods are considered static controllers that discard the underlying dynamics of the system. Developing high-order dynamic models for soft robots is difficult and computationally expensive for controllers. A first-order dynamic modeling approach is proposed to reduce the computational space without affecting the controller’s performance [107].
The model-free methods for modeling soft robots mainly use data-driven machine learning and deep learning techniques to find a mapping between the inputs and outputs of the soft system [108]. Input actuation signals and robot states can be obtained by sensors, either embedded or external visual tracking sensors. The data can then be used with different supervised learning, unsupervised learning, and reinforcement learning techniques to develop models and controllers for soft robots. Examples of commonly used techniques include feedforward neural network (FNN) [109,110,111,112], recurrent neural network (RNN) [113], convolutional neural network (CNN) [114], and echo state network (ESN) [109], based on the reservoir computing framework. FNNs are widely used for modeling soft systems. One approach is the use of an FNN to model the work envelope of an SPA with a variable inclination angle [115]. Another group used FEM-generated training data to learn the kinematic model of a 3D motion SPA using an FNN [111]. An RNN used sensory data from cPDMS resistive sensors and a load cell to predict the deformation and force models of a soft pneumatic finger [113]. A reservoir computing approach with the ESN architecture was used to model the 2D motion of a bioinspired turtle actuated through soft pneumatic flippers [109].
Learning-based techniques also proved to be successful in learning the dynamic models of soft robots. One approach involved using a nonlinear autoregressive exogenous (NARX) model to develop a dynamic model for a soft manipulator, which was used to implement a task space controller [116]. A deep neural network (DNN) model learned the non-linear dynamics for a single degree of freedom inflatable pneumatic robot. The model was used to implement a model predictive control (MPC) algorithm for pressure control [117].

3.4. Control

Models of soft robots developed using the discussed approaches are used to develop kinematic and dynamic control for these complex systems. Model-based controllers rely on models obtained from analytical kinematic methods such as PCC, whereas model-free controllers use data-driven techniques [91]. Different control algorithms are used, depending on the level at which the controller operates. Low-level controllers drive the actuators, whereas mid-level ones are responsible for the kinematic and dynamic control, and high-level control involves advanced trajectory and path planning for tasks such as obstacle avoidance.
The main task of soft robots’ controllers is to manage the whole-body deformation of the robot; it may also include controlling the exact position and orientation of an end effector in the case of soft manipulators. One approach uses open-loop control for a soft manipulator [118]. The implemented open-loop dynamic controller uses a data-driven model with only mechanical feedback. The swimming eel-like robot [119] was modeled using the geometrically exact beam theory with a torque control algorithm. A different approach uses an energy-shaping approach to develop the control law for a soft continuum manipulator [120]. MPC is another technique that was employed for the control of large-scale soft robots [121]. The MPC algorithm relies on a PCC model alongside a kinematic representation for efficient state prediction. A model reference predictive adaptive control (MRPAC) was also implemented on the same model and showed robustness to model uncertainties. Closed-loop control methods have also been demonstrated for the position control of soft robots [122,123].
One of the most promising approaches for the model-free control in the robotics field, in general, is reinforcement learning (RL), which has been proven successful for soft robots [124]. RL can be described as an iterative learning process where the agent takes an action, its new state is observed, and a response in the form of a reward function is given to it based on the resulting interaction with its environment. When the agent learns a policy to map appropriate state–action pairs, then the learning is successful. RL can be implemented regardless of whether a model of the system is known. The use of deep reinforcement learning (DRL) and imitation learning algorithms in the development of soft robots has been shown in several experimental examples. DRL methods are now integrated with soft robots in various applications such as biomedical and edible robotics. Some research focused on the deep Q-network (DQN) algorithm, which was used in a soft robotic fish used for underwater exploration [40]. Other common algorithms are deep deterministic policy gradient (DDPG), normalized advantage function (NAF), and advantage actor-critic (A2C). However, problems arise from the differences between the simulation environments where the robots are trained and the real-world environment. The use of generative adversarial network (GAN) is a suggested solution to help perform domain adaptation and narrow the gap between the simulation and real-world environments. Imitation learning is also beneficial when it is difficult to formulate a reward function to train a DRL model. The imitation learning algorithms use demonstrations constructed by an expert agent that are transferred to the soft robot. The most common imitation learning approaches are behavior cloning, inverse reinforcement learning, and generative adversarial imitation learning. The future scope is the combination of both deep reinforcement learning and imitation learning algorithms to benefit from both approaches and train better agents.One example is a soft artificial-muscle-driven robot mimicking cuttlefish actuated by a dielectric elastomer (DE) membrane [66]. The robot uses reinforcement learning to provide the actuation. To simplify the problem, only two actions are considered, with only two voltage amplitudes: 0 and 6.8 kV. The robot is trained through trial-and-error interaction with its environment in order to find an optimal policy to maximize its reward function, which is the displacement at each time step. The displacement is monitored using a camera and then fed to the reinforcement learning algorithm.
Applying reinforcement learning with soft robots is a costly operation due to their non-linear hyperelastic properties. To overcome this issue, the reinforcement learning approach used to control the Honeycomb Pneu-Nets Soft Robot ignores the specific properties of the materials and the structural characteristics of the robot [125], concentrating only on the geometric model, which simplifies the modeling task. Another challenge to implementing RL is the lack of accurate simulators for soft robots. To overcome this problem, researchers performed the training process on both the simulation as well as the physical hardware to obtain a more realistic control policy that works well with the actual robot. The reinforcement learning framework consists of two parts: formulating the set of representations for the robot’s states and actions, and the training process to search for accurate results in the problem space. The physical hardware uses air pumps and valves to achieve the actuation needed and uses the OptiTrack motion capture system for visual sensing to determine the actual state of the robot. Two methods of control were used to execute the learned RL policy: open-loop and closed-loop. The closed-loop method obtains the actual robot state from the sensor as the input to the trained control policy function, while in the open-loop method, the robot’s state is obtained from the simulator, leading to some errors.
Despite providing a good solution in many cases, RL may be impractical to implement when the reward function canonot be clearly defined. Imitation learning solves this issue by using demonstrations performed by an expert agent. One group used imitation learning to perform motion control and trajectory planning for soft continuum robots [126]. They proposed the learning from demonstration (LfD) approach and implemented it on the Bionic Handling Assistant (BHA) robot, which resembles an elephant trunk. One of the simplest methods to implement LfD is kinesthetic teaching, which is achieved by directly recording demonstrations on the target robot to collect the position and orientation data of the end effector. The problem of using kinesthetic teaching with soft robots arises from the dynamic complexity of non-linear elasticity associated with these soft materials. They proposed an active compliant control to record the demonstrations during kinesthetic teaching, then encoded the recorded data with a task-parametrized Gaussian mixture model (TP-GMM).
A similar approach was used with the soft cylindrical robot arm STIFF-FLOP [127]. However, the researchers worked on transferring the movement patterns of an octopus arm to the STIFF-FLOP. The octopus arm movements were obtained from a database of the cartesian position of several octopuses’ arms with an average of 100 points along the arm. They exploited several methods for the representation of the octopus reaching motion to help transfer the arm movements to the soft robot. They considered spatiotemporal representation and dynamical movement primitives to allow for a more robust movement transfer. They then used Gaussian mixture regression (GMR) for encoding, and applied a self-refinement algorithm with a weighted reward function according to different tasks.
Following a similar thought process to imitation learning, a trending new paradigm called morphological computation aims to learn from living creatures, but on the level of morphology instead of the locomotion level. Morphological computation is a process related to embodied intelligence, where some of the computation needed for actuation and perception is conducted automatically by the body of the living creature instead of sending sensory information to the brain and waiting for a control signal [128]. Morphological computation can be exploited in the context of soft robots to offload some of the control to the body. This process is possible owing to the adaptable compliance exhibited by soft materials, which act as a reservoir computer that can process inputs from its environment and take appropriate actions.
A morphological computation framework with a mathematical realization was proposed using a reservoir of recurrent non-linear mass-spring systems, which is a model for actual physical soft bodies [129]. The model demonstrated the ability to learn the end-effector trajectory of a robotic arm. The group also showed that adding feedback to the morphological computation system allows it to perform autonomous periodic patterns such as the ones responsible for locomotion [130].

4. Prospective Directions

Soft robotics is still a new field with many challenges and obstacles, but shows promising potential. Current research is aimed toward the establishment of a unified foundation and framework for the process of building complete and functional soft robots, the same as the basis available for conventional rigid robots.
One promising approach is the attempt to automate and optimize the various aspects of soft robots’ development, including the design, manufacturing, choice of actuation, and sensing components. The exploration of new soft materials with variable controlled stiffness, suitable for different environments, and the exploitation of their morphology in actuation, sensing, and control may provide several solutions to the current challenges. The use of rapid design and manufacturing processes and of the technological advances in additive manufacturing is needed to allow the development of complex soft robotic systems. Another approach is to develop completely soft robots, void of any type of rigid elements. All actuators, sensors, and processing units will need to be implemented using completely soft materials. In addition, the development of general modeling and control techniques that can be used for all types of soft robots would strongly boost advances in the field.
Embodied intelligence provides a possible solution for building soft robots that are close to their biological inspirations. Similar to the connection between the brains and bodies of living creatures, where the morphology of the body takes part in the interaction with the environment, soft robotic systems may be built to exploit their compliance in the computation and processing of data they receive while navigating and adapting to their surroundings. Another potential direction is the investigation of using organic living tissues to help build soft robotic systems. Since soft robots try to mimic actual living creatures, the use of organic parts to fabricate soft robots would be beneficial. These living tissues may be embedded in soft robots to exploit their biological actuation and sensory features. In addition, it is important to develop soft sensors that can easily be embedded within the structures of soft robots, without significantly affecting their compliance. This would enable proprioceptive sensing of the robots’ deformation, allowing better control and more interaction with the environment. Finally, taking more inspiration from biological systems while developing soft robots, by integrating all the robotic components from brain to morphology, to actuation, and sensing, into homogeneous entities may help with the development of more biological-like systems.

Author Contributions

All authors contributed to the conception and the structure of the paper. S.M.Y. compiled the references and wrote the manuscript. All authors contributed to editing and reviewing. All authors have read and agreed to the published version of the manuscript.

Funding

The authors would like to thank the Academy of Scientific Research and Technology (ASRT), Egypt, for funding project #4779, and Nile University for facilitating all procedures required to complete this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chutia, S.; Kakoty, N.M.; Deka, D. A review of underwater robotics, navigation, sensing techniques and applications. Proc. Adv. Robot. 2017, 8, 1–6. [Google Scholar]
  2. Aracri, S.; Giorgio-Serchi, F.; Suaria, G.; Sayed, M.E.; Nemitz, M.P.; Mahon, S.; Stokes, A.A. Soft robots for ocean exploration and offshore operations: A perspective. Soft Robot. 2021, 8, 625–639. [Google Scholar] [CrossRef] [PubMed]
  3. Sivčev, S.; Coleman, J.; Omerdić, E.; Dooly, G.; Toal, D. Underwater manipulators: A review. Ocean. Eng. 2018, 163, 431–450. [Google Scholar] [CrossRef]
  4. Christ, R.D.; Wernli, R.L., Sr. The ROV Manual: A User Guide for Remotely Operated Vehicles; Butterworth-Heinemann: Oxford, UK, 2013. [Google Scholar]
  5. Di Vito, D.; De Palma, D.; Simetti, E.; Indiveri, G.; Antonelli, G. Experimental validation of the modeling and control of a multibody underwater vehicle manipulator system for sea mining exploration. J. Field Robot. 2021, 38, 171–191. [Google Scholar] [CrossRef]
  6. Jones, D.O. Using existing industrial remotely operated vehicles for deep-sea science. Zool. Scr. 2009, 38, 41–47. [Google Scholar] [CrossRef]
  7. Di Lillo, P.; Simetti, E.; Wanderlingh, F.; Casalino, G.; Antonelli, G. Underwater intervention with remote supervision via satellite communication: Developed control architecture and experimental results within the dexrov project. IEEE Trans. Control Syst. Technol. 2020, 29, 108–123. [Google Scholar] [CrossRef]
  8. Rus, D.; Tolley, M.T. Design, fabrication and control of soft robots. Nature 2015, 521, 467–475. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Galloway, K.C.; Becker, K.P.; Phillips, B.; Kirby, J.; Licht, S.; Tchernov, D.; Wood, R.J.; Gruber, D.F. Soft robotic grippers for biological sampling on deep reefs. Soft Robot. 2016, 3, 23–33. [Google Scholar] [CrossRef] [PubMed]
  10. Sinatra, N.R.; Teeple, C.B.; Vogt, D.M.; Parker, K.K.; Gruber, D.F.; Wood, R.J. Ultragentle manipulation of delicate structures using a soft robotic gripper. Sci. Robot. 2019, 4, eaax5425. [Google Scholar] [CrossRef] [PubMed]
  11. Ren, Z.; Hu, W.; Dong, X.; Sitti, M. Multi-functional soft-bodied jellyfish-like swimming. Nat. Commun. 2019, 10, 1–12. [Google Scholar] [CrossRef]
  12. Chen, T.; Bilal, O.R.; Shea, K.; Daraio, C. Harnessing bistability for directional propulsion of soft, untethered robots. Proc. Natl. Acad. Sci. USA 2018, 115, 5698–5702. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Zhu, J.; White, C.; Wainwright, D.K.; Di Santo, V.; Lauder, G.V.; Bart-Smith, H. Tuna robotics: A high-frequency experimental platform exploring the performance space of swimming fishes. Sci. Robot. 2019, 4, eaax4615. [Google Scholar] [CrossRef]
  14. Giorgio-Serchi, F.; Arienti, A.; Corucci, F.; Giorelli, M.; Laschi, C. Hybrid parameter identification of a multi-modal underwater soft robot. Bioinspiration Biomim. 2017, 12, 025007. [Google Scholar] [CrossRef] [PubMed]
  15. Sfakiotakis, M.; Lane, D.M.; Davies, J.B.C. Review of fish swimming modes for aquatic locomotion. IEEE J. Ocean. Eng. 1999, 24, 237–252. [Google Scholar] [CrossRef] [Green Version]
  16. Hermes, M.; Ishida, M.; Luhar, M.; Tolley, M.T. Bioinspired Shape-Changing Soft Robots for Underwater Locomotion: Actuation and Optimization for Crawling and Swimming. In Bioinspired Sensing, Actuation, and Control in Underwater Soft Robotic Systems; Springer: Berlin/Heidelberg, Germany, 2021; pp. 7–39. [Google Scholar]
  17. Cano-Barbacil, C.; Radinger, J.; Argudo, M.; Rubio-Gracia, F.; Vila-Gispert, A.; García-Berthou, E. Key factors explaining critical swimming speed in freshwater fish: A review and statistical analysis for Iberian species. Sci. Rep. 2020, 10, 1–12. [Google Scholar] [CrossRef]
  18. Palstra, A.P.; Kals, J.; Böhm, T.; Bastiaansen, J.W.; Komen, H. Swimming performance and oxygen consumption as non-lethal indicators of production traits in Atlantic salmon and gilthead seabream. Front. Physiol. 2020, 11, 759. [Google Scholar] [CrossRef] [PubMed]
  19. Videler, J.; Wardle, C. Fish swimming stride by stride: Speed limits and endurance. Rev. Fish Biol. Fish. 1991, 1, 23–40. [Google Scholar] [CrossRef]
  20. Bainbridge, R. The speed of swimming of fish as related to size and to the frequency and amplitude of the tail beat. J. Exp. Biol. 1958, 35, 109–133. [Google Scholar] [CrossRef]
  21. Chen, B.; Jiang, H. Swimming performance of a tensegrity robotic fish. Soft Robot. 2019, 6, 520–531. [Google Scholar] [CrossRef]
  22. Zhao, W.; Ming, A.; Shimojo, M. Development of high-performance soft robotic fish by numerical coupling analysis. Appl. Bionics Biomech. 2018, 2018. [Google Scholar] [CrossRef] [PubMed]
  23. Li, K.; Jiang, H.; Wang, S.; Yu, J. A soft robotic fish with variable-stiffness decoupled mechanisms. J. Bionic Eng. 2018, 15, 599–609. [Google Scholar] [CrossRef]
  24. Maertens, A.; Triantafyllou, M.S.; Yue, D.K. Efficiency of fish propulsion. Bioinspiration Biomim. 2015, 10, 046013. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Schultz, W.W.; Webb, P.W. Power requirements of swimming: Do new methods resolve old questions? Integr. Comp. Biol. 2002, 42, 1018–1025. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Liu, G.; Yu, Y.L.; Tong, B.G. Optimal energy-utilization ratio for long-distance cruising of a model fish. Phys. Rev. E 2012, 86, 016308. [Google Scholar] [CrossRef]
  27. Eloy, C. On the best design for undulatory swimming. J. Fluid Mech. 2013, 717, 48–89. [Google Scholar] [CrossRef] [Green Version]
  28. Tokić, G.; Yue, D.K. Optimal shape and motion of undulatory swimming organisms. Proc. R. Soc. B Biol. Sci. 2012, 279, 3065–3074. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Verma, A. Status of animal phyla in different kingdom systems of biological classification. Int. J. Biol. Innov. 2020, 2, 149–154. [Google Scholar] [CrossRef]
  30. Trivedi, D.; Rahn, C.D.; Kier, W.M.; Walker, I.D. Soft robotics: Biological inspiration, state of the art, and future research. Appl. Bionics Biomech. 2008, 5, 99–117. [Google Scholar] [CrossRef]
  31. Walker, I.D. Continuous backbone “continuum” robot manipulators. Int. Sch. Res. Not. 2013, 2013. [Google Scholar] [CrossRef] [Green Version]
  32. Hannan, M.W.; Walker, I.D. Kinematics and the implementation of an elephant’s trunk manipulator and other continuum style robots. J. Robot. Syst. 2003, 20, 45–63. [Google Scholar] [CrossRef]
  33. Hannan, M.W.; Walker, I.D. Analysis and experiments with an elephant’s trunk robot. Adv. Robot. 2001, 15, 847–858. [Google Scholar] [CrossRef] [PubMed]
  34. Camarillo, D.B.; Carlson, C.R.; Salisbury, J.K. Configuration tracking for continuum manipulators with coupled tendon drive. IEEE Trans. Robot. 2009, 25, 798–808. [Google Scholar] [CrossRef]
  35. Barrett, D.S. Propulsive Efficiency of a Flexible Hull Underwater Vehicle. Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 1996. [Google Scholar]
  36. Anderson, J.M.; Chhabra, N.K. Maneuvering and stability performance of a robotic tuna. Integr. Comp. Biol. 2002, 42, 118–126. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Liu, J.; Hu, H. Biological inspiration: From carangiform fish to multi-joint robotic fish. J. Bionic Eng. 2010, 7, 35–48. [Google Scholar] [CrossRef]
  38. Clapham, R.J.; Hu, H. iSplash-I: High performance swimming motion of a carangiform robotic fish with full-body coordination. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 322–327. [Google Scholar]
  39. Mazumdar, A.; Alvarado, P.V.Y.; Youcef-Toumi, K. Maneuverability of a robotic tuna with compliant body. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 19–23 May 2008; pp. 683–688. [Google Scholar]
  40. Katzschmann, R.K.; DelPreto, J.; MacCurdy, R.; Rus, D. Exploration of underwater life with an acoustically controlled soft robotic fish. Sci. Robot. 2018, 3, eaar3449. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Cloitre, A.; Subramaniam, V.; Patrikalakis, N.; Valdivia y Alvarado, P. Design and control of a field deployable batoid robot. In Proceedings of the 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy, 4–27 June 2012; pp. 707–712. [Google Scholar]
  42. Chen, F.; Wang, M.Y. Design optimization of soft robots: A review of the state of the art. IEEE Robot. Autom. Mag. 2020, 27, 27–43. [Google Scholar] [CrossRef]
  43. Lipson, H. Challenges and opportunities for design, simulation, and fabrication of soft robots. Soft Robot. 2014, 1, 21–27. [Google Scholar] [CrossRef]
  44. Hiller, J.; Lipson, H. Automatic design and manufacture of soft robots. IEEE Trans. Robot. 2011, 28, 457–466. [Google Scholar] [CrossRef]
  45. Talamini, J.; Medvet, E.; Nichele, S. Criticality-Driven Evolution of Adaptable Morphologies of Voxel-Based Soft-Robots. Front. Robot. AI 2021, 8, 172. [Google Scholar] [CrossRef] [PubMed]
  46. Van Diepen, M.; Shea, K. A spatial grammar method for the computational design synthesis of virtual soft locomotion robots. J. Mech. Des. 2019, 141, 101402. [Google Scholar] [CrossRef]
  47. Ma, P.; Du, T.; Zhang, J.Z.; Wu, K.; Spielberg, A.; Katzschmann, R.K.; Matusik, W. DiffAqua: A Differentiable Computational Design Pipeline for Soft Underwater Swimmers with Shape Interpolation. arXiv 2021, arXiv:2104.00837. [Google Scholar]
  48. Walker, S.; Yirmibeşoğlu, O.; Daalkhaijav, U.; Mengüç, Y. Additive manufacturing of soft robots. In Robotic Systems and Autonomous Platforms; Elsevier: Amsterdam, The Netherlands, 2019; pp. 335–359. [Google Scholar]
  49. Stano, G.; Percoco, G. Additive manufacturing aimed to soft robots fabrication: A review. Extrem. Mech. Lett. 2021, 42, 101079. [Google Scholar] [CrossRef]
  50. Coyle, S.; Majidi, C.; LeDuc, P.; Hsia, K.J. Bio-inspired soft robotics: Material selection, actuation, and design. Extrem. Mech. Lett. 2018, 22, 51–59. [Google Scholar] [CrossRef]
  51. Calisti, M.; Giorelli, M.; Levy, G.; Mazzolai, B.; Hochner, B.; Laschi, C.; Dario, P. An octopus-bioinspired solution to movement and manipulation for soft robots. Bioinspiration Biomim. 2011, 6, 036002. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  52. Polygerinos, P.; Correll, N.; Morin, S.A.; Mosadegh, B.; Onal, C.D.; Petersen, K.; Cianchetti, M.; Tolley, M.T.; Shepherd, R.F. Soft robotics: Review of fluid-driven intrinsically soft devices; manufacturing, sensing, control, and applications in human-robot interaction. Adv. Eng. Mater. 2017, 19, 1700016. [Google Scholar] [CrossRef]
  53. Chou, C.P.; Hannaford, B. Measurement and modeling of McKibben pneumatic artificial muscles. IEEE Trans. Robot. Autom. 1996, 12, 90–102. [Google Scholar] [CrossRef] [Green Version]
  54. Tondu, B.; Lopez, P. Modeling and control of McKibben artificial muscle robot actuators. IEEE Control Syst. Mag. 2000, 20, 15–38. [Google Scholar]
  55. Shepherd, R.F.; Ilievski, F.; Choi, W.; Morin, S.A.; Stokes, A.A.; Mazzeo, A.D.; Chen, X.; Wang, M.; Whitesides, G.M. Multigait soft robot. Proc. Natl. Acad. Sci. USA 2011, 108, 20400–20403. [Google Scholar] [CrossRef] [Green Version]
  56. Marchese, A.D.; Komorowski, K.; Onal, C.D.; Rus, D. Design and control of a soft and continuously deformable 2d robotic manipulation system. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 2189–2196. [Google Scholar]
  57. TolleyMichael, T.; ShepherdRobert, F.; GallowayKevin, C.; WoodRobert, J.; WhitesidesGeorge, M. A resilient, untethered soft robot. Soft Robot. 2014, 1. [Google Scholar] [CrossRef]
  58. Marchese, A.D.; Onal, C.D.; Rus, D. Towards a self-contained soft robotic fish: On-board pressure generation and embedded electro-permanent magnet valves. In Experimental Robotics; Springer: Berlin/Heidelberg, Germany, 2013; pp. 41–54. [Google Scholar]
  59. Marchese, A.D.; Onal, C.D.; Rus, D. Autonomous soft robotic fish capable of escape maneuvers using fluidic elastomer actuators. Soft Robot. 2014, 1, 75–87. [Google Scholar] [CrossRef] [Green Version]
  60. Onal, C.D.; Chen, X.; Whitesides, G.M.; Rus, D. Soft mobile robots with on-board chemical pressure generation. In Robotics Research; Springer: Berlin/Heidelberg, Germany, 2017; pp. 525–540. [Google Scholar]
  61. Wehner, M.; Truby, R.L.; Fitzgerald, D.J.; Mosadegh, B.; Whitesides, G.M.; Lewis, J.A.; Wood, R.J. An integrated design and fabrication strategy for entirely soft, autonomous robots. Nature 2016, 536, 451–455. [Google Scholar] [CrossRef]
  62. Katzschmann, R.K.; De Maille, A.; Dorhout, D.L.; Rus, D. Cyclic hydraulic actuation for soft robotic devices. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 3048–3055. [Google Scholar]
  63. Katzschmann, R.K.; Marchese, A.D.; Rus, D. Hydraulic autonomous soft robotic fish for 3D swimming. In Experimental Robotics; Springer: Berlin/Heidelberg, Germany, 2016; pp. 405–420. [Google Scholar]
  64. Ishida, M.; Drotman, D.; Shih, B.; Hermes, M.; Luhar, M.; Tolley, M.T. Morphing structure for changing hydrodynamic characteristics of a soft underwater walking robot. IEEE Robot. Autom. Lett. 2019, 4, 4163–4169. [Google Scholar] [CrossRef]
  65. Guo, Y.; Liu, L.; Liu, Y.; Leng, J. Review of Dielectric Elastomer Actuators and Their Applications in Soft Robots. Adv. Intell. Syst. 2021, 3, 2000282. [Google Scholar] [CrossRef]
  66. Yang, T.; Xiao, Y.; Zhang, Z.; Liang, Y.; Li, G.; Zhang, M.; Li, S.; Wong, T.W.; Wang, Y.; Li, T.; et al. A soft artificial muscle driven robot with reinforcement learning. Sci. Rep. 2018, 8, 14518. [Google Scholar] [CrossRef]
  67. Christianson, C.; Bayag, C.; Li, G.; Jadhav, S.; Giri, A.; Agba, C.; Li, T.; Tolley, M.T. Jellyfish-inspired soft robot driven by fluid electrode dielectric organic robotic actuators. Front. Robot. AI 2019, 6, 126. [Google Scholar] [CrossRef] [Green Version]
  68. Chen, Z.; Um, T.I.; Bart-Smith, H. Bio-inspired robotic manta ray powered by ionic polymer–metal composite artificial muscles. Int. J. Smart Nano Mater. 2012, 3, 296–308. [Google Scholar] [CrossRef] [Green Version]
  69. Yang, T.; Chen, Z. Development of 2D maneuverable robotic fish propelled by multiple ionic polymer-metal composite artificial fins. In Proceedings of the 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China, 6–9 December 2015; pp. 255–260. [Google Scholar]
  70. Ye, Z.; Hou, P.; Chen, Z. 2D maneuverable robotic fish propelled by multiple ionic polymer–metal composite artificial fins. Int. J. Intell. Robot. Appl. 2017, 1, 195–208. [Google Scholar] [CrossRef]
  71. Jin, H.; Dong, E.; Alici, G.; Mao, S.; Min, X.; Liu, C.; Low, K.; Yang, J. A starfish robot based on soft and smart modular structure (SMS) actuated by SMA wires. Bioinspiration Biomim. 2016, 11, 056012. [Google Scholar] [CrossRef] [PubMed]
  72. Laschi, C.; Cianchetti, M.; Mazzolai, B.; Margheri, L.; Follador, M.; Dario, P. Soft robot arm inspired by the octopus. Adv. Robot. 2012, 26, 709–727. [Google Scholar] [CrossRef]
  73. Chu, W.S.; Lee, K.T.; Song, S.H.; Han, M.W.; Lee, J.Y.; Kim, H.S.; Kim, M.S.; Park, Y.J.; Cho, K.J.; Ahn, S.H. Review of biomimetic underwater robots using smart actuators. Int. J. Precis. Eng. Manuf. 2012, 13, 1281–1292. [Google Scholar] [CrossRef]
  74. Villanueva, A.; Smith, C.; Priya, S. A biomimetic robotic jellyfish (Robojelly) actuated by shape memory alloy composite actuators. Bioinspiration Biomim. 2011, 6, 036004. [Google Scholar] [CrossRef] [PubMed]
  75. Wang, Z.; Wang, Y.; Li, J.; Hang, G. A micro biomimetic manta ray robot fish actuated by SMA. In Proceedings of the 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO), Guilin, China, 19–23 December 2009; pp. 1809–1813. [Google Scholar]
  76. Cianchetti, M.; Calisti, M.; Margheri, L.; Kuba, M.; Laschi, C. Bioinspired locomotion and grasping in water: The soft eight-arm OCTOPUS robot. Bioinspiration Biomim. 2015, 10, 035003. [Google Scholar] [CrossRef] [PubMed]
  77. Mao, S.; Dong, E.; Jin, H.; Xu, M.; Zhang, S.; Yang, J.; Low, K.H. Gait study and pattern generation of a starfish-like soft robot with flexible rays actuated by SMAs. J. Bionic Eng. 2014, 11, 400–411. [Google Scholar] [CrossRef]
  78. Robertson, M.A.; Efremov, F.; Paik, J. RoboScallop: A bivalve inspired swimming robot. IEEE Robot. Autom. Lett. 2019, 4, 2078–2085. [Google Scholar] [CrossRef]
  79. Christianson, C.; Goldberg, N.N.; Deheyn, D.D.; Cai, S.; Tolley, M.T. Translucent soft robots driven by frameless fluid electrode dielectric elastomer actuators. Sci. Robot. 2018, 3, eaat1893. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  80. Baines, R.L.; Booth, J.W.; Fish, F.E.; Kramer-Bottiglio, R. Toward a bio-inspired variable-stiffness morphing limb for amphibious robot locomotion. In Proceedings of the 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft), Seoul, Korea, 14–18 April 2019; pp. 704–710. [Google Scholar]
  81. Gatto, V.L.; Rossiter, J.M.; Hauser, H. Robotic Jellyfish Actuated by Soft FinRay Effect Structured Tentacles. In Proceedings of the 2020 3rd IEEE International Conference on Soft Robotics (RoboSoft), New Haven, CT, USA, 15 May–15 July 2020; pp. 144–149. [Google Scholar]
  82. Patterson, Z.J.; Sabelhaus, A.P.; Chin, K.; Hellebrekers, T.; Majidi, C. An Untethered Brittle Star-Inspired Soft Robot for Closed-Loop Underwater Locomotion. In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24 October–24 January 2020; pp. 8758–8764. [Google Scholar]
  83. Du, T.; Hughes, J.; Wah, S.; Matusik, W.; Rus, D. Underwater Soft Robot Modeling and Control With Differentiable Simulation. IEEE Robot. Autom. Lett. 2021, 6, 4994–5001. [Google Scholar] [CrossRef]
  84. Webster, R.J., III; Jones, B.A. Design and kinematic modeling of constant curvature continuum robots: A review. Int. J. Robot. Res. 2010, 29, 1661–1683. [Google Scholar] [CrossRef]
  85. Gravagne, I.A.; Rahn, C.D.; Walker, I.D. Large deflection dynamics and control for planar continuum robots. IEEE/ASME Trans. Mechatron. 2003, 8, 299–307. [Google Scholar] [CrossRef] [Green Version]
  86. Jones, B.A.; Walker, I.D. Kinematics for multisection continuum robots. IEEE Trans. Robot. 2006, 22, 43–55. [Google Scholar] [CrossRef]
  87. Mahl, T.; Mayer, A.E.; Hildebrandt, A.; Sawodny, O. A variable curvature modeling approach for kinematic control of continuum manipulators. In Proceedings of the 2013 American Control Conference, Washington, DC, USA, 17–19 June 2013; pp. 4945–4950. [Google Scholar]
  88. Mahl, T.; Hildebrandt, A.; Sawodny, O. A variable curvature continuum kinematics for kinematic control of the bionic handling assistant. IEEE Trans. Robot. 2014, 30, 935–949. [Google Scholar] [CrossRef]
  89. Camarillo, D.B.; Carlson, C.R.; Salisbury, J.K. Task-space control of continuum manipulators with coupled tendon drive. In Experimental Robotics; Springer: Berlin/Heidelberg, Germany, 2009; pp. 271–280. [Google Scholar]
  90. Penning, R.S.; Jung, J.; Ferrier, N.J.; Zinn, M.R. An evaluation of closed-loop control options for continuum manipulators. In Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA, 14–18 May 2012; pp. 5392–5397. [Google Scholar]
  91. George Thuruthel, T.; Ansari, Y.; Falotico, E.; Laschi, C. Control strategies for soft robotic manipulators: A survey. Soft Robot. 2018, 5, 149–163. [Google Scholar] [CrossRef] [PubMed]
  92. Wang, H.; Chen, W.; Yu, X.; Deng, T.; Wang, X.; Pfeifer, R. Visual servo control of cable-driven soft robotic manipulator. In Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3–7 November 2013; pp. 57–62. [Google Scholar]
  93. Katzschmann, R.K.; Della Santina, C.; Toshimitsu, Y.; Bicchi, A.; Rus, D. Dynamic motion control of multi-segment soft robots using piecewise constant curvature matched with an augmented rigid body model. In Proceedings of the 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft), Seoul, Korea, 14–18 April 2019; pp. 454–461. [Google Scholar]
  94. Till, J.; Bryson, C.E.; Chung, S.; Orekhov, A.; Rucker, D.C. Efficient computation of multiple coupled Cosserat rod models for real-time simulation and control of parallel continuum manipulators. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 6–30 May 2015; pp. 5067–5074. [Google Scholar]
  95. Lang, H.; Linn, J.; Arnold, M. Multi-body dynamics simulation of geometrically exact Cosserat rods. Multibody Syst. Dyn. 2011, 25, 285–312. [Google Scholar] [CrossRef]
  96. Xavier, M.S.; Fleming, A.J.; Yong, Y.K. Finite Element Modeling of Soft Fluidic Actuators: Overview and Recent Developments. Adv. Intell. Syst. 2021, 3, 2000187. [Google Scholar] [CrossRef]
  97. Zhang, Z.; Bieze, T.M.; Dequidt, J.; Kruszewski, A.; Duriez, C. Visual servoing control of soft robots based on finite element model. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 2895–2901. [Google Scholar]
  98. Duriez, C. Control of elastic soft robots based on real-time finite element method. In Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2013; pp. 3982–3987. [Google Scholar]
  99. Largilliere, F.; Verona, V.; Coevoet, E.; Sanz-Lopez, M.; Dequidt, J.; Duriez, C. Real-time control of soft-robots using asynchronous finite element modeling. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 26–30 May 2015; pp. 2550–2555. [Google Scholar]
  100. Runge, G.; Wiese, M.; Günther, L.; Raatz, A. A framework for the kinematic modeling of soft material robots combining finite element analysis and piecewise constant curvature kinematics. In Proceedings of the 2017 3rd International Conference on Control, Automation and Robotics (ICCAR), Nagoya, Japan, 24–26 April 2017; pp. 7–14. [Google Scholar]
  101. Chenevier, J.; González, D.; Aguado, J.V.; Chinesta, F.; Cueto, E. Reduced-order modeling of soft robots. PLoS ONE 2018, 13, e0192052. [Google Scholar] [CrossRef] [Green Version]
  102. Goury, O.; Duriez, C. Fast, generic, and reliable control and simulation of soft robots using model order reduction. IEEE Trans. Robot. 2018, 34, 1565–1576. [Google Scholar] [CrossRef] [Green Version]
  103. Katzschmann, R.K.; Thieffry, M.; Goury, O.; Kruszewski, A.; Guerra, T.M.; Duriez, C.; Rus, D. Dynamically closed-loop controlled soft robotic arm using a reduced order finite element model with state observer. In Proceedings of the 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft), Seoul, Korea, 14–18 April 2019; pp. 717–724. [Google Scholar]
  104. Thieffry, M.; Kruszewski, A.; Duriez, C.; Guerra, T.M. Control design for soft robots based on reduced-order model. IEEE Robot. Autom. Lett. 2018, 4, 25–32. [Google Scholar] [CrossRef] [Green Version]
  105. Huang, W.; Huang, X.; Majidi, C.; Jawed, M.K. Dynamic simulation of articulated soft robots. Nat. Commun. 2020, 11, 1–9. [Google Scholar] [CrossRef]
  106. Calisti, M.; Corucci, F.; Arienti, A.; Laschi, C. Dynamics of underwater legged locomotion: Modeling and experiments on an octopus-inspired robot. Bioinspiration Biomim. 2015, 10, 046012. [Google Scholar] [CrossRef] [PubMed]
  107. George Thuruthel, T.; Renda, F.; Iida, F. First-order dynamic modeling and control of soft robots. Front. Robot. AI 2020, 7, 95. [Google Scholar] [CrossRef] [PubMed]
  108. Kim, D.; Kim, S.H.; Kim, T.; Kang, B.B.; Lee, M.; Park, W.; Ku, S.; Kim, D.; Kwon, J.; Lee, H.; et al. Review of machine learning methods in soft robotics. PLoS ONE 2021, 16, e0246102. [Google Scholar] [CrossRef]
  109. Soliman, M.; Mousa, M.A.; Saleh, M.A.; Elsamanty, M.; Radwan, A.G. Modelling and implementation of soft bio-mimetic turtle using echo state network and soft pneumatic actuators. Sci. Rep. 2021, 11, 1–11. [Google Scholar] [CrossRef] [PubMed]
  110. Melingui, A.; Merzouki, R.; Mbede, J.B.; Escande, C.; Benoudjit, N. Neural networks based approach for inverse kinematic modeling of a compact bionic handling assistant trunk. In Proceedings of the 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE), Istanbul, Turkey, 1–4 December 2014; pp. 1239–1244. [Google Scholar]
  111. Runge, G.; Wiese, M.; Raatz, A. FEM-based training of artificial neural networks for modular soft robots. In Proceedings of the 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), Macau, China, 5–8 December 2017; pp. 385–392. [Google Scholar]
  112. Giorelli, M.; Renda, F.; Calisti, M.; Arienti, A.; Ferri, G.; Laschi, C. Neural network and jacobian method for solving the inverse statics of a cable-driven soft arm with nonconstant curvature. IEEE Trans. Robot. 2015, 31, 823–834. [Google Scholar] [CrossRef]
  113. Thuruthel, T.G.; Shih, B.; Laschi, C.; Tolley, M.T. Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot. 2019, 4, eaav1488. [Google Scholar] [CrossRef]
  114. Zhang, Y.; Gao, J.; Yang, H.; Hao, L. A novel hysteresis modelling method with improved generalization capability for pneumatic artificial muscles. Smart Mater. Struct. 2019, 28, 105014. [Google Scholar] [CrossRef]
  115. Soliman, M.; Saleh, M.A.; Mousa, M.A.; Elsamanty, M.; Radwan, A.G. Theoretical and experimental investigation study of data driven work envelope modelling for 3D printed soft pneumatic actuators. Sens. Actuators A Phys. 2021, 331, 112978. [Google Scholar] [CrossRef]
  116. Thuruthel, T.G.; Falotico, E.; Renda, F.; Laschi, C. Learning dynamic models for open loop predictive control of soft robotic manipulators. Bioinspiration Biomim. 2017, 12, 066003. [Google Scholar] [CrossRef] [PubMed]
  117. Gillespie, M.T.; Best, C.M.; Townsend, E.C.; Wingate, D.; Killpack, M.D. Learning nonlinear dynamic models of soft robots for model predictive control with neural networks. In Proceedings of the 2018 IEEE International Conference on Soft Robotics (RoboSoft), Livorno, Italy, 24–28 April 2018; pp. 39–45. [Google Scholar]
  118. Thuruthel, T.G.; Falotico, E.; Manti, M.; Laschi, C. Stable open loop control of soft robotic manipulators. IEEE Robot. Autom. Lett. 2018, 3, 1292–1298. [Google Scholar] [CrossRef]
  119. Boyer, F.; Porez, M.; Khalil, W. Macro-continuous computed torque algorithm for a three-dimensional eel-like robot. IEEE Trans. Robot. 2006, 22, 763–775. [Google Scholar] [CrossRef] [Green Version]
  120. Franco, E.; Garriga-Casanovas, A. Energy-shaping control of soft continuum manipulators with in-plane disturbances. Int. J. Robot. Res. 2021, 40, 236–255. [Google Scholar] [CrossRef]
  121. Hyatt, P.; Johnson, C.C.; Killpack, M.D. Model reference predictive adaptive control for large-scale soft robots. Front. Robot. AI 2020, 7, 132. [Google Scholar] [CrossRef] [PubMed]
  122. Diteesawat, R.S.; Fishman, A.; Helps, T.; Taghavi, M.; Rossiter, J. Closed-loop Control of Electro-ribbon Acutators. Front. Robot. AI 2020, 7, 144. [Google Scholar] [CrossRef] [PubMed]
  123. Schiller, L.; Seibel, A.; Schlattmann, J. A gait pattern generator for closed-loop position control of a soft walking robot. Front. Robot. AI 2020, 7, 87. [Google Scholar] [CrossRef] [PubMed]
  124. Bhagat, S.; Banerjee, H.; Ho Tse, Z.T.; Ren, H. Deep reinforcement learning for soft, flexible robots: Brief review with impending challenges. Robotics 2019, 8, 4. [Google Scholar] [CrossRef] [Green Version]
  125. Zhang, H.; Cao, R.; Zilberstein, S.; Wu, F.; Chen, X. Toward effective soft robot control via reinforcement learning. In International Conference on Intelligent Robotics and Applications; Springer: Berlin/Heidelberg, Germany, 2017; pp. 173–184. [Google Scholar]
  126. Soter, G.; Conn, A.; Hauser, H.; Rossiter, J. Bodily aware soft robots: Integration of proprioceptive and exteroceptive sensors. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 2448–2453. [Google Scholar]
  127. Shih, B.; Shah, D.; Li, J.; Thuruthel, T.G.; Park, Y.L.; Iida, F.; Bao, Z.; Kramer-Bottiglio, R.; Tolley, M.T. Electronic skins and machine learning for intelligent soft robots. Sci. Robot. 2020, 5, eaaz9239. [Google Scholar] [CrossRef] [PubMed]
  128. Zambrano, D.; Cianchetti, M.; Laschi, C.; Hauser, H.; Füchslin, R.; Pfeifer, R. The Morphological Computation Principles as a New Paradigm for ROBOTIC design. Opinions and Outlooks on Morphological Computation. 2014, pp. 214–225. Available online: https://philpapers.org/rec/HAUOAO (accessed on 1 January 2022).
  129. Hauser, H.; Ijspeert, A.J.; Füchslin, R.M.; Pfeifer, R.; Maass, W. Towards a theoretical foundation for morphological computation with compliant bodies. Biol. Cybern. 2011, 105, 355–370. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  130. Hauser, H.; Ijspeert, A.J.; Füchslin, R.M.; Pfeifer, R.; Maass, W. The role of feedback in morphological computation with compliant bodies. Biol. Cybern. 2012, 106, 595–613. [Google Scholar] [CrossRef] [Green Version]
Micromachines 13 00110 g001 550
Figure 1. (a) The underwater forces acting on the fish during swimming. (b) Fish anatomy showing the different fins fish use to swim and stabilize.
Figure 1. (a) The underwater forces acting on the fish during swimming. (b) Fish anatomy showing the different fins fish use to swim and stabilize.
Micromachines 13 00110 g001
Micromachines 13 00110 g002 550
Figure 2. An overview of the different modeling and control techniques used in soft robotics.
Figure 2. An overview of the different modeling and control techniques used in soft robotics.
Micromachines 13 00110 g002
Table
Table 1. Classification of various underwater soft robotic systems.
Table 1. Classification of various underwater soft robotic systems.
ReferenceRobotBiomimicryActuationSwimmingCompliance
[37]Multi-Joint FishCarangiform FishElectric Actuators (Servomotors)BCF UndulationMedium
[69,70]Biomimetic FishFishIPMCBCF/MPF OscillationMedium
[40,59,63]SoFiFishFEA (Pneumatic/Hydraulic)BCF UndulationHigh
[41]Stingray RobotStingrayElectric Actuators (Servomotors)MPF UndulationMedium
[51]Octopus ArmOctopusMotor-driven CablesCrawlingHigh
[72]Octopus ArmOctopusMotor-Driven Cables/SMA Springs-High
[76]Octopus RobotOctopusMotor-Driven Cables/SMACrawlingMedium
[66]Cuttlefish RobotCuttlefishDEAJet PropulsionMedium
[74]RobojellyJellyfishSMAPropulsionHigh
[61]OctobotOctopusFEA (Chemical Reaction)-High
[64]Morphing Underwater Walking Robot-FEA (Hydraulic)Walking/CrawlingMedium
[67]Jellyfish-Inspired Soft RobotJellyfishDEAPropulsionHigh
[69]Robotic Manta RayManta RayIPMCMPF UndulationMedium
[75]Micro Biomimetic Manta RayManta RaySMAMPF UndulationMedium
[71]Starfish RobotStarfishSMA WiresPropulsionHigh
[77]Starfish-Like Soft RobotStarfishSMACrawlingHigh
[78]RoboScallopScallopFEAJet PropulsionMedium
[79]Eel-like RobotLeptocephalus (Eel Larva)Fluid Electrode DEA (FEDEA)BCF UndulationHigh
[80]Morphing Limb Amphibious Turtle RobotTurtle/TortoiseVariable Stiffness Material-pneumatic ActuatorsDrag-induced Swimming/WalkingMedium
[81]FinRay Robotic JellyfishJellyfishFinRay Actuators driven with ServomotorsPropulsionMedium
[82]PATRICK: Brittle Star-Inspired Soft RobotBrittle StarSMA WiresCrawlingHigh
[83]Soft Underwater StarfishStarfishServo-driven Tendon WiresPropulsionHigh
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Youssef, S.M.; Soliman, M.; Saleh, M.A.; Mousa, M.A.; Elsamanty, M.; Radwan, A.G. Underwater Soft Robotics: A Review of Bioinspiration in Design, Actuation, Modeling, and Control. Micromachines 2022, 13, 110. https://doi.org/10.3390/mi13010110

AMA Style

Youssef SM, Soliman M, Saleh MA, Mousa MA, Elsamanty M, Radwan AG. Underwater Soft Robotics: A Review of Bioinspiration in Design, Actuation, Modeling, and Control. Micromachines. 2022; 13(1):110. https://doi.org/10.3390/mi13010110

Chicago/Turabian Style

Youssef, Samuel M., MennaAllah Soliman, Mahmood A. Saleh, Mostafa A. Mousa, Mahmoud Elsamanty, and Ahmed G. Radwan. 2022. "Underwater Soft Robotics: A Review of Bioinspiration in Design, Actuation, Modeling, and Control" Micromachines 13, no. 1: 110. https://doi.org/10.3390/mi13010110

APA Style

Youssef, S. M., Soliman, M., Saleh, M. A., Mousa, M. A., Elsamanty, M., & Radwan, A. G. (2022). Underwater Soft Robotics: A Review of Bioinspiration in Design, Actuation, Modeling, and Control. Micromachines, 13(1), 110. https://doi.org/10.3390/mi13010110

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop