Virtual Teleoperation System for Mobile Manipulator Robots Focused on Object Transport and Manipulation

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

In this manuscript. The authors developed a virtual teleoperation system for mobile manipulation robots focused on object transport and manipulation. An omnidirectional platform and an anthropomorphic robotic arm with 4 degrees of freedom were integrated into the virtual environment. This work provides a cheaper and safer solution to the robot usage training. Some small issues still need to be fixed before consideration of acceptance.

1.      Is it possible to evaluate the difference between the real robot and the restored virtual one? If so, how?

2.      There are some errors in the Reference Part. For example, page numbers are missed in Ref. 1. Please go through this part and correct any errors.

Author Response

A file with the reviewer's reply is attached.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Detail comments:

Line 29: “psycho therapeutic” is spelled “psychotherapeutic.”

Line 29: “and for the treatment of diseases such as COVID-19, the Internet of Medical Things” – why Covid is special case here when it does not require any online monitoring for long time? - I would suggest being more general in this part and saying “and for treatment of diseases requiring prolonged patient observation that can be handled by online monitoring”.

Lines 45-55: Authors mention Lego and STEM toys that can be used for educational purposes. This is good point, however it is worth mentioning also their limitations and the need for real robotic systems, or something a bit more capable of load handling, as then the case of simulation becomes much stronger. With toys simulation could aid development of the system in terms of function implementation and debugging logic, but when the robot can do actual handling of relatively heavy objects or collide with a human/object causing damage there is another layer of difficulty – how to handle strong robot control in the presence of people and other robots and static objects that can get damaged. It must be ensured that the robot cannot harm anyone, by providing safety features and sensing. Development of the sensor placement is tricky in physical prototypes, and it requires virtual environment or a lot of trial and error and that makes a strong case to use simulation. Think of a reference to such systems that could be included here.

Lines 75-80: Authors mention first need for operation in unstructured environment and then conclude that for transportation and logistics the environment needs to be constrained. It helps to have a constrained environment as then certain assumptions simplify the control procedure. However, in unconstrained environment one has to expect anything, as the base assumptions have to be very limited to avoid failure if they are violated in reality. That paragraph should be rephrased to make it clear how much constraints one really needs to make things work and how far from unconstrained environments is the current development of robotic systems. Some examples would be beneficial.

Line 80: “trans-parent” is spelled as “transparent.”

Line 84: “This allows us to develop a mental model” what do authors mean by “mental model”? I would assume you are refereeing to “mathematical model” or “kinematical model”.

Lines 85-86: “Subsequently, we build a formal model based on physical and geometric principles, resulting in the mathematical model of two independent robots.” – You mentioned about a single robot but you get two models of it, what are the differences between those models, is one just geometry for collision detection and the other is kinematics or is it something else. Please clarify in the text.

End of section 2 would require a bit more detail on what is actually part of your work output. The use of Unity is clear, but what did you build there? How much work user has to put to simulate a robot in your project? Is it accepting models in some specific format (visualization and functional) or is everything hardcoded with scripts either in Unity or Matlab? Assume the reader doesn’t know what you did, have not seen it yet and needs to have a picture of what can be expected from your framework. It is sufficient to mention it as a bullet list, but could be a paragraph or two of text as well.

Line 103: “have the ce camcity and dexterity” there are typos.

In equation 3 there are S1, S2…, C1, C2.. terms that are not explained. Please provide explanation what they are.

At the end of line 160 one symbol does not fit within line and goes to the margin of the page.

Line 260 “robot , such” there appears to be a space before comma.

Figure 11: The label in the figure “Menu” contains accent over u.

Summary and general comments:

The idea of robot simulation combined with the haptic control is really great. Especially in the education domain that the authors address. In figure 13 authors show the division of the tasks between software pieces and in figure 15 it is visible that the UI was designed in Matlab while visualization comes from Unity. The teleoperation in my opinion in this context is a bit of a stretch. The robot could be in theory real and located in remote location, but then the shared memory concept would have to be replaced with TCP/IP or UDP communication to allow for teleoperation. In current implementation it is all on one machine and the restriction comes from the software communication design, so it is a hard constraint. Therefore, from the title of the paper I would remove“teleoperation” as it is misleading. Otherwise, the title is fine.

Secondly, it is not clear where the robot dynamics is implemented, it should be shown if it is part of Matlab or Unity implementation.

You did not discuss how grasping is implemented. Is it a constrain on/off for attaching object to the end effector, or is it friction-based implementation? That is important concept and should be addressed in the paper.

Thirdly, what part of implementation is actually done by the authors, I’m guessing the dynamics is part of Unity engine based on model provided by the user – is that correct? Shared memory communication between Matlab and Unity is probably authors’ implementation, similarly like the user interface in Unity and Matlab. This could be listed in the paper for clarity or be part of a figure and discussed in the text.

How about the autonomous control? – it is mentioned in the figure but what does it mean in the implementation, how much autonomy there really is? What can be done automatically, and what requires manual operation?

Authors mentioned exporting animations from 3D studio Max to Unity, are the animations used for driving the robot’s joints motions? Authors do mention torque actuation in some place.

How is the driving base moving over the ground, is there friction between omni wheels and ground that determines the motion or is it based on kinematic model motion (how is the dynamics then included) or is the base simply sliding over the area of the floor?

Gamification of the learning process is a good point, so definition of target zone and scoring for good and bad manipulation. Is the bad manipulation happening only if the user drops the object in the wrong place, or are there any other categories that define failure like collisions with environment? If there is anything more to this part it should be described.

What are the plans regarding your implementation distribution, is it going to be open-source, or is it only a tool that you intend to use internally? It goes without saying that open-source is a massive impact booster for any work that involves software.

Supplementary material like video showing example manipulation exercise would be beneficial. A demo showing how the environment modeling and robot modeling is done could also be of high interest among readers. If there is any autonomy implemented then how it is used should be shown.

I see potential in this work, just from reading the paper, there are some parts that are well described and some that raise questions about details. Therefore, I would be for the publication of this work after reviewing the comments from reviewers and implementing modifications to the text.

Comments on the Quality of English Language

Language is good and understandable, there are a few errors that I have indicated in my detailed comments and suggestions.

Author Response

Dear reviewer, thank you for your comments and suggestions. Please find attached the file with the changes made to the new manuscript.

Author Response File: Author Response.pdf