A Mobile Gait Training System Providing an Active Interaction

Abstract

In this paper, we propose an interactive mobile gait training system that allows trainees to interact actively with its content (the gait training content). The proposed system is a new type of gait training one combining a mobile robot with virtual reality (contents). It is a mobile system that projects virtual contents (for example, virtual footprints) for gait training on the actual ground (or floor). The performance and effectiveness of the proposed system were examined through a trainee’s foot recognition test and usability evaluation. The test results confirmed that the proposed system showed an average recognition ratio of more than 97%, meaning that the system could accurately recognize the trainee’s foot. In addition, as a result of usability evaluation, the overall satisfaction was 86%, confirming that the proposed system is effective.

1. Introduction

Stroke is a medical condition that occurs when the blood supply to part of the brain is blocked, causing functional impairment of the body [1]. Symptoms in stroke trainees typically include weakness in lower extremity muscle strength [2]. According to previous studies, stroke trainees with weakened extremity muscle strength have abnormal gait patterns, and the speed and duration of gait decrease [3]. Gait function plays an important role in the quality of stroke trainees’ lives, and the decreased gait function is reported to cause a great sense of loss to trainees with stroke symptoms [4]. Therefore, gait training to strengthen the gait function of stroke trainees is very important in improving the health and quality of life of chronic stroke trainees in a long-term perspective [5]. The representative method of gait training is traditionally based on treadmills and robots [6,7,8,9]. Gait training using a treadmill provides training effects by stimulating gait patterns symmetrically [10]. With the robot that is pre-programmed with ordinary gait style of general people inducing the movement of the trainee’s lower extremities, gait training based on robots provides a training effect that allows trainees to experience ordinary gait [11].

Recently, there has been increasing studies to enhance the interest and immersion of trainees by adding virtual reality (VR) gait training content to these traditional gait training systems [12,13]. As virtual reality-based interactive training systems can provide strong feedback effects to trainees and improve motivation and adaptability [14,15,16]. LokoMat [17] is a state-of-the-art customized rehabilitation robot that are different from conventional methods that require traditional auxiliary equipment such as guardians or cane. It allows trainees to walk on a treadmill and provides feedback through a monitor screen via VR content. Walkbot [18] is a rehabilitation robot for trainees with gait disabilities. As a wearable robot, it supports rehabilitation training that focuses on trainees’ joints (knee joint, hip joint, and ankle joint) and children. The G-EO [19] robot provides gait training that simulates gait situations on flat land, slopes, and stairs, and the trainer can easily customize simple content settings for individual trainees, providing efficient training in a short time. Cosmos Robowalk [20] provides gait training using front and rear systems together. Training can be conducted by adjusting the angle of the support/resistance cable either vertically or horizontally. Rehawalk [21] is a robot designed for the analysis and treatment of gait disabilities in neurology, orthopedic, and elderly rehabilitation. Trainees with significantly reduced gait function (including ones in a wheelchair as well) can utilize it, and all training elements can be set by trainers. However, since these systems are carried out on a treadmill, they force the trainee to walk at a certain speed or higher and provide content at a set speed.

Table 1. Comparison of existing and proposed systems.

Attributes	Lokomat [17]	Walkbot [18]	G-EO [19]	Robowalk [20]	Rehawalk [21]	Proposed System1 [22] (80%)	Proposed System2 (98%)
Immersion	O	O	O	O	O	O	O
Activation	X	X	X	X	X	O	O

shows the results of classifying existing systems (i.e., treadmill systems) into some categories and comparing them with our systems in terms of two attributes (immersion and activeness). Most immersive (virtual) gait contents by the existing systems are provided in a passive way in which its content and training are conducted regardless of a trainee’s gait.

Therefore, in this paper, we propose an interactive mobile gait training system that allows trainees to interact actively with the gait training content. The proposed system recognizes the trainee’s gait and thus enables the interaction in which the training content responds. Furthermore, the proposed system enables active gait training of the trainer by continuously updating training content along the distance the trainee walked. The paper is composed as follows. Section 2 describes the proposed system. Section 3 conducts experiments to examine the performance of the proposed system and presents the results. The conclusion is given in Section 4.

2. Proposed System

The proposed system is a system that provides interactive training content in real-time according to the trainee’s gait, for which the system consists of (1) Sensing part, (2) Moving part, and (3) Augmenting part, as shown in Figure 1. The proposed system operates as shown in Figure 2. The detailed explanations of the block diagram are described in the following subsections.

The system proposed in this paper is an extended version that improves the low recognition rate (80%→98%) which was lacking in our previous research [22]. Since the previous system recorded an unstable recognition rate of 80%, it was intended to improve this. The system proposed in this study changed the recognition part from our existing system (Image processing method, Camera) to a precision measurement method (Triangulation Method, 2D-LIDAR (Light Detection and Ranging)).

2.1. Sensing Part

The sensing part is responsible for measuring the trainee’s feet coordinates when the trainee interacts with the training content provided by the system. It shown in Figure 2 provides a measurement of 2D pointcloud data and a measurement of foot coordinates. Generally, trainees are accompanied by gait aids (walkers, cane, etc.) because they have difficulty in gait. Therefore, the sensing part of the system distinguishes the trainee’s feet from the assistive device and measures the feet’s coordinates. The sensing part measures the trainee’s feet coordinates based on 2D point cloud data and uses a 2D-LIDAR. as a sensor for data collection. Figure 3 shows the procedure for measuring the feet coordinates of a trainee using the 2D-LIDAR.

First, in step (a), 2D point cloud data are collected in real time through a sensor. In step (b), 2D point cloud data collected through the clustering process is classified into one object unit. In step (c), the gait aid devices and the trainee’s feet are distinguished. All objects classified in the clustering process are used as foot position candidates, and the candidates are compared to find the actual trainee’s foot position. Since gait aids generally have sizes and thicknesses smaller than those of the trainee’s foot, the trainee’s foot classification proceeds based on the size of the objects. By comparing the sizes of the candidates, the two largest candidates are selected as the feet of the trainer. Finally, in step (d), among the selected two candidates, the candidate on the right is recognized as the right foot, and the candidate on the left is recognized as the left foot.

Through the above procedure, the sensing part measures the coordinates of the trainee’s feet in real time and transmits them to the moving part and the augmenting part, allowing the robot to automatically move and the trainee to interact with the content.

2.2. Moving Part

The moving part is responsible for moving the system and transmitting the training distance to the augmenting part. It keeps the distance between the robot and trainee maintained constantly based on a trainee’s position and a robot’s travel distance. The moving part uses a mobile robot and operates according to the procedure shown in Figure 4.

In step (a), the distance between the system and the trainee is calculated. Both foot coordinates are received from the sensing part, and the average value of the data is calculated as the distance between the trainee and the system. In step (b), the direction of driving is determined based on the calculated distance. If the distance between the system and the trainee is less than 1 m, the system drives straight forward, or if the distance exceeds 1 m, it drives backward. In step (c), the system moves the robot. The robot keeps the 1 m distance by going straight and backward in the pre-programmed direction. Finally, in step (d), the gait distance of the trainee is calculated through the amount of change in the position of the robot. When the robot is automatically moved by the distance-keeping algorithm, the robot’s self-position recognition function is used to accurately calculate the robot’s position and travel distance. The self-position recognition function is a function of calculating the position of the robot itself through measurements such as the diameter of the wheels specified in the robot’s specifications, RPM, and the number of wheels the motor has rotated. Through this function, the distance traveled by the robot is measured in real time. Since the robot automatically moves the distance as the trainee moves, the trainee’s gait distance can be obtained through the amount of change in the robot’s position.

Through the above procedure, the moving part continuously updates training content according to the distance the trainee walked, enabling active gait training.

2.3. Augmentng Part

The augmenting part is responsible for augmenting the virtual content on the floor, providing interaction with the trainee, and updating it in real time according to the training distance. That is, it takes roles in matching a coordinate system, generating a virtual object on foot coordinates (called virtual guide), and updating it based on training distance. In this system, the content augmented on the floor is displayed in a size of 150 cm × 250 cm. In order to provide interaction within the content to the trainee, the trainee’s feet coordinates must be generated as objects inside the content. The above function operates according to the procedure shown in Figure 5.

First, in step (a), both feet coordinates are received from the sensing part, and the system matches the coordinates from the sensing part and the augmenting part to use the trainee’s both feet coordinates for content. Coordinate matching between the sensing part and the augmenting part is performed using a checkboard. The checkboard is projected according to the size of the projected content. Calculate the number of the projected checkboard squares (C_x) of the horizontal length and the number of checkboard squares (C_y) of the vertical length. During this process, the projected checkboard quadrilateral is a square and has a length of A in the content. Using this, the proportional expression is as shown in Equation (1).

$$150cm=C_x\times A, 250cm=C_y\times A$$

(1)

In step (b), a virtual object for tracking the position of the trainee’s feet in real time is generated using Equation (1). Through the above process, virtual feet objects centered on each foot coordinates of the trainee are generated to interact with objects inside the content. In addition, for real-time content updates based on the trainee’s gait distance, the trainee’s gait distance must be calculated from the coordinates of the robot which are transmitted to the augmenting part.

The above function operates according to the procedure shown in Figure 6. First, in step (a), the travel distance of the robot is received from the moving part, and the gait distance of the trainee is calculated accordingly. After this step, in order to reflect the trainee’s gait distance in the content, the system matches the coordinates between the moving part and the augmenting part. As described previously, a checkboard is used for coordinate matching. Project the checkboard as the size of the projected content, and move the robot forward by R_y. The proportional equation is obtained by counting the number of checkboard squares that the robot passed. In step (b), the content is updated in real time according to the travel distance of the trainee by moving the position of the virtual camera inside the content using Equation (2).

$$R_y=C_y\times A$$

(2)

Through the above process, virtual feet objects centered, respectively on the trainee’s two-foot coordinates can be created to interact with the objects inside the content, and the content can be updated according to the trainee’s travel distance.

3. Experiment

In this section, in order to examine the performance and usefulness of the proposed system, trainee’s feet recognition experiments and usability evaluation were performed.

3.1. Experimental Environment

The experimental environment is shown in Figure 7. The experiment was conducted on 10 people, and 100 virtual footprints (gait guides) were augmented within a space of 1 m × 20 m, as shown in Figure 7. After that, it was confirmed whether the trainee’s feet were properly recognized in the virtual footprints. The experiment was conducted with an ordinary participant on a gait aid although the ultimate goal of this system is to provide training to actual brain lesion patients. To prevent wrong recognition for the trainee’s feet, no object except only a trainee is allowed within the space of the range is 1 m × 20 m as shown in Figure 7.

3.1.1. Experiment 1: An Experiment for Foot Recognition

Prior to usability evaluation, trainee feet recognition experiments are performed to verify the performance of the system. This experiment is to determine whether the system accurately recognizes the trainee’s feet and interacts with augmented content on the ground. Before verifying the foot recognition rate of the proposed system, the accuracy was examined and compensated by comparing the known coordinates (i.e., denoted on the floor) and the measured ones obtained using the 2D-LiDAR. In the experiment, the cookie-shaped virtual footprints were augmented as shown in Figure 8, and the experiment was conducted through the corresponding content.

The criterion for whether the recognition is successful is shown in Figure 9. When the trainee steps on the virtual footprint, the ratio of the area where the trainee’s foot object overlaps the virtual footprint is calculated. If the calculated ratio is more than 30%, and overlapping lasts for 0.5 s or more, it is classified as a success. On the other hand, if the calculated rate is less than 30%, and maintained for less than 0.5 s, it is classified as failed.

Since the proposed system projects virtual content on the floor, recognition performance (recognition rate) may be affected by intensity of illumination. Therefore, the experiment 1 was conducted under three illuminance (50 lux, 100 lux, 150 lux) considering the various illuminance environments of Korean hospitals.

3.1.2. Experiment 2: Usability Evaluation

Usability evaluation was performed to verify the usefulness of the system. This evaluation is carried out according to the procedure (20 min per person) shown in Figure 10.

Previous to the experiment, the test subjects were informed about the experiment for three minutes and prepared a consent form. In this experiment, the test subjects were to participate in gait training content for 5 min. After resting for 2 min, the test subjects participated in gait training content for 5 min while wearing a walker to verify whether the content was actually useful. In the usability evaluation, the crosswalk concept as shown in Figure 11 was provided as the gait training content.

After the experiment, a survey was conducted to verify the effectiveness of the system, and the questions are shown in

Table 2. Questionnaire used for usability evaluation.

No			Questionnaire	Answer Negative ↔ Positive
System		1	Gait training system maintained a certain distance	1-2-3-4-5-6-7
		2	The speed of movement of the system was appropriate	1-2-3-4-5-6-7
		3	System was safe	1-2-3-4-5-6-7
Content	Feedback	4	Positive feedback helped improve the trainee’s sense of accomplishment	1-2-3-4-5-6-7
		5	Negative feedback helped correct a trainee’s misguided gait	1-2-3-4-5-6-7
		6	Encouraging feedback allowed the trainee to carry out the gait training to the end	1-2-3-4-5-6-7
	Virtual footprints	7	The gait guide spacing of the gait training system was appropriate	1-2-3-4-5-6-7
		8	The size of the virtual footprints projected onto the ground by system was appropriate	1-2-3-4-5-6-7

. In the questionnaire, a total of eight questions were asked whether the basic functions provided in the content were appropriate and suitable for training. Non-validated questionnaires were used to carry out certain measures of interest (that is, the basic functions provided by the proposed system) in the experiment.

Questionnaire 1 (Q1) confirms whether the moving part’s distance maintenance function is properly performed (i.e., whether the set distance is properly maintained). Questionnaire 2 (Q2) checks whether the robot’s moving speed (0.3 m/s) is appropriate for gait training. In addition, Questionnaire 3 (Q3) verifies whether the system is safe. Confirm whether the feedback provided is effective through Questionnaire 4 (Q4), Questionnaire 5 (Q5), and Questionnaire 6 (Q6). The feedback provided in Questions 4, 5, and 6 is shown in Figure 12. Questionnaire 7 (Q7) and Questionnaire 8 (Q8) check whether the walking guide is appropriate. A total of eight questions as above confirm whether the function of the system is useful for gait training.

3.2. Experimental Results

3.2.1. Result of Foot Recognition Experiment

The experiment on recognition performance was conducted on 10 subjects with three illuminations under the premise that the foot should be recognized for 1 s with more than 30% overlapping with virtual gait content (such as virtual footprints).

As a result of the recognition experiment, as shown in Figure 13 the recognition rate was 98.3% for 50 lux, 98.2% for 100 lux, and 98.4% for 150 lux. Whether the recognition performance of the proposed system remains robust against various illuminance values was reviewed through the Kruskal–Wallis Test, a nonparametric test. The result of the Kruskal–Wallis test is shown in

Table 3. Result for foot recognition experiment—Kruskal–Wallis Test.

Variable	Value
H	0.635
Df	0.635
p-value	0.727

and showed that H = 0.635 and p = 0.727, and the null hypothesis that the recognition rate is the same for the three illuminations is confirmed as the p value is greater than 0.05. It is statistically confirmed that the proposed system can maintain a recognition performance above a certain level (98% or more on average) for the above three illuminations.

3.2.2. Result of the Usability Evaluation

Usability evaluation was conducted as shown in Figure 11 below. By providing the trainee with highlighted virtual footprints to be stepped on, and by providing positive feedback with the next virtual footprints when the trainee stepped on the virtual footprint correctly (as described earlier, it is considered correct if the foot is recognized for 1 s with more than 30% overlapping with the virtual footprint (gait guide)). When recognition failed (if the trainee did not step properly on the provided virtual footprint), it provided negative feedback (incorrect) and encouraging feedback at the same time, allowing the training to continue. The questions for usability evaluation analysis were classified into three types as follows. Q1, Q2, and Q3 were classified as evaluation types for the system, Q4, Q5, and Q6 as evaluation types for feedback of the content, and finally Q7 and Q8 as evaluation types for virtual footprints (walking guide) of the content. The evaluation results for each of the three types are shown in Figure 14 below.

As a result of the evaluation, it was confirmed that the satisfaction of the system was higher than that of the content in general. The Kruskal–Wallis test was performed to confirm whether the measured results were significant, and the results are shown in

Table 4. Result for usability evaluation—Kruskal–Wallis Test.

Variable	Value
H	3.21
Df	2
p-value	0.2

.

As a result of the Kruskal–Wallis test, it was confirmed that the satisfaction on the three evaluation types was not statistically significant as the p value was greater than 0.05. In other words, since the three types do not have a significant effect on satisfaction, the questions cannot be analyzed by types. Therefore, the 7-point scale of each question was converted into a percentage and then analyzed as follows based on the average value.

Result for usability evaluation was shown in

Table 5. Result for usability evaluation.

Variable	AVG	SD	SE
Q1	83.375	19.912	7.04
Q2	85.4125	13.89	4.91
Q3	93.75	12.39	4.38
Q4	89.5875	19.79	6.99
Q5	81.2375	16.51	5.83
Q6	83.325	26.73	9.45
Q7	87.5	14.76	5.21
Q8	85.4125	18.76	6.63
All	86.19

. Q1 was a question about maintaining the distance of the moving part, scored an average of 83.375 points, and the SD and SE were second highest. The results imply that an error may occur due to a slip phenomenon when the robot is moving. Q2 was a question about the speed of the system movement, with an average score of 85.4125 points, indicating that the speed of the system movement is appropriate. Q3 was a question about the safety of the system and scored an average of 93.75 points. As the safety of this system was verified, it is thought that it may be used for actual gait training later on. Q4 was a question about positive feedback, and the highest average among feedback, 89.5878 points, was obtained. With the positive feedback, it is confirmed that the trainee could have the greatest motivation. Q5 obtained 81.2375 points with the lowest average as a question about negative feedback, meaning that it is confirmed that negative feedback was useful for gait training, but not helpful compared to other feedback (positive, encouraging). Q6 was a question about encouraging feedback and scored an average of 83.325 points. For Q6, 33.3 points (3 points on a 7-point scale) as the lowest, 100 points (7 points on a 7-point scale) as the highest, and SD and SE appeared to be the highest. This suggests that the motivation effects among trainees may appear differently. Since it can be either a disturbance for some trainees or a motivating factor for others, it would be necessary to provide more general encouragement feedback before applying it to actual gait training. Q7 was a question about the spacing of the gait guide (virtual footprints) and earned 87.5 points which was the second highest average. Q8 was a question about the size of the gait guide (virtual footprints) and earned 85.4125 points. These scores imply it is confirmed that the gait guide may be helpful for training.

The overall satisfaction was 86.19%, and the usefulness of the system could be verified through usability evaluation.

4. Conclusions

In this paper, we proposed an interactive mobile gait training system that allows trainees to interact actively with the gait training content. The proposed system consisted of three parts: a sensing part, a moving part, and an augmenting part. The performance and effectiveness of the proposed system were reviewed and evaluated through two tests (trainee’s feet recognition and usability) using 10 subjects.

The foot recognition experiment was performed on a total of three illuminations, and it was confirmed through the Kruskal–Wallis test that the recognition rate was not affected by illumination. It was confirmed that the recognition rate was more than 97% for all illuminance and that the trainee’s feet could be accurately recognized through the system.

Usability evaluation was performed on a total of three types (system, feedback, and virtual footprint), and the Kruskal–Wallis test confirmed that the evaluation types had no significant impact on satisfaction. Therefore, the analysis for each item was performed, and as a result, an average of 86% or more satisfaction was obtained, and it was confirmed that the system was useful for gait training.

As this study is a result obtained from ordinary subjects, future studies need to expand the experiments on actual patients. One of the future studies is to verify the effectiveness of the proposed system by conducting with more than 20 children with brain lesions. Additionally, the system’s performance on different gait speeds would be examined as one of future works.