Development of Serious Games for the Rehabilitation of the Human Vertebral Spine for Home Care

Abstract

With the occurrence of pandemics, such as COVID-19, which lead to social isolation, there is a need for home rehabilitation procedures without the direct supervision of health professionals. The great difficulty of treatment at home is the cost of the conventional equipment and the need for specialized labor to operate it. Thus, this paper aimed to develop serious games to assist health professionals in the physiotherapy of patients with spinal pain for clinical and home applications. Serious games integrate serious aspects such as teaching, rehabilitation, and information with the playful and interactive elements of video games. Despite the positive indication and benefits of physiotherapy for cases of chronic spinal pain, the long treatment time, social isolation due to pandemics, and lack of motivation to use traditional methods are some of the main causes of therapeutic failure. Using Unity 3D (version 2019.4.24f1) software and a personal computer with a webcam, we developed aesthetically pleasing, smooth, and attractive games, while maintaining the essence of seriousness that is required for rehabilitation. The serious games, controlled using OpenPose (version v1.0.0alpha-1.5.0) software, were tested with a healthy volunteer. The findings demonstrated that the proposed games can be used as a playful tool to motivate patients during physiotherapy and to reduce cases of treatment abandonment, including at home.

1. Introduction

Responsible for sustaining the human bipedal position, the spine plays a key role in posture, weight-bearing, locomotion, and protection of the spinal cord and nerve roots [1]. Spinal movements are divided into flexion and extension in the sagittal plane, right and left lateralization in the coronal plane, and rotation or circular in the longitudinal plane [2].

A recent report estimated that approximately 34.3 million adults over 18 years of age in Brazil have a chronic back problem [3]. According to the Global Burden of Disease, low back pain is the leading cause of chronic noncommunicable pain, with neck pain ranking ninth, in terms of years lived with disability [4]. These chronic spinal injuries, popularly known as “back pain”, include neck and low back pain, as well as more specific conditions such as spondylosis, radiculopathies, and intervertebral disc disorders [5].

Given the number and diversity of cases, back pain is one of the greatest demands on the health service [6]. With this high demand, physiotherapy services stand out in the health sector for the treatment of this condition. Physiotherapy for patients with back pain aims to control the pain intensity, increase range of motion, teach return to work activities, relieve tension in the affected muscles, reorganize muscle activity patterns through agonist and antagonist action, and maintain balance and spinal functions [7,8].

Since the beginning of the COVID-19 pandemic, an increase in the proportion of musculoskeletal complaints, such as back pain, has been observed, as well as a greater number of chronic cases related to the spine. Certainly, musculoskeletal disorders, including muscle weakness, fatigue, and the lifestyle imposed by social isolation, have contributed to this increase [9].

Despite the positive indication and benefits of physiotherapy, the long period required for treatment and lack of motivation to use traditional methods have been reported as reasons for abandoning physiotherapy treatment, which is one of the major causes of therapeutic failure [10,11].

Thus, this paper aimed to develop serious games, to assist health professionals in the physiotherapy of patients with spinal pain for clinical and home applications. These serious games, which are applications that integrate serious aspects with the playful and interactive elements of video games [12], using OpenPose software represent a potential tool for motivating patients during physiotherapy and reducing treatment dropout rates. To achieve this objective, this paper presents the development of serious games using the Unity 3D engine with a plugin for OpenPose, which allows tracking the movements of the human body with a camera. A laptop-type personal computer with a webcam was also used.

The use of serious games for the physical rehabilitation of back pain is an active area of research [13]. A systematic review exploring the use of virtual reality in the treatment of adults with chronic low back pain showed that virtual reality can significantly reduce pain intensity in patients with this condition [14].

The Valedo system, developed by Hocoma [15], is a commercial device used for back training with three wireless inertial sensors. The sensors transfer the trunk movements into a serious game environment and guide patients through exercises specifically designed for low back pain therapy [16]. A previous report indicated that the results of improving lumbopelvic movement control were more effective using this system than with feedback from a mirror or no feedback [17].

Reference [18] showed that training through virtual reality games with the Pro-Kin system (PK 252 N Techno body) resulted in a long-term improvement in clinical and sports performance compared to the other forms of training applied to soccer players with chronic low back pain.

A prior study developed serious games for spine rehabilitation using a Kinect device and Unity 3D. The authors revealed that the serious game was able to determine spine movement angles, with no noticeable delay [19].

In our survey, we found no papers using OpenPose for the rehabilitation of low back pain. Most literature required specific/dedicated equipment to acquire spinal movements, making implementation more difficult and increasing the costs of home care applications. This paper presents contributions for measuring the use of OpenPose in the development of serious games applied in the rehabilitation of the human spine for home care, using a simple webcam connected to a personal computer. The use of serious games described in this paper for back pain rehabilitation can bring economy, motivation, and time reduction to therapist labor.

This paper is divided into a brief presentation of serious games and OpenPose in Section 2, followed by the planning of the serious game as a function of the selected spine movements in Section 3. Section 4 presents the games developed using Unity 3D with OpenPose. Section 5 describes the validation results, and Section 6 summarizes the paper and presents the conclusions.

2. Serious Games and OpenPose Software

Serious games aim to treat patients with the exercises of a conventional physiotherapeutic intervention session, but in a more playful and interactive way, aiming at a greater effectiveness of muscle action, improvements in quality of life, and a reduction in the dropout rates of spinal rehabilitation treatment [20,21].

Notably, the use of serious games is enhanced by technologies that allow human body tracking. Technologies for body motion tracking can use specific equipment without markers, such as Kinect [22,23,24,25]; an algorithm coupled to a webcam; or markers placed on the body. Open-source software for pose estimation is an alternative that can reduce the costs associated with motion capture, as machine learning partially eliminates the need for specialized cameras and equipment [26]. In this paper, the OpenPose software was used.

Responsible for game control, OpenPose was the first open-source real-time software for estimating two-dimensional (2D) human poses in images, including body, foot, hand, and facial key points [26]. This algorithm has a plugin for Unity 3D that allows its use in game development [27].

OpenPose uses the entire image as input to a convolutional neutral network (CNN) to predict 2D confidence maps and detect body parts [26]. Figure 1 illustrates the OpenPose keypoints of the human body skeleton.

The efficiency and accuracy of OpenPose have been reported in the literature [26,27,28,29,30,31], which described the performance when using a set of 2D vector fields that encode the location and orientation of the human body over the image domain [28].

In reference [26], OpenPose was used to perform pose estimation of healthy gait kinematics and spatiotemporal gait parameters, with results that could be used in clinical and research settings when marker-based systems are not available. The hip kinematics and knee kinematics had an error of 3.7 ± 1.3 deg and 5.1 ± 2.5 deg (mean ± std), respectively, compared to the Vicon MX-T40 marker system.

In reference [29], Open Pose was used to conduct ergonomic postural assessments, and it produced better results than a Kinect-based system.

In reference [30], OpenPose was compared to a video-based human pose estimation, and the mean absolute error of the sagittal plane hip, knee, and ankle angles was 4.0°, 5.6°, and 7.4°, respectively.

In reference [31], the range of motion of the upper extremity was analyzed using OpenPose and compared with goniometry. Comparing OpenPose with manual measurements, the shoulder abduction was 0.51°, the shoulder elevation was 2.87°, the elbow flexion was 0.38°, the elbow extension was 0.65°, and the wrist extension was 0.78°. The most significant difference was found in wrist flexion, with a mean value of 8.96°.

The accuracy assessments of OpenPose reported in the literature were performed for specific situations [26,27,28,29,30,31]. No studies focusing on applications in the human spine were found.

Used in conjunction with OpenPose, Unity 3D is a well-known game development software used to create 2D and 3D games. Games developed with Unity can be used, with the proper licenses, on different platforms and even embedded in a web page [24].

Unity 3D applies the concept of object composition to a component-based architecture, to customize the game elements [32]. In this model, a game object is specified through the composition of several functionalities, which can be added (or removed). Each functionality is implemented using a component (a class that inherits from a basic component). This generic container is called the Game Object and works as a repository of functionality, or more specifically, components. In the development of games with Unity 3D, assets are used, which are representations of any object that can be used in the game. An asset can be a specific file created outside of Unity, such as a 3D model, an image, an audio file, or any other file supported by Unity 3D [32].

3. Planning and Development of Serious Games for Spine Rehabilitation

During the development of the initial test game following a patient care session, some typical spinal rehabilitation exercises could not be used, due to the limitations of OpenPose, such as the existence of only two points of articulation in the spine (Figure 1) and the technological barriers to the inclusion of these exercises in the project; for example, the use of two or more auxiliary cameras per exercise and the positioning of the cameras required by the exercises, making their application unfeasible. Exercises that could not be addressed included those requiring twisting the spine around its own axis and the cat-cow stretch pose exercise, where the patient mobilizes the entire thoracic spine, significantly increasing the thoracic kyphosis; first upwards slowly and then downwards with hands and knees on the floor.

Six physical therapy exercises were chosen based on the limitations of OpenPose: lumbar extension (McKenzie method) in the lying position (Figure 2a); lumbar extension (McKenzie method) in the upright position (Figure 2b); lumbar flexion in the upright position (Figure 2c); hip extension, lying down (Figure 2d); left lateral flexion (Figure 2e); and right lateral flexion (Figure 2f).

The McKenzie method is an advanced evidence-based physiotherapy assessment and self-treatment technique that offers conditions for the patient to treat himself quickly, safely, and effectively, without relying on medication, heat, ice, ultrasound, or surgery. This approach is known worldwide as the McKenzie Method of Mechanical Diagnosis and Therapy and uses both extension and flexion exercises [33].

The Williams exercises were developed by observing that most patients with chronic low back pain had degenerative skeletal changes secondary to intervertebral disc injuries. Therefore, the Williams exercises use spine and hip flexion as the treatment principle. To reduce pain and stabilize the trunk, the physical exercises actively develop the flexor muscles and passively stretch the lumbosacral extensor muscles. Williams emphasizes that the pelvic tilt is essential for optimal treatment results [34].

Six physical therapy exercises were chosen and divided into four serious games. The asteroids game used lumbar extension (McKenzie method) in a lying position (Figure 2a); the park day game explored lumbar extension (McKenzie method) in the upright position (Figure 2b) and lumbar flexion in the upright position (Figure 2c); the fishing game used hip extension, lying down (Figure 2d). The infinite run game used left (Figure 2e) and right lateral flexion (Figure 2f). The games were developed using free Unity 3D resources and elements built by the authors.

Modeling was carried out so that the health professional responsible for the patient and with knowledge of therapy and rehabilitation can supervise and monitor the stages of the serious games; that is, from the patient’s data entry to the end of the game.

In the main menu, the physical therapist has access to five paths: registration, calibration, games, options, and exit. In the options tab, only informative text about the game is shown, without the need for user interaction. With the exit option, the user exits the menu and games. The registration option directs the user to the second screen, the “NewPatient” scene. This screen allows the user to enter a new patient in the menu database, by inputting patient information.

With the patient’s data entered, the user’s next option is to have the patient calibrate movements intended for the current physical therapy section, where movement limits will be recorded for use in movement control in games. Such calibration is done in the “calibration” scene.

After calibrating the patient’s movements, the next option is to start the game chosen for the current physical therapy section, as previously chosen and planned by the professional. In this environment screen, the player who interacts with the game is the patient/user, represented by the main player avatar of each game. The objectives of each game are always related to the spinal movements required for the patient’s treatment, where visual effects appear in the form of awards for correct movements and staying in a position for the necessary time. This screen also displays the score and time for each game, along with a pause menu for the exclusive use of the therapist. In this menu, which is found in each game, the therapist has access to four paths: continue, main menu, save, and exit.

4. Games Developed Using Unity 3D with OpenPose Software

Through the use of Unity 3D software, it was possible to improve and facilitate the development of aesthetically pleasing, smooth, and attractive games, while maintaining the seriousness of rehabilitation. We created simple and intuitive scripts whenever possible, so the objects would be easy to reference within the game.

The patient interacts with the main character of the games through the movements captured by the camera, either a webcam or another auxiliary camera connected and configured with the computer and processed by OpenPose.

The avatar’s movement is controlled, in real-time, by OpenPose through points two (neck) and nine (central point of the hip) of the column (Figure 1). In this paper, “real-time” is defined as the generation of interactive content using computer graphics that are faster than human perception.

4.1. Asteroids Game

Figure 3 shows the default screen for the asteroids game. In this game, the player controls a character in a spaceship via a camera, in a scenario that represents a battle against asteroids in outer space. These asteroids appear on the screen over time, in predefined positions.

Figure 3 displays, at the top center, a sentence explaining what the player must do in this game as a tutorial. With the game running, it is possible to observe, in the upper right corner, the game scoreboard and its execution time, as well as one of the asteroids to be destroyed.

The avatar is the main character of this game, with the function of simulating in the virtual environment the movements of the spine of the patient user, who performs the physiotherapy exercises in front of the camera. This character is a model freely available online for download through assets. It represents a spaceship and responds to the movement of two specific points on the spine. One of these points, located on the patient’s neck, is responsible for performing the character’s movements. The other point, located in the central part of the patient’s hip, is used as a reference to verify that the patient is correctly positioned to carry out the exercise in question, which is the lumbar extension exercise (McKenzie method), performed with the patient lying on the floor with the front of the body facing the floor.

The character moves between three specific positions to fulfill the exercise. These points are the upper and lower limits of spine movement (adding to these values a positioning margin, both above and below), in which, when moving the column, the patient has visual confirmation that he is positioned correctly (Figure 4). Besides the two limit points described, there is a third point belonging to the lumbar extension exercise, which is a midpoint between the lower and upper limits. This central point is obtained through a simple mathematical formulation.

The Asteroids are the game objects to be destroyed by the player. They appear on the screen at specific and predefined points and periods of time so that the patient performs the movements that the lumbar extension exercise (McKenzie method) requires.

The asteroids are positioned on the screen in the same three positions where the spaceship moves. Each asteroid appears at a predefined time, to make the patient perform the exercise correctly. The asteroid animation gives the player the visual illusion that it is heading towards the spaceship, but it stops moving at a certain point on the screen close to where it first appeared to avoid forcing the patient to perform sudden movements, due to possible nervousness when observing an asteroid approaching very close to the spaceship.

4.2. Park Day Game

Figure 5 shows the main screen of the park day game. In this game, the player controls a character, represented by a child on the left of the screen, who encourages the animals to reach their owners. The scenario represents a wooded park with animals that interact with the user in a way that conveys a feeling of calm and tranquility.

Figure 5 shows the three animals that must be encouraged to reach their respective owners. The left part of the park illustrates the dog, the rabbit, and the cat, which must reach their owners in the respective order on the right part of the park. At the top center, a sentence explains what the player must do, working as a game tutorial. The right part shows the scoreboard for the game. The other animals in the scene intend to immerse the user even further in the context of the park, with some even having a comic nature to entertain the patient.

The avatar is the main character of the game and simulates in the virtual environment the spine movements that the patient performs in front of the camera. This character is a model asset available for free and represents a child who encourages the “pets” to run to their respective owners and responds to the movement of two specific points on the spine, one located on the patient’s neck and the other on the central part of the hip. The two exercises required of the patient are lumbar flexion (upright position) (Figure 2c) and lumbar extension (McKenzie method—upright position) (Figure 2b).

The avatar accompanies the movements made by the user in front of the camera, referring to the lumbar flexion and extension exercises. These movements respect the limits of the angle the patient can perform with their spine, as defined in the calibration scene. The character moves between three specific positions, to achieve the exercise required. Positions are obtained of the patient’s spine at two pre-established points. These points are taken as the upper and lower limits of spine movement (adding to these values a positioning margin, both more and less), with which, when moving the column, the patient has visual confirmation that he is moving correctly during the exercise. The central point is the meeting point of the two exercises and the starting point of both. The avatar’s movement is controlled in real-time using OpenPose with points two (neck) and nine (central point of the hip) of the column (Figure 1). Figure 6 shows the movement of the column controlling the avatar, which, in turn, triggers the movement of animals (secondary characters) in the game. In Figure 6a, the player performs a lumbar extension and moves the game avatar in the top position of the screen. In Figure 6b, the player performs lumbar flexion, moving the avatar in the lower position of the screen.

Points are awarded for the arrival of the pets, initially on the left part of the park, to their owners, on the right part. Each pet that reaches its respective owner adds one point to the counter.

4.3. Fishing Game

Figure 7 shows the default screen for the fishing game, illustrating the characters controllable by the player via OpenPose and the camera, which are a fisherman and his elevator platform, a flying manta ray, in a fishing scenario represented by a landscape with waterfalls, a river, trees in the background, and fish jumping in the water.

This game aims to induce the player to make the necessary movements to correctly perform hip extension in a lying position with the back facing the ground. The player needs to catch as many fish as possible, which are constantly jumping out of the water on the screen (Figure 7).

This avatar represents the aforementioned fisherman, always accompanied by his flying manta ray, and who is only responsible for simulating fishing movements and giving points to the player for catching fish. The fisherman and the manta ray respond to the movement of two specific points on the spine. One of these points, located on the patient’s neck, is responsible for performing the character’s movements. The other point, located in the central part of the patient’s hip, is used as a reference to verify if it is correctly positioned to perform the exercise in question.

This script checks the current position of the player for each frame. If it is within the limits of one of the three pre-established positions, in case the player remains in this position during the execution time of the animation (which is the fisherman making a movement to catch a fish); then a fish is caught, and a point is added to the score. Figure 8 shows the movement of the column controlling the avatar, which in turn performs the action of catching fish.

If the player remains in one of the three predefined positions during the execution time (which is the fisherman moving to catch a fish), a fish is caught and a point is added to the score.

4.4. Infinite Run Game

Figure 9 displays the default screen for the infinite run game. In this game, the character, controllable by the player via OpenPose and the camera, is a robot in a scenario composed of an infinite bridge over the open sea.

This game aims to induce the player to perform the necessary movements to correctly complete left lateral flexion and right lateral flexion exercises in an upright position (Figure 2). In this game, the player needs to collect as many coins as possible. The purpose of this game is to make the player maintain the position required by the proposed exercises for the necessary time. For that purpose, the coins alternately appear on the screen in three predefined positions.

Ten platforms were made for infinite run, and during the game’s execution, the platforms are aligned in sequence, forming an infinite bridge where the avatar runs. At the end of each platform, they are recycled to optimize the computational cost, making it lighter and run more smoothly.

The game controls and counts the distance covered by the character, estimated by converting the execution time into a proportional distance in meters, giving the player the impression of being in a race and thus increasing immersion in the game. Furthermore, there is control over the number of coins the player collects as he travels across the bridge.

Figure 10 shows the movement of the column controlling the avatar, which moves on the track, collecting the coins.

Importantly, the movement limits of the game are defined by health professionals, with a follow-up in the clinic or via video call in the event of pandemics, which lead to social isolation.

5. Results

Validation of the angles obtained by OpenPose during the execution of the serious games was carried out using a Miotec goniometer with a precision of ±0.05° and an acquisition rate of 2000 Hz.

The experiments were approved by the local ethics committee, and the participant provided written informed consent prior to testing. The test subject was a 27-year-old male with a height of 1.67 cm and a weight of 65 kg. The experimental environment to test the infinite run game and park day games was a laptop with a camera positioned on a table 0.75 m high and with the subject one meter from the laptop screen. In the fishing game, the laptop was positioned on the floor, with one meter between the screen and the subject. The laptop used for developing the serious games was a Samsung with a seventh-generation Intel Core i7 processor, 2 GB NVIDIA GeForce 940 MX graphics card, 8 GB RAM, 1 TB hard drive, and a Microsoft webcam (original factory setting).

Figure 11a illustrates the graphs obtained from OpenPose for the infinite run game, and Figure 11b shows the data obtained with the Miotec goniometer.

Table 1. Data acquired from the infinite run game (°).

Peak OpenPose	Valley OpenPose	Peak Sensor	Valley Sensor	Comparison Peak	Peak Error (%)	Comparison Valley	Valley Error (%)
25.1777	−21.8261	34.5663	20.7398	9.3886	27.1612	1.0863	−5.2378
27.1744	−22.8939	27.6530	20.7398	0.4786	1.7307	2.1541	−10.3865
22.0160	−23.0193	27.6530	20.7398	5.6370	20.3848	2.2795	−10.9911
23.1296	−25.8529	27.6530	20.7398	4.5234	16.3578	5.1131	−24.6537
25.1304	−22.3809	20.7398	20.7397	4.3906	21.1700	1.6412	−7.9131
24.0696	−25.1092	20.7398	20.7397	3.3298	16.0551	4.3695	−21.0683
23.8739	−23.0177	27.6531	13.8265	3.7792	13.6664	9.1912	−66.4751
25.2302	−20.6711	27.6530	27.6530	2.4228	8.7615	6.9819	−25.2484

depicts the data for the movements performed in the infinite run game. The data were divided according to peak and valley graphical curves.

According to Table 1, the measurements/movements from OpenPose showed repeatability (24.48° ± 1.57° peak and 23.10° ± 1.68° valley) when playing infinite run. The measurements were made by simulating a patient playing the game and performing the movements of left lateral flexion and right lateral flexion stimulated with the proposed theme. The OpenPose and goniometer measurement errors were 15.66% at the peak and 21.50% at the valley. The values given by OpenPose for the lateral flexion of the spine cannot be used directly for precision measurement but can be used to follow the evolution of the patient.

Table 2. Data acquired from the Fishing game (°).

Peak OpenPose	Valley OpenPose	Peak Sensor	Valley Sensor	Comparison Peak	Peak Error	Comparison Valley	Valley Error
7.8892	−4.3440	6.9133	−6.9132	0.9759	14.1162	−2.5692	37.1636
7.4072	−5.5048	6.9132	−6.9133	0.4940	7.1457	−1.4085	20.3737
6.5475	−7.3497	6.9132	−6.9133	0.3657	5.2898	0.4364	6.3124
6.3751	−6.0910	6.9132	−6.9133	0.5381	7.7836	−0.8223	11.8944
7.0015	−5.4362	6.9132	−6.9133	0.0883	1.2772	−1.4771	21.3660
7.8486	−6.0189	6.9132	−6.9133	0.9354	13.5306	−0.8944	12.9373
6.5428	−6.2912	6.9132	−6.9133	0.3704	5.3578	−0.6221	8.9985
6.7035	−5.1816	6.9132	−6.9133	0.2097	3.0333	−1.7317	25.0488
7.9554	−3.7400	6.9132	−6.9133	1.0422	15.0755	−3.1733	45.9013
7.2640	−5.0176	6.9132	−6.9133	0.3508	5.0743	−1.8957	27.4210
6.1430	−7.3818	6.9132	−6.9133	0.7702	11.1410	0.4685	6.7767
6.3507	−9.5735	6.9132	−6.9133	0.5625	8.1366	2.6602	38.4794
7.6371	−5.9728	6.9132	−6.9133	0.7239	10.4712	−0.9405	13.6042
9.7088	−3.9009	6.9132	−6.9133	2.7956	40.4385	−3.0124	43.5739
7.8057	−6.0170	6.9132	−6.9133	0.8925	12.9100	−0.8963	12.9648

presents the data on the movements performed in the fishing game.

According to Table 2, the OpenPose measurements showed repeatability (7.28° ± 0.92° peak and 5.85° ± 1.48° valley) when playing the fishing game. The OpenPose and goniometer measurement errors were 10.72% at the peak and 22.19% at the valley. The fishing game allowed reproduction of the proposed movements, to stimulate spinal rehabilitation exercises.

Table 3. Data acquired from the park day game (°).

Peak OpenPose	Valley OpenPose	Peak Sensor	Valley Sensor	Comparison Peak	Peak Error	Comparison Valley	Valley Error
17.3296	37.6831	13.8265	34.5663	3.5031	−25.3361	3.1168	−9.0169
17.5499	43.4869	13.8265	34.5663	3.7234	−26.9294	8.9206	−25.8072
16.2637	40.6741	13.8265	34.5663	2.4372	−17.6270	6.1078	−17.6698
14.9639	33.9152	13.8265	34.5663	1.1374	−8.2262	0.6511	1.8836
15.0036	33.8901	13.8265	34.5663	1.1771	−8.5134	0.6762	1.9562
14.8255	34.8205	13.8265	34.5663	0.9990	−7.2253	0.2542	−0.7354
19.0809	31.9137	13.8265	34.5663	5.2544	−38.0024	2.6526	7.6739
17.8143	35.6055	13.8265	34.5663	3.9878	−28.8417	1.0392	−3.0064
16.7086	35.7589	13.8265	34.5663	2.8821	−20.8448	1.1926	−3.4502
15.2712	36.3465	13.8265	34.5663	1.4447	−10.4488	1.7802	−5.1501
18.3985	34.9237	13.8265	34.5663	4.5720	−33.0669	0.3574	−1.0340
20.0040	34.0739	13.8265	34.5663	6.1775	−44.6787	0.4924	1.4245
18.2420	36.9440	13.8265	34.5663	4.4155	−31.9351	2.3777	−6.8787
15.9775	33.6312	13.8265	34.5663	2.1510	−15.5571	0.9351	2.7052
14.2202	34.9848	13.8265	34.5663	0.3937	−2.8474	0.4185	−1.2107

summarizes the data for the movements performed during the park day game.

According to the findings, the OpenPose measurements showed repeatability (16.78° ± 1.74° peak and 35.91° ± 2.93° valley) when playing the park day game. The measurements were made while simulating a patient playing the game, and with the movements of lumbar flexion and extension (upright position) stimulated by the proposed theme. The OpenPose and goniometer measurement errors were 21.34% at the peak and 5.97% at the valley (Table 3). The park day game allowed the reproduction of the desired movements for spine rehabilitation, and the collected data can be used to monitor the patient’s evolution.

Figure 12 shows the respective graphs for the asteroids (Figure 12a) and infinite run (Figure 12b) games with the scores achieved. With the newly developed graphical interface for scoring, physiotherapy professionals would have quantitative tools to better evaluate human spine rehabilitation procedures.

Comparing the results of the serious games developed in this paper using OpenPose and Unity 3D, we obtained values equivalent to those obtained using Kinect [19]. The inertial sensors of the Valedo system have a better resolution [15,16,17]. No accuracy analyses for OpenPose were found in the literature applied to the assessment of human spine movements.

Besides the impossibility of using some physical therapy exercises in the games, other difficulties were noted during their development. For instance, when working with animations, the spawn method initially implemented was unfeasible, as its computational cost was very high, even causing the computer to crash and closing the software, due to a lack of response. This problem was solved using script optimization; and with less demanding cases where significant optimization was not necessary, the animation spawned and it was destroyed after it had finished. An example of optimization is the “recycling” of platforms in the infinite run game.

Another factor was the scoring method initially implemented. In some cases, this could give the player the impression that he was facing a penalty if he could not complete what was needed to score points. This problem was resolved and everything that could give this impression was removed, such as the asteroids that moved to collide with the spaceship that the player controlled if they were not destroyed in time, which could generate a sudden movement from the patient to divert the spaceship. We solved this problem by causing the asteroid to appear slowly on the screen, and even when it was not destroyed, it stopped in a position far from the spaceship, waiting for the new position of the spaceship for its destruction.

Another example is the fact that, initially, there were obstacles on the bridge of the infinite run game; even if the patient did not lose points when colliding with these obstacles, he could make sudden movements to avoid them. We solved this problem by removing the obstacles, and instead of the user receiving points for dodging them, he received points for collecting coins that appeared on the screen, as the robot avatar controlled by the user traveled across the bridge.

Motion exercise and rehabilitation during the COVID-19 pandemic are different from the containment measures during medical treatment. Normally, post-care patients who return home are required to perform physical exercises on their own, which is crucial to improve physical fitness [35]. Thus, there is a need to help patients with motion exercises and physical rehabilitation, by providing them with suitable and inexpensive equipment at home, such as personal computers with a webcam. This paper focused on back pain rehabilitation using OpenPose and Unity 3D software.

6. Conclusions

In this study, serious games were developed, to assist in the rehabilitation of the human spine. Importantly, these games are designed to improve the patients’ rehabilitation processes, especially in terms of engagement with and the continuity of treatment. The planning and choice of exercises were characterized as the main activities because, despite being an instrument of playful creation, the final objective of games was to assist in the rehabilitation treatment of patients.

Pandemics such as the COVID-19 bring new challenges to humanity, and rehabilitation procedures had to be discontinued due to the need for isolation. The COVID-19 pandemic increased the proportion of musculoskeletal complaints, such as back pain. The great difficulty of home treatment, due to the high cost of conventional equipment and the need for specialized labor to operate it, opened up new paths for developing solutions for home rehabilitation. This paper presented the development of serious games to assist health professionals in the physiotherapy of patients with spinal pain, during clinical and home applications, and to solve the isolation issues caused by the pandemic.

To create software for the proposed serious games, Unity 3D and OpenPose were used. Despite the measurement errors of the points related to the spine tracked by OpenPose and used in the serious games, the movements performed were repeatable. The values obtained using OpenPose and the score achieved could be used to monitor the patient’s evolution in spinal rehabilitation treatments.

Ultimately, the proposed games allowed the reproduction of the desired movements for the rehabilitation of the human spine, and the collected data could be used to follow the patient’s progression. However, further studies, including clinical trials with patients suffering from back pain are required, to validate these newly developed serious games.