Wireless Motion Capture System for Upper Limb Rehabilitation

Abstract

This work is devoted to the presentation of a Wireless Sensor System implementation for upper limb rehabilitation to function as a complementary system for a patient’s progress supervision during rehabilitation exercises. A cost effective motion capture sensor node composed by a 9 Degrees-of-Freedom (DoF) Inertial Measurement Unit (IMU) is mounted on the patient’s upper limb segments and sends wirelessly the corresponding measured signals to a base station. The sensor orientation and the upper limb individual segments movement in 3-Dimensional (3D) space are derived by processing the sensors’ raw data. For the latter purpose, a biomechanical model which resembles that of a kinematic model of a robotic arm based on the Denavit-Hartenberg (DH) configuration is used to approximate in real time the upper limb movements. The joint angles of the upper limb model are estimated from the extracted sensor node’s orientation angles. The experimental results of a human performing common rehabilitation exercises using the proposed motion capture sensor node are compared with the ones using an off-the-shelf sensor. This comparison results to very low error rates with the root mean square error (RMSE) being about $0.02$ m.

1. Introduction

In the rehabilitation process, the patient is expected to perform sets of physical exercises and activities under the supervision of the corresponding medical staff, in order to achieve a functioning level of body segments, which may have been impaired by an accident or a surgery, or as a consequence of pathological conditions, such as in case of a stroke. The objective in these circumstances is to decrease the recovery time for the patient by rehabilitating his physiological motor capabilities. Research studies and clinical results have shown that intensive rehabilitation can lead to optimal outcomes in terms of the patient’s motor capabilities and of the corresponding recovery time as well [1,2]. Within this challenging field in the broader context of medical science, human motion science and selected fields of biomedical engineering have contributed to the advancement of movement rehabilitation techniques by taking into account quantitative measurements and parametric models of human movements [3,4].

1.1. Types of Upper Limb Rehabilitation Systems

In the field of upper limb rehabilitation, a wide range of systems along with a variety of technologies and methodologies are presented in the literature. Human Motion Tracking systems, have been introduced in the rehabilitation supervision procedure [4]. However, the accurate localization that such systems perform are in direct conflict with their high cost. Furthermore, occlusion problems and the issues regarding the line-of-sight (LoS) during experimental process are drawbacks that reduce the effectiveness of these systems. On the other side, robot-based systems, where the patient installs his body segments, for rehabilitation purposes are extensively used in cases of severe disabilities [5,6,7]. Such systems are able to move, guide, even apply resistance to the limb motion, aiming at muscles strengthening and functionality improvement of the affected body segment.

Taking into account the issues inherent to visual-based systems, such as LoS, occlusion cases [4] and the problems related to the expensive robot assisting devices, which are cumbersome and require specialized medical staff, the alternative of lower cost, small-size sensor nodes to be used in motion tracking [3,8] was risen. In sensor-based integrations, such nodes are worn by the patient and gather, in an unobtrusive way, motion, position and physiological state data, without interfering with the patient’s normal behavior. This low cost technology is motivated by the several benefits regarding self organization, flexibility and the ability to provide long-term monitoring. The technical challenges of WSN within this application field of rehabilitation are presented in [9].

1.2. WSN-based Upper Limb Rehabilitation Systems

WSN-based Rehabilitation Systems for the Upper Limb have been introduced in the literature since 2010, indicating that WSN rehabilitation applications is a recent field of research. The sensor nodes that are attached to human body segments, forming a Body Area Network, consist of specific sensor units [10]. Individual accelerometers, gyroscopes and magnetometers or directly a unified Inertial Measurement Unit (IMU) can be integrated into these nodes. These systems rely on both custom-made sensor nodes [11,12,13] and off-the-shelf items [14,15].

These sensors nodes collect data, such as accelerations or angular velocities, in order to extract information about the position, the motion and/or the direction of body segments. Apart from the IMU, the optical linear encoder (OLE) [16] is another type of sensor that is met in the literature regarding WSN-based rehabilitation applications. It can provide direct information about a joint angle (only 1 DoF), but it is necessary to be combined with an accelerometer at least, to derive further information for the rest of the rotations, which are performed by this joint [17,18].

Another fundamental point in an upper limb rehabilitation application is the networking protocol that is going to be implemented. Within the literature, Wi-Fi [19], Bluetooth [14], IEEE 802.15.4 [11], ZigBee [20] and MiWi [13] are proposed, affecting in a varying degree the energy consumption of the total system, as well as the reliability of the sensors’ data transmission. Depending on the particular application’s objectives, either Time Division Multiple Access (TDMA) scheduling algorithms for node synchronization are used [15,20] or Carrier Sense Multiple Access/Collision Detection (CSMA/CA) for collision avoidance [21]. As far as the packet loss issue is concerned, due to the high sampling rate of the sensor, a lost data packet does not significantly reduce the accuracy of the collected data, since the missing information can be restored from neighbor sensors’ data using sensor fusion methods.

The interdisciplinary research field of rehabilitation procedure supervision using WSNs appears to be emerging and promising for the future. The achievements and complete rehabilitation solutions developed by relevant companies, such as Libelium [22], Deltason [23], Xsens [24], The IoT Marketplace [25] and Shimmer [26] indicate a rising interest in this field.

1.3. Upper Limb Motion Reconstruction

In reliable WSN-based rehabilitation systems, multiple accelerometers, gyroscopes and magnetometers collect raw data related to linear accelerations, angular rates and magnetic field data of the patient body movements. This data and the corresponding timestamps are transmitted to a base station for accurate estimation of the body segments movements in a global coordinate reference system [27]. More accurate estimations of the segments’ orientations can be obtained by taking into account restrictions related to the feasible orientation directions and ranges of the joint angles. At the next step, the reconstruction of the posture of the subject takes place, adopting the Denavit-Hartenberg (DH) configuration [17] or a Quaternion-based [14] approach.

In the final processing stage, using kinematic analysis, the limb posture and orientation is visualized in a 3-Dimensional (3D) digital representation. Using this representation, the extraction of several distinct characteristics, such as the movement amplitude, segments’ range of motion, movement velocity, is accomplished. This could give useful information during each exercise within a rehabilitation session. Furthermore, in some cases, the extraction of motion patterns is achieved via Principal Component Analysis (PCA) [28] or Machine Learning algorithms and techniques [19].

1.4. Motivation & Objectives

Several successful examples exist in the literature regarding the estimation of the upper limb movements using a wide variety of techniques along with IMU sensors. An inertial tracking system can be used in rehabilitation applications to derive the upper limb movement and its trajectory in real time. Furthermore, such a system can provide additional information, such as the movement amplitude, the range of motion, the angles between upper limb segments, or even the movement velocity. This information allows the medical stuff and the patient to evaluate the upper limb exercises, either at a medical center or at home. In the latter case, the patient can be monitored by the therapists without the need to visit a medical facility.

In this work a sensor system for monitoring the upper limb rehabilitation procedure is presented. The core of this system is a 9 Degrees of Freedom (DoF) IMU. Thus, the sensor node is aimed to be attached on various human body segments for motion capturing purposes. Regarding the specific case of upper limb, such sensors can be mounted on upper arm, forearm and hand, gather measurements and transmit the data to a base station. Further processing and filtering of these raw data result to each segment individual orientation, while the estimation of the upper limb joints and segments position and orientation in the 3D space, is accomplished by a reconstruction procedure based on the upper limb model.

The overall processing procedure includes a series of well defined mathematical expressions for the calculation of the upper limb’s joint angles in closed form. This is performed by exploiting the extracted orientation of the individual segments. Then, these joint angles represent the DoFs angles of a biomechanical model of the upper limb structure based on the Denavit-Hartenberg configuration. The efficiency of this methodology is evaluated by experimental studies on some common rehabilitation exercises, which are applied on a human’s upper limb. Furthermore, the comparison between the cost effective motion capture sensor node, that is proposed in this work, and an off-the-shelf sensor, emphasizes the effectiveness of the proposed system along with the total processing methodology.

The present work is an extension of our previous paper [29], where a custom-made sensor node was used for motion acquisition and 3D reconstruction of the upper limb segments movement. Here, the sensor fusion algorithm has been improved, in order to account for the relationship between the Euler angles’ time rates of change and the angular velocity resolved in the body fixed frame, as is presented in [30]. Moreover, we turn the previous device into a low-cost wearable device that did not exceed the cost of 20 dollars. Rehabilitation exercises were performed wearing the device to ensure that it is comfortable for the patient. Finally, motion capturing results using this sensor node and a much more expensive off-the-shelf device are compared to verify the validity of the custom-made device.

The upper limb model representation based on the Denavit-Hartenberg configuration, is adopted to achieve the 3D motion reconstruction of the upper limb motion, and, is presented in Section 2. In Section 3, the followed methodology regarding the gathering, processing and filtering of sensor nodes’ inertial measurements is described. This results to the calculation of the links/segments orientation, where the sensor nodes are attached to. The estimation of the joints’ angles for the upper limb model using this orientations, is also presented for each segment of the upper limb. In Section 4, the experimental results of a human performing a set of typical rehabilitation exercises are shown. Based on the motion capture sensor node raw data, the proposed methodology is utilized to provide the upper limb segments trajectories. The estimated 3D trajectories are compared with the trajectories that are concurrently derived by Shimmer3 motion capture sensor nodes. Finally, a discussion on the proposed methodology and over the experimental results is given in Section 5, followed by the conclusions.

2. Upper Limb Modeling

2.1. Upper Limb Kinematic Model

Anatomically, the upper limb consists of the pectoral girdle, the upper arm, the forearm and the hand [31]. The three joints of the upper limb are: the glenohumeral joint, the elbow, and the wrist joint. Hence, the upper limb could be modeled as a kinematic chain consisting of these three joints along with the corresponding DoFs, which account for the feasible rotations for each joint.

Related configuration of upper limb for motion tracking, based on the DH parameters can be found in the literature. In [32], the authors studied the movement of the shoulder and the elbow, and they designed a 4 DoF kinematic chain to express only the shoulder glenohumeral rotations and the elbow flexion. In [17], the human upper limb is also represented by rigid links connected with joints, where the upper limb kinematics are described with a 7 DoF model. This model has the shoulder as the origin of the kinematic chain and represents the shoulder joint mobility by 3 DoFs, the elbow joint by only 1 DoF, and the wrist joint by 3 DoFs. The latter correspond to wrist flexion-extension and deviation and the third one, is the forearm pronation-supination motion. In our work, the forearm pronation-supination has been defined as a wrist joint rotational DoF.

In [33], the authors assume a 7 DoF model of the left upper limb. The motions of the scapula and clavicle are also modeled by means of the humerus head ability to elevate and retract. This concludes to two extra DoFs that represent the clavicle elevation-depression and profusion-retraction, respectively. In this model, 3 rotational DoFs account for the shoulder abduction-adduction, internal rotation, and flexion-extension, followed by the 2 elbow DoFs performing the elbow flexion-extension and the forearm pronation-supination. The wrist joint DoFs is omitted in this model. Hense, this work is focused on the clavicle range of motion and on the shoulder joint DoFs.

In [34], the authors proposed a more complex kinematic model, based on the DH parameters, as well. This refers to the total upper body of a rower along with the seat rail for the case of an indoor rowing performance assessment. Each upper limb is represented as a 7 DoF kinematic chain with 3 revolute joints for the shoulder abduction-adduction, rotation and flexion-extension, 2 DoFs for the elbow representing flexion-extension and rotation and, 2 DoFs for the wrist abduction-adduction and flexion-extension. Moreover, this model accounts for the clavicle DoF, representing it as a revolute one.

In the present paper the forward kinematics were extracted for the human upper limb, which was designed as a kinematic chain consisting of three joints, namely, the shoulder, the elbow and the wrist and 7 DoFs. The shoulder/glenohumeral joint formed by the humeral head of the scapula and the glenoid cavity [35], performs three rotational DoFs: the flexion-extension one, abduction-adduction, and at last the internal-external rotation. The elbow can be represented as a hinge joint permitting elbow flexion-extension. The bones, muscles and ligaments structure in the elbow joint close area restricts its mobility. Nevertheless, a second DoF that originates a bit distal from the elbow is responsible for the pronation-supination of the forearm. Regarding the wrist joint, two DoFs are present, performing flexion-extension and deviation, respectively. The described kinematic chain used to model the human upper limb, is illustrated in Figure 1.

The coordinate frames were defined for each DoF, with respect to a set of rules from robotics theory and the DH configuration convention. These are presented in Figure 1 and summarized in

Table 1. Frames of the upper limb model.

Location	Description
Chest {C}	Chest Position—Origin Frame
Shoulder {S1}	Flexion and Extension
Shoulder {S2}	Ab- and Adduction
Shoulder {S3}	In- and External Rotation
Elbow {E1}	Flexion and Extension
Elbow {E2}	Pronation and Supination
Wrist {W1}	Flexion and Extension
Wrist {W2}	Deviation
Hand {End-effector}	End-effector

. The origin point of the kinematic chain is located at the human’s chest. The corresponding coordinate frame is noted as {C} and is also regarded as the origin one. The shoulder joint’s 3 DoFs follow with the frames {S1}, {S2} and {S3} assigned on them denoting the flexion-extension, abduction-adduction and internal-external rotation, accordingly. The elbow rotational DoFs are depicted as {E1} and {E2} frames accounting for the flexion-extension and pronation-supination DoFs, while {W1} and {W2} frames stand for the wrist joint DoFs, namely flexion-extension and deviation. The end-effector position and orientation correspond to the ones of the tip of the hand.

2.2. Range of Motion for the Upper Limb Model

In [36], the shoulder’s joint motion is explained in detail, while defining the corresponding ranges of motion. From an anatomic point of view the shoulder girdle consists of the clavicle, the scapula and the humerus. The shoulder joint motion is determined by a combination of these body parts distinct motions, making the analysis more complex. A ball-and-socket joint is adopted to model the shoulder girdle motion ability. Hence, the shoulder is described by 3 DoFs, permitting shoulder abduction-adduction motion, flexion-extension and internal-external rotation.

The elevation plain of upper limb defines the kind of exercise that the subject executes. The elevation plane of $0^{\circ }$ denotes the upper limb elevation over the frontal plane. In this case, the abduction-adduction of the upper limb occurs with the corresponding range of motion being $[0^{\circ },{180}^{\circ }]$. The elevation plane of ${90}^{\circ }$ indicates that the shoulder flexion-extension takes place. The corresponding range of motion is $[0^{\circ },{180}^{\circ }]$ during elevation through flexion and $[0^{\circ },-{60}^{\circ }]$ for the shoulder’s elevation through extension (hyperextension). In both cases, the rotation of $0^{\circ }$ is defined for an upper limb posture with its shaft parallel to the thorax vertical axis. The upper arm internal-external rotation is performed when the shoulder’s elevation angle is $0^{\circ }$ and the elbow is flexed at ${90}^{\circ }$, leading the forearm to be lying in the sagittal plane. The range of motion for this shoulder’s DoF that accomplishes the upper arm internal-external rotation is $[-{90}^{\circ },{20}^{\circ }]$, with the negative value being when the upper arm rotates internally towards the human body.

The elbow joint main rotation is performed around a fixed axis that is located among the center point of the trochlear sulcus and the center point of the capitulum of the humerus. This DoF permits the elbow flexion-extension with a range of $0^{\circ }$ for the case of the full extension for the elbow to ${130}^{\circ }$ depicting its flexion. Furthermore, a second rotation occurs a bit distal from the elbow joint, which is responsible for the forearm pronation-supination motion. In this case, the forearm rotates around an axis that is defined among the center points of the radial head and the distal ulna. The range of this motion is ${90}^{\circ }$ for the pronation case to −${90}^{\circ }$ for the supination one. The forearm neutral position, where its rotation is $0^{\circ }$, occurs when having the shoulder and also the wrist at their neutral positions, the hand is lying in the sagittal plane.

At last, the motion of the wrist joint around the axes that are defined in [37], allow for the wrist flexion-extension and wrist deviation. The first ranges from $-{70}^{\circ }$ for the case of extension to ${70}^{\circ }$ for the wrist flexion. The wrist deviation is defined as radial and ulnar. In the radial, the wrist deviates internally towards the body and till $-{10}^{\circ }$, while during the ulnar deviation the wrist deviates outwards of the body and to an upper limit of ${25}^{\circ }$. The neutral position of $0^{\circ }$ for both flexion and deviation DoFs of the wrist occurs when the third metacarpal and the longitudinal axis of the forearm are aligned.

Taking into account the previous assumptions, the ranges of motion for the upper limb degrees of freedom of our model are summarized in

Table 2. Upper limb DoFs ranges of motion.

Motion	DoF Range
Shoulder extension-flexion	$-{60}^{\circ }$–180${}^{\circ }$
Shoulder adduction-abduction	0${}^{\circ }$–180${}^{\circ }$
Shoulder internal-external rotation	$-{90}^{\circ }$–20${}^{\circ }$
Elbow extension-flexion	0${}^{\circ }$–130${}^{\circ }$
Elbow supination-pronation	$-{90}^{\circ }$–90${}^{\circ }$
Wrist extension-flexion	$-{70}^{\circ }$–70${}^{\circ }$
Wrist radial-ulnar deviation	$-{10}^{\circ }$–25${}^{\circ }$

.

2.3. Dh Parameters of Upper Limb Model

To describe the forward kinematics of the upper limb model, the DH parameters have to be derived. A quadruple of such parameters represent the relation among two sequential coordinate frames of the kinematic chain model. In

Table 3. DH parameters for the 7 DoF robotic arm.

Frames	Links	ai	${\mathit{\alpha }}_{\mathit{i}}$	${\mathit{d}}_{\mathit{i}}$	${\mathit{\theta }}_{\mathit{i}}$
{C}	0	0	0	$l_c$	$-90$°
{S1}	1	0	90°	0	${\theta }_1+90$°
{S2}	2	0	90°	0	${\theta }_2+90$°
{S3}	3	0	90°	$-l_{ua}$	${\theta }_3+90$°
{E1}	4	0	90°	0	${\theta }_4+180$°
{E2}	5	0	$-90$°	$-l_{fa}$	${\theta }_5$
{W1}	6	0	90°	0	${\theta }_6-90$°
{W2}	7	$-l_h$	90°	0	${\theta }_7$

, the DH parameters are presented, describing the model configuration that is depicted in Figure 1. The variables $l_c$, $l_{ua}$, $l_{fa}$ and $l_h$ stand for the length from the chest to the shoulder joint, the upper arm length and the forearm and hand lengths, respectively. The estimation of these lengths is based on biomechanics literature [35] and correspond to appropriate fractions of the human height H. Hence, if these parameters cannot be measured, the estimation of the upper limb segments’ lengths are given by: $l_c=0.129\ast H$, $l_{ua}=0.186\ast H$, $l_{fa}=0.146\ast H$ and $l_h=0.108\ast H$.

The $A_{i-1}^i$ of Equation (1) corresponds to the transformation matrix between the two sequential coordinate frames $i-1$ and i:

$$A_{i-1}^i=\left[\begin{array}{cccc}c{\theta }_i& -s{\theta }_{i}c{\alpha }_i& s{\theta }_{i}s{\alpha }_i& a_{i}c{\theta }_i\\ s{\theta }_i& c{\theta }_{i}c{\alpha }_i& -c{\theta }_{i}s{\alpha }_i& a_{i}c{\theta }_i\\ 0& s{\alpha }_i& c{\alpha }_i& d_i\\ 0& 0& 0& 1\end{array}\right]$$

(1)

where s and c represent sine and cosine of the corresponding angle and the four quantities ${\theta }_i$, ${\alpha }_i$, $d_i$, and $a_i$ are parameters associated with link i and joint i. These four parameters of Equation (1) are generally given the names joint angle, link twist, link offset and link length, respectively, and represent specific aspects of the geometric relationship between two coordinate frames [38]. Since the matrix $A_{i-1}^i$ is a function of a single variable, as shown in Table 3, it concludes that the three out of the four parameters for a link have constant values and the only variable is ${\theta }_i$ or $d_i$ in case of a revolute or a prismatic joint, accordingly.

The transformation matrix from the chest position, where the origin frame is assumed, to the end-effector $T_0^7$ is given by Equation (2).

$$T_0^7={\prod }_{i=0}^7{A}_{i-1}^i$$

(2)

3. DoFs Angles Estimation

Having modeled the upper limb as described in the previous Section, we present a methodology to estimate the angles’ values of the defined DoFs for the upper limb model. This methodology starts using the inertial sensors measurements. If a 9-DoF inertial sensor is attached on a link of upper limb, the gathered measurements during this link’s motion could give the link orientation in the 3D space. The calculation approach, which is described in the sequel, assumes that the coordinate frames of the sensor node resemble these of the MPU-9150 [39]. In case of a different sensor or coordinate frames configuration, the mathematical equations should be modified accordingly. Furthermore, an important step before the extraction of the orientation is the filtering process, as these are calculated by the raw sensors data, which suffer from gyroscopes drifting and accelerometers noise.

Nevertheless, the calculation of each individual segment orientation is insufficient for the estimation of the human upper limb posture and orientation in the 3D space. Hence, in the proposed methodology, the orientation of each upper limb link is initially estimated individually based on the corresponding sensor node. Then, as each link is a part of the described in the previous Section upper limb model, we manage to get the related DoFs angles in a closed form. The intermediate calculation steps for each upper limb segment and the corresponding joints angles (3 DoFs for shoulder, 2 for elbow and 2 for wrist) are presented in Section 3.2, Section 3.3 and Section 3.4. At the last step, the calculated joint angles ${\theta }_i$ are used to solve the Forward Kinematics of the upper limb model, following the procedure that was presented in Section 2. Thus, the reconstruction and representation of the upper limb motion in the 3D space is finally accomplished.

3.1. Sensor Fusion and Orientation Estimation

The sensor node orientation can be estimated either by integrating the angular velocities or by processing the accelerometer measurements and estimating the gravity vector. However, to avoid the accumulative errors due to the integration of the gyroscope measurements and to minimize the noise in the orientation vector derived from the accelerometer signal, both estimators are implemented and fused over a complementary filter [40,41]. Although, there are improved versions of the complementary filter [42,43,44], we used this simpler version, which concludes to satisfactory results regarding the experimental test cases, while it also provides quick and accurate performance [45]. In the sequel, the estimation and filtering procedure that concludes to the calculation of links orientation is presented in detail.

The simplest method to extract the orientation angles of an IMU-based sensor node is by integrating the measurements of its gyroscope adopting the first-order approximation of Taylor series:

$$\begin{array}{c}\hfill \varphi \left[n\right]=\varphi [n-1]+\frac{\delta {\varphi }_g\left[n\right]}{\delta t}T\\ \hfill \theta \left[n\right]=\theta [n-1]+\frac{\delta {\theta }_g\left[n\right]}{\delta t}T\\ \hfill \psi \left[n\right]=\psi [n-1]+\frac{\delta {\psi }_g\left[n\right]}{\delta t}T\end{array}$$

(3)

where $\varphi$, $\theta$ and $\psi$ are the roll, pitch and yaw rotations around y—the longitudinal, x—the transverse and z—the vertical axes of the sensor, and T is the sampling period. In our implementation the value of T is constant, but filtering with variable sampling periods can also be adopted.

The rates of Euler angles that are presented as coefficients in Equation (3), are given by the first-order differential equations [30] as:

$$\left[\begin{array}{c}\frac{\delta {\varphi }_g}{\delta t}\\ \frac{\delta {\theta }_g}{\delta t}\\ \frac{\delta {\psi }_g}{\delta t}\end{array}\right]=\left[\begin{array}{ccc}\frac{\mathrm{s}{\varphi }_a\mathrm{s}{\theta }_a}{\mathrm{c}{\theta }_a}& 1& \frac{\mathrm{c}{\varphi }_a\mathrm{s}{\theta }_a}{\mathrm{c}{\theta }_a}\\ \mathrm{c}{\varphi }_a& 0& -\mathrm{s}{\varphi }_a\\ \frac{\mathrm{s}{\varphi }_a}{\mathrm{c}{\theta }_a}& 0& \frac{\mathrm{c}{\varphi }_a}{\mathrm{c}{\theta }_a}\end{array}\right]\left[\begin{array}{c}{G}_x\\ G_y\\ G_z\end{array}\right]$$

(4)

where $G_x$, $G_y$, $G_z$ are the measurements of the gyroscope for x, y and z axis, respectively and s and c represent sine and cosine of the corresponding angle.

The transformation from the raw gyroscope measurements with respect to the sensor body axis coordinate system to the fixed coordinate system is derived from the direction cosine matrix shown in Equation (4). This relation allowed to update the orientation of the upper limb link with time, improving also its accuracy, compared to the previous implementation [29], where we used the raw gyroscope measurements in the calculations. Using the raw measurements, we get the angular velocities with respect to the moving body frame and not the fixed 3-dimensional global frame that is defined. This is the reason which restricted us from testing successively the full range of the motions, which were presented in our previous work, and instead we presented the half range of motions, compared to the results that we will present in the sequel, in Section 4.

The parameters ${\varphi }_a$, ${\theta }_a$ and ${\psi }_a$ are the orientation Euler angles that are estimated from the accelerometer measurements. Due to the discrete integration (Equation (3)), the errors and the sensor noise result to the angular error, known as drift.

These orientation angles can be retrieved by identifying the direction of the gravity vector for a $R_y\left(\varphi \right)R_x\left(\theta \right)R_z\left(\psi \right)$ configuration [46] as:

$$\begin{array}{cc}\hfill {\varphi }_a& =\mathrm{atan}2\left(\frac{A_x}{A_z}\right)\hfill \\ \hfill {\theta }_a& =\mathrm{atan}2\left(\frac{A_y}{\sqrt{A_x^2+A_z^2}}\right)\hfill \end{array}$$

(5)

where $A_x$, $A_y$, $A_z$ correspond to accelerometer measurements over x, y and z axis. The $atan2\left(Y,X\right)$ denotes the operation $arctan\left(\frac{Y}{X}\right)$ of inverse tangent, which accept solutions in the angle range of $\left[-\frac{\pi }{2},\frac{\pi }{2}\right]$. The estimation of the rotation ${\psi }_a$ in z axis is derived from the compass measurements along with the already estimated orientation angles ${\varphi }_a$ and ${\theta }_a$, as shown in the following equation. This is performed in order to improve the accuracy of the rotation estimation in this axis:

$${\psi }_a=\mathrm{atan}2\left(C_y,C_x\right)$$

(6)

where

$$\begin{array}{cc}\hfill C_x& =M_{x}cos\left({\varphi }_a\right)+M_{y}sin\left({\varphi }_a\right)sin\left({\theta }_a\right)+M_{z}sin\left({\varphi }_a\right)cos\left({\theta }_a\right)\hfill \\ \hfill C_y& =M_{y}cos\left({\theta }_a\right)-M_{z}sin\left({\varphi }_a\right)\hfill \end{array}$$

(7)

and the parameters $M_x$, $M_y$, and $M_z$ represent the compass measurements along the corresponding axes.

Subsequently, sensors’ orientation can be estimated based on the gyroscope measurements on a short-term horizon, due to the resulting drifting, and by relying on the accelerometer and magnetometer measurements for a long-term orientation estimation.

A complementary filter is suggested, in order to provide more accurate estimations of the orientation angles for the sensor node. This filter accounts for drifting compensation and noise reduction and is formed as:

$$\begin{array}{cc}\hfill \varphi \left[n\right]& =\left[\varphi [n-1]+\frac{\delta {\varphi }_g\left[n\right]}{\delta t}T\right]k+(1-k){\varphi }_a\left[n\right]\hfill \\ \hfill \theta \left[n\right]& =\left[\theta [n-1]+\frac{\delta {\theta }_g\left[n\right]}{\delta t}T\right]k+(1-k){\theta }_a\left[n\right]\hfill \\ \hfill \psi \left[n\right]& =\left[\psi [n-1]+\frac{\delta {\psi }_g\left[n\right]}{\delta t}T\right]k+(1-k){\psi }_a\left[n\right]\hfill \end{array}$$

(8)

Thus, concluding to the next form taking into account the Equations (3), (5) and (6):

$$\begin{array}{cc}\hfill \varphi \left[n\right]& =\left[\varphi [n-1]+\frac{\delta {\varphi }_g\left[n\right]}{\delta t}T\right]k+(1-k)\mathrm{atan}2\left(\frac{A_x\left[n\right]}{A_z\left[n\right]}\right)\hfill \\ \hfill \theta \left[n\right]& =\left[\theta [n-1]+\frac{\delta {\theta }_g\left[n\right]}{\delta t}T\right]k+(1-k)\mathrm{atan}2\left(A_y\left[n\right],\sqrt{A_x{\left[n\right]}^2+A_z{\left[n\right]}^2}\right)\hfill \\ \hfill \psi \left[n\right]& =\left[\psi [n-1]+\frac{\delta {\psi }_g\left[n\right]}{\delta t}T\right]k+(1-k)\mathrm{atan}2\left(C_y\left[n\right],C_x\left[n\right]\right)\hfill \end{array}$$

(9)

where k∈$(0.5,1)$ is a variable weighting the effect of its term and its typical value is close to 1.

3.2. Estimation of the Shoulder Joint Angles

The collected motion data from the IMU, mounted on the upper limb segments, are filtered and processed, as described in Section 3.1. Hence, the orientation of all the individual links and for each timestamp is determined in terms of the extracted roll, pitch and yaw angles. Using these orientation angles, the direction cosine matrix is calculated for each upper limb segment, specifying also the required rotation sequence. This is a dynamic procedure and takes place for every timestamp that updated inertial measurements are received by the sensor nodes.

Having described before the shoulder joint as a spherical wrist configuration for a robotic arm model [38], the shoulder joint angles’ variables ${\theta }_1$, ${\theta }_2$ and ${\theta }_3$ can be assigned to the roll, pitch and yaw angles, with respect to the coordinate frame $O_0{x}_0{y}_0{z}_0$ (Figure 1). The mathematical expressions that relate the two sets of angles, result from the following analysis.

The shoulder joint rotations are described by a sequence of the three elemental rotations that occur around the axes of the origin coordinate system, which remains motionless, adopting the convention of the Tait-Bryan angles [47]. The sequence of elemental rotations is important in the estimation process. So, in case of a different definition in the sequence of the elemental rotations, the corresponding transformation matrix representation must be re-estimated.

The orientation transformation among the origin frame 0 and the frame 4 of the upper limb model (Figure 1) is given by the Equation (10). This equation results from the transformation matrix $T_0^4$ of the forward kinematics based on the DH parameterization. On the other side, the corresponding representation for the Tait-Bryan angles $Z_{\alpha }{X}_{\beta }{Y}_{\gamma }$ configuration has the form of Equation (11). X, Y and Z are the transformation matrices of the elemental rotations around the fixed frame axes x, y and z, accordingly. Hence, this equation represents an angle $\alpha$ rotation around z axis, followed by an angle $\beta$ rotation around x axis and, an angle $\gamma$ rotation around y axis of the fixed frame.

$$R_0^4=\left(\begin{array}{ccc}s{\theta }_{1}c{\theta }_3+c{\theta }_{1}s{\theta }_{2}s{\theta }_3& c{\theta }_{1}c{\theta }_2& s{\theta }_{1}s{\theta }_3-c{\theta }_{1}s{\theta }_{2}c{\theta }_3\\ -c{\theta }_{1}c{\theta }_3+s{\theta }_{1}s{\theta }_{2}s{\theta }_3& s{\theta }_{1}c{\theta }_2& -c{\theta }_{1}s{\theta }_3-s{\theta }_{1}s{\theta }_{2}c{\theta }_3\\ -c{\theta }_{2}s{\theta }_3& s{\theta }_2& c{\theta }_{2}c{\theta }_3\end{array}\right)$$

(10)

$$\mathcal{R}=Z_{\alpha }{X}_{\beta }{Y}_{\gamma }=\left(\begin{array}{ccc}c\alpha c\gamma -s\alpha s\beta s\gamma & -s\alpha c\beta & c\alpha s\gamma +s\alpha s\beta c\gamma \\ s\alpha c\gamma +c\alpha s\beta s\gamma & c\alpha c\beta & s\alpha s\gamma -c\alpha s\beta c\gamma \\ -c\beta s\gamma & s\beta & c\beta c\gamma \end{array}\right)$$

(11)

The parameter $\alpha$ of Equation (11) should not be confused with the Denavit-Hartenberg parameter $\alpha$ noted in Table 3 and Equation (1). Solving over the corresponding items of the matrices presented in Equations (10) and (11), we conclude to the results of Equation (12):

$$\begin{array}{cc}\hfill {\theta }_1& =\alpha +pi/2\hfill \\ \hfill {\theta }_2& =\beta \hfill \\ \hfill {\theta }_3& =\gamma \hfill \end{array}$$

(12)

By replacing the estimated values of angles ${\theta }_1$, ${\theta }_2$ and ${\theta }_3$ in the transformation matrix $T_0^4$, the posture of the upper arm is extracted with respect to the $O_C{x}_C{y}_C{z}_C$ coordinate frame, as it is defined in Figure 1.

3.3. Estimation of the Elbow Joint Angles

The position and orientation of the elbow joint, as it is the distal point of the upper arm, can be estimated from the upper arm position and orientation as shown in Section 3.2. Following a similar approach for the motion data, which are collected by the sensor node that is attached to the human’s forearm, the orientation of this segment is extracted in terms of roll, pitch and yaw angles. The kinematic model variables that relate to the elbow joint are the angles ${\theta }_4$ and ${\theta }_5$. The elbow joint revolute DoFs can be estimated from the orientation angles, taking into account the representation of Equation (11) for the Tait-Bryan angles, as:

$${\theta }_4=\mathrm{atan}2\left(-\mathcal{R}\left(3,1\right),\mathcal{R}\left(3,3\right)\right)$$

(13)

$${\theta }_5=\mathrm{atan}2\left(-\mathcal{R}\left(1,2\right),\mathcal{R}\left(2,2\right)\right)$$

(14)

By replacing these estimated angles’ values in the transformation matrix $T_4^6$, the posture of the forearm is extracted with respect to the $O_3{x}_3{y}_3{z}_3$ coordinate frame (Figure 1).

3.4. Estimation of the Wrist Joint Angles

The wrist joint rotations ${\theta }_6$ and ${\theta }_7$ of the kinematic model can be derived by a similar procedure as following:

$${\theta }_6=\mathrm{atan}2\left(\mathcal{R}\left(3,1\right),-\mathcal{R}\left(3,3\right)\right)$$

(15)

$${\theta }_7=\mathrm{atan}2\left(\mathcal{R}\left(1,2\right),-\mathcal{R}\left(2,2\right)\right)$$

(16)

Hence, by the transformation matrix $T_6^8$, the posture of the hand is extracted with respect to the $O_5{x}_5{y}_5{z}_5$ coordinate frame (Figure 1).

4. System Implementation and Experimental Results

4.1. Motion Sensors

The implemented device, shown in Figure 2, consists of inertial sensors that capture the node’s motion with respect to the inertial frame and a microcontroller with Wi-Fi capabilities. Specifically, the device consists of an IMU, a microcontroller and a removable and rechargeable small-size battery. The IMU that was used is a 9 DoF Micro-Electro-Mechanical System (MEMS) IMU. It is the single unit chip MPU-9150, produced by InvenSense Inc. The microcontroller that was used is a low-cost WiFi chip with full TCP/IP stack. It is the ESP8266 and it is produced by Espressif Systems [48]. The purpose of this device is to retrieve and transmit the sensor data through WiFi to a base station. At the base station, the data is processed, stored and visualized. At last, the battery, which was used, is a removable RCR123A Lithium, high current, rechargeable battery that allows for a long-life and easy replacement.

Through an Inter Integrated Circuit (I${}^2$C) protocol [49], the data received from the IMU sensors are used to estimate the orientation of the IMU chip. The ESP8266 micro-controller serves every device that requests data from the sensor node. The fact that HTTP protocol [50] applies large headers, while it also lacks full duplex communication, makes this protocol not suitable for the designed application. Moreover, the time-restrictions and the requirement for transferring a great amount of data from the sensor to the base station, in a request-response messaging protocol, imposed the use of a more appropriate protocol. Thus, a websockets [51] protocol was adopted. This protocol offers a persistent TCP/IP connection, while client and server can exchange packets, avoiding to burden the communication channel with a large volume of irrelevant data.

Each sensor node requires a calibration process before the first use. There are 3 sensors in the node and there is a different procedure for each sensor. For the magnetometer we should rotate the sensor multiple times at different directions. We store only the minimum and the maximum value on each axis. Having rotated for a while, we find the offset of each axis by taking the mean value of the maximum and minimum values for that axis. Then, we subtract this offset by the measurements. For the gyroscope, letting the device on a surface for a few seconds can produce the mean value of the sensor, which is supposed to be zero. Thus, we subtract this mean value from each measurement, in order to get the correct angular rotation of the device. The accelerometer calibration needs to lay the sensor with all its faces down. Hence, the gravity vector can be identified on each axis, and the values are saved on the microcontroller.

During the experimental sessions, a motion capturing sensor node is mounted on the outer side of the selected upper limb segment with the Y axis of the sensor being aligned with the longitudinal axis of the segment. The right placement of the sensor is ensured using a special designed fabric case, which is strapped in the circumference of the limb segment. Then, the sensor is switched on and the communication with the base station is established. The objectives are the acquisition, storing and concurrent visualization of the node’s IMU measurements during the subject’s exercise session.

Concurrently, a Shimmer3 sensor node [26] has been aligned with the custom-made sensor, proposed in this work, and both were attached to the human’s upper limb segments. A custom-made piece with two sockets was used to successfully align these two sensors. The measurements form Shimmer3 sensor node were captured directly into Matlab^® using the Shimmer Matlab Instrument Driver. This driver is provided by the corresponding company and establishes the communication with the device via the Bluetooth protocol. Before the motion capturing session, the Shimmer3 units were calibrated using the Shimmer 9DOF Calibration Application, that is also derived by the same company. This application implements an automated process that calculates the calibration parameters for Shimmer’s integrated accelerometers, gyroscopes and magnetometers sensors. These calibration parameters are finally stored in the unit memory, so as the sensor measurements can be automatically corrected before they are sent to the paired device (e.g., a computer that functions as base station).

In the sequel, the collected sensors measurements, for both type of sensors, are processed and filtered as described in the previous Section. Then, the DoFs rotations ${\theta }_i$, for the upper limb model, are extracted by Equation (12) for the shoulder, Equations (13) and (14) for the elbow and Equations (15) and (16) for the wrist joint, respectively. In the following Subsections, the 3D reconstructed upper limb joints trajectories are presented in common rehabilitation exercises, as the elbow’s flexion-extension, the shoulder’s abduction-adduction and the wrist’s flexion-extension exercise. In the processing stage, the sensor’s sampling period is selected as $T=20$ msec, while the values of the model lengths are defined as $l_c=0.2322$ m, $l_{ua}=0.3348$ m, $l_{fa}=0.2628$ m and $l_h=0.1944$ m, which were estimated for the subject’s height of $H=1.80$ m. Furthermore, the first two experimental cases do not account for any wrist rotation, since there was not any sensor node attached over the subject’s hand. Hence, the corresponding DoFs angles are assumed as ${\theta }_6={\theta }_7=0^{\circ }$.

4.2. Elbow Joint Flexion-Extension Exercise

During this exercise session, the subject performs flexion and extension of the elbow. This exercise is repeated a few times and the sensor measurements of the node, which is attached to the subject’s forearm, are recorded and processed. This results to the estimation of the node’s orientation angles $\varphi$, $\theta$ and $\psi$. Using the node’s orientation angles, we can estimate the elbow DoF angles. Comparison between the proposed custom-made sensor node and the Shimmer3 sensor unit for the estimated angles ${\theta }_4$ and ${\theta }_5$ are shown in Figure 3 and Figure 4, respectively.

The trajectories of the shoulder, the elbow, the wrist joints and the tip of the hand in x, y, and z axes are presented in Figure 5. The distances between the segments joints, as shown in this figure, are in accordance with the corresponding fractions of human’s height. The motion takes place in the $y-z$ plane and the value of x component for each joint is constant and equal to the $l_c$, due to the dependence of the segments’ position only by the angle ${\theta }_4$. In this case, the position of the forearm is calculated by the transformation matrix $T_4^6$, which depends only on the variable ${\theta }_4$. The angle ${\theta }_5$ affects only the orientation of the forearm and not its position. Furthermore, the hand is supposed to be aligned with the forearm, thus angles ${\theta }_6$ and ${\theta }_7$ of the wrist joint are zeroed.

The resulted joints’ trajectories, as extracted by the upper limb model presented in Section 2, for the calculated elbow DoF angles, are illustrated in Figure 6. The trajectories, as extracted for the case of the custom-made sensor node, are noted with the ‘o’ symbol and represent the exercise that the patient has performed. The upper limb segments’ trajectories for the case of the Shimmer3 sensor unit’s measurements are also presented in the same figure. Deviations between the trajectories, especially in the upper and lower limit of the motion range, are identified in this figure. In the case of Shimmer3, the trajectories are more limited (noted with deep purple and deep red color). This is resulted by the elbow DoF angle ${\theta }_4$ that ranges from ${13}^{\circ }$ to ${85}^{\circ }$ approximately, while the corresponding angle calculated by the measurements of the proposed sensor node ranges between $8^{\circ }$ to ${88}^{\circ }$, as shown in Figure 3.

The processing and filtering methodology, described in Section 3.1, were followed for the IMU measurements of both sensor nodes. The nodes were calibrated before the capturing session. It was noticed that the differences in the elbow angles’ ranges, which were observed along a series of experiments, were related to the calibration accuracy of the sensors. Moreover, it was noticed that the forearm configuration during the exercise execution does not account for the gimbal lock problem. Finally, the points that are visualized in the trajectories’ 3D plots correspond to the total number of the exercise’s repetitions and not to only one.

The mean absolute error (MAE), the maximum error (MaxError) and root mean square error (RMSE) for the wrist and the tip of the hand (end-effector) trajectories, as were calculated between the proposed sensor node and the Shimmer one, are summarized in

Table 4. Elbow joint’s MAE, maximum and RMSE errors.

Trajectory	MAE [m]	MaxError [m]	RMSE [m]
Wrist	0.0126	0.0509	0.0161
End-Effector	0.0182	0.0706	0.0229

.

Where

$$\begin{array}{cc}\hfill MAE& =\sum _{i=1}^n\frac{|Pse{n}_i-Pshi{m}_i|}{n}\hfill \\ \hfill MaxError& =max\left|Pse{n}_i-Pshi{m}_i\right|,i=\left[1,n\right]\hfill \\ \hfill RMSE& =\sqrt{\sum _{i=1}^n\frac{{\left(Pse{n}_i-Pshi{m}_i\right)}^2}{n}}\hfill \end{array}$$

(17)

and the 3D trajectory points $Pse{n}_i$ for the custom-made sensor node and $Pshi{m}_i$ for the Shimmer one are given as $P_i=\sqrt{x_i^2+y_i^2+z_i^2}$, while n denotes the total number of points of the trajectory. The estimated error metrics, which represent the difference among the trajectories extracted by these two sensor nodes, show a quite high level of accuracy.

4.3. Shoulder Joint Abduction-Adduction Exercise

Before the exercise session, we attach the wearable device to the subject’s upper arm. During this exercise session, the subject performs abduction and adduction of the shoulder joint for his right upper limb. The sensor’s on-board processor estimates the node’s orientation angles $\varphi$, $\theta$ and $\psi$. Then, it calculates the shoulder rotation angles ${\theta }_1$, ${\theta }_2$, ${\theta }_3$ based on the procedure described above. The results are presented in Figure 7, Figure 8 and Figure 9, accordingly, while the corresponding elbow angles ${\theta }_4$ and ${\theta }_5$ are fixed, during this exercise. As it is shown in the aforementioned figures, the shoulder’s joint angles, as extracted from both sensor nodes, are almost identical. Therefore, an accurate reconstruction of the upper limb movement can be achieved either using a shimmer sensor node or the proposed custom-made one.

The trajectories of the shoulder, the elbow, the wrist and the tip of the hand in x, y, and z axes are presented in Figure 10. Deviations in the y axis occur, while the subject’s motion took place in the $x-z$ (sagittal) plane. This was caused because of the not ideal mounting of the node on the subject’s upper arm and, thus, a sensor-to-body frame transformation should be defined.

An issue that might occur during this motion is the gimbal lock problem. This results from the use of the Euler orientation angles $\varphi$, $\theta$ and $\psi$ to find, after processing, the position and orientation of the corresponding segment. Actually, the shoulder joint can perform three rotations. When the ${\theta }_2$ angle rotation comes up to the furthest point, the shoulder joint axes $z_0$ and $z_2$ of the first and third shoulder’s DoF, accordingly, are collinear. This is a singular configuration and only the sum ${\theta }_1+{\theta }_3$ or the difference ${\theta }_1-{\theta }_3$ can be determined. One solution is to choose the one angle arbitrarily and then determine the other using trigonometric equations. Nevertheless, the range of motion in the present exercise is not affected by the gimbal lock issue.

The 3D reconstructed trajectories of each upper limb joint are presented in Figure 11, noted with ’o’ symbol, for the case of the custom-made sensor node. A few consecutive repetitions of abduction-adduction exercise are performed, which explains the deviations of the visualized points. The extracted upper limb segments’ trajectories for the case of the Shimmer3 unit are also presented in the same figure. As in the comparison of the previous exercise (elbow flexion-extension), the trajectories of the upper limb segments, which are extracted from the Shimmer3 sensor’s measurements (noted with light blue, deep purple and deep red color), are more limited than the corresponding ones resulted from the custom-made motion capture sensor node. This results from the minor differences in the calculated shoulder DoFs’ ranges between the two sensors nodes. Once again, in this experiment, it was observed that the shoulder angles’ ranges were related to the accuracy of the calibration parameters for each sensor.

The mean absolute error (MAE), the maximum error (MaxError) and root mean square error (RMSE) for the elbow, wrist and tip of the hand (end-effector) trajectories, between the two sensor nodes, as extracted using Equations (17), are summarized in

Table 5. Shoulder joint’s MAE, maximum and RMSE errors.

Trajectory	MAE [m]	MaxError [m]	RMSE [m]
Elbow	0.0061	0.0462	0.0087
Wrist	0.0072	0.0547	0.0103
End-Effector	0.0076	0.0577	0.0109

.

4.4. Wrist Joint Flexion-Extension Exercise

In this case, the subject performs a few repetitions of wrist joint flexion-extension exercise of his right upper limb. The extracted sensor orientation angles $\varphi$, $\theta$ and $\psi$ contribute to the wrist DoFs angles estimation. During this exercise the subject’s elbow is steadily flexed at ${\theta }_4={90}^{\circ }$ being perpendicular to the coronal plane, and the hand is repeatably flexed with the palm towards the chest at almost ${\theta }_6=-{55}^{\circ }$ and extended until almost ${\theta }_6={25}^{\circ }$, as shown in Figure 12. In Figure 13, a deviation around a mean value is presented that corresponds to the second DoF of the wrist, this of radial-ulnar deviation. Actually, a small deviation is present during the subject’s wrist motion, thus the deviation in this graph is reasonable.

The concluded joints’ trajectories, as extracted by the upper limb model for the wrist DoF angles ${\theta }_6$ and ${\theta }_7$ is illustrated in Figure 14. It represents the exercise that the subject has performed, along with this small deviation. The corresponding trajectories of the shoulder, the elbow, the wrist and the tip of the hand in x, y, and z axes are presented in Figure 15.

In this case, a comparison among the custom-made and the Shimmer3 sensor node was not realized, because it was not possible to attach the array of the aligned sensors on the subject’s hand, due to the hardware dimensions.

Such graphs of upper limb trajectories and joint DoFs angles during exercise sessions are useful for a subject’s upper limb status evaluation by the physical therapists. Especially, the analysis over the range of motion for each joint angle or even the estimation of the speed during the execution of an exercise indicate the status and progress of a patient with upper limb movement disorders. Hence, the therapists could tune the sequel rehabilitation sessions accordingly.

5. Discussion

During the elaboration of this work and especially the experimental part, in order to validate the sensor fusion algorithms along with the upper limb modeling for the successful 3D motion reconstruction, several issues were encountered. Some of them need more attention and further investigation, aiming at the improvement of the upper limb motion tracking.

The experimental results showed that the proposed custom-made motion capturing system had an acceptable performance, as shown by the estimated error indicators in Table 4 and Table 5. Using optical motion capturing systems instead, like Vicon, the position error would be much smaller [52]. Nevertheless, in order to avoid the occlusion problem, which is the main drawback of this kind of systems, multiple cameras should be installed in the rehabilitation center. However, the overall cost may be prohibited for an average healthcare facility.

The choice of Euler angles representation found to be more understandable to handle and to identify failing results in the implementation that then could be solved. At the current status of research, the range of motions, which were performed during the experimental tests, is not affected by the gimbal lock problem. Thus, the adoption of Euler configuration was finally chosen. A quaternion-based option that is currently examined is the Geometrical Constraints Modeling and Fusion methodology presented in [53,54], where each IMU placement defines the corresponding segment’s reference frame. When the quaternion representation of the sensors’ measurements flow is provided, a fusion algorithm accounts for the conversion from quaternion sequences into the axis-angular sequences. In this case, an upper limb model is also needed, in order to express the biomechanical characteristics and contribute to the 3D motion reconstruction.

Regarding the current state of the presented custom-made sensor system, upper limb movements, such as horizontal abduction that has a range of 130${}^{\circ }$ towards the human chest or scapula protraction-retraction, has not been examined yet. In this case, actually, an extra DoF has to be included in the upper limb model. The complete model for both upper limbs can be defined, as well, as an extension to this work.

6. Conclusions

In this paper, an upper limb kinematic model that resembles a robotic arm with rigid links is proposed. It is formed as a 7 DoF kinematic chain, while the Denavit-Hartenberg configuration is adopted to describe this model. Furthermore, the Forward Kinematics matrices that can give the position and orientation of each upper limb segment are derived.

For this model, the shoulder, the elbow and the wrist joint angles are estimated by exploiting the upper arm, forearm and hand segments’ orientation vectors, respectively. For this reason, inertial sensors are attached to the upper limb segments, in order to capture theirs motion. This is finally accomplished by a two steps procedure. At first, a sensor fusion algorithm is applied to the inertial sensors’ data along with a complementary filter, in order to extract each individual upper limb’s segment orientation. Then, these orientations contribute to the estimation of the rotational DoFs that are defined in the aforementioned upper limb model.

This concludes to the 3D reconstruction of the upper limb model’s motion by applying the Forward Kinematics of the DH configuration. The 3D reconstruction results, the trajectory tracking and the visualization of a subject’s upper limb for both the prototype motion capture sensor node and the off-the-shelf Shimmer3 sensor unit, are performed, for comparison reasons. The accuracy of the results derived by the custom-made sensor node is also validated with an RMSE of $0.02$ m and a maximum error of $0.07$ m, when these are compared to the results of the similar commercial wireless motion capturing device.

These validation results demonstrate the accuracy of the inertial technology based motion analysis, which is presented in this work, and can be a guide for further improvement. The same methodology could also be extended to estimate the lower limb, or the whole body movements in the 3D space, using additional sensors and similar robotic modeling of the human body segments.