Wireless Epidermal Six-Axis Inertial Measurement Units for Real-Time Joint Angle Estimation

Abstract

Technological advances in wireless communications, miniaturized sensors, and low-power electronics have made it possible to implement integrated wireless body area networks (WBANs). These developments enable the applications of wireless wearable systems for diagnosis, health monitoring, rehabilitation, and dependency care. Across the current range of commercial wearable devices, the products are not firmly fixed to the human body. To minimize data error caused by movement of the human body and to achieve accurate measurements, it is essential to bring the wearable device close to the skin. This paper presents the implementation of a patch-type, six-axis inertial measurement unit (IMU) with wireless communication technology. The device comprises hard-electronic components on a stretchable elastic substrate for application in epidermal electronics, to collect precise data from the human body. Instead of the commonly used cleanroom processes of implementing devices on a stretchable substrate, a simple and inexpensive “cut-solder-paste” method is adopted to fabricate complex, convoluted interconnections. Thus, the signal distortions in the proposed device can be minimized during various physical activities and skin deformations when used in gait analysis. The inertial sensor data measured from the motion of the body can be sent in real-time via Bluetooth to any processing unit enabled with such a widespread standard wireless interface. For performance evaluation, the implemented device is mounted on a rotation plate in order to compare performance with the conventional product. In addition, an experiment on joint angle estimation is performed by attaching the device to an actual human body. The preliminary results of the device indicate the potential to monitor people in remote settings for applications in mobile health, human-computer interfaces (HCIs), and wearable robots.

1. Introduction

Technological advancements in wireless communications, miniaturized sensors, and low-power electronics have made it possible to implement integrated wireless body area networks (WBANs). These developments enable the establishment of highly reliable wireless, wearable systems, which provide continuous physiological biometric information that can be used for applications in diagnosis [1,2,3,4], health monitoring [5,6,7], rehabilitation [8], and dependency care [9,10]. Regarding these applications, the modeling and monitoring of human movement in daily environments has received considerable attention in recent years, including clinical and health sciences [11], robotics [12], and human-computer interfaces (HCIs) [13]. Analyzing kinematic parameters such as the joint range of motion and various gait parameters can track the progress of disease or evaluate the effects of rehabilitation in many neuromuscular and musculoskeletal disorders. Currently, these studies are being conducted in order to promote the health of elderly individuals, as well as in the monitoring of athletic training and injury prevention in sports medicine [4,14,15,16]. Joint angle estimation is a key component of human motion analysis, and many research groups have developed wearable sensors for continuous human motion analysis in real time through the use of optoelectronic systems [17], inertial measurement units (IMUs) [18,19,20], or ultrasonic sensors [21]. Optoelectronic systems are known to be the most accurate systems for estimating the joint angle. These systems use an infrared camera to analyze predefined anatomical positions in order to calculate the joint angles of a three-dimensional skeletal model. However, optoelectronic systems require experiments to be carried out in a controlled laboratory environment and require trained personnel to analyze the data. Thus, despite their accuracy, optoelectronic systems are not suitable for the continuous monitoring of joint movement in the ambulatory environment.

Recently, due to the rapid development of flexible/stretchable electronic devices, a number of soft wearable and portable devices have been reported [1,2,3,7]. By using thin and soft materials, the flexible bioelectronic modules can reliably interface with a part of the human body to monitor human motion with high fidelity. The skin-like wearable electronic modules have soft form-factors in order to achieve close contact with the human body, which prevents mechanical sliding and results in significantly improved performance. From the inherent skin-like characteristics of flexible and stretchable substrates, recent research has focused on the implementation of highly scalable and modularized epidermal electronic devices. Most commercially available wearable devices are implemented on a rigid substrate comprising silicon-based integrated circuit (IC) components. However, there is a large mechanical mismatch in the interface between the rigid devices and the human skin. To minimize data errors caused by movement of the human body, and to perform accurate measurements, it is essential that the wearable device is brought close to the skin. Additionally, in order to improve the fidelity between the system and the skin, the implementation of robust interconnecting parts is fundamental in the development of system-level-based epidermal electronic devices. Furthermore, the integration of wireless communication units enables continuous data logging with mobility, which can be used in ambulatory monitoring.

This paper presents an epidermal, patch-type, six-axis IMU for real-time joint angle estimation, which can be used in gait analysis, alleviating the effects of musculoskeletal disorders, or in the monitoring of daily routines. In the previous report [14], a comparison of the performance between an inertial sensor and an optoelectronic system was presented. In this paper, a total of 17 IMUs were installed for whole-body joint kinetic analysis. The inertial sensors were fixed to the head, trunk, pelvis, and upper/lower extremities of the subject. Wearable inertial sensors were used to measure joint forces and moments in the whole-body joints of the human subject, with reports indicating that they were suitable for normal movements. The proposed system is implemented using silicon-based ICs soldered onto bonding pads, with the metal interconnects between the ICs having serpentine structures. Typically, it has been reported that human skin can endure an ultimate failure strain of approximately 27% [22]. For commercialized wearable IMUs, it is recommended to wear straps for the purpose of preventing the devices from moving as a result of joint movement and breathing. To resolve this issue, the whole device is positioned on a soft adhesive substrate with stretchable serpentine-structured interconnects which is to be attached to the skin on any part of the human body with high conformity. The proposed system is configured by a commercialized six-axis accelerometer, a Bluetooth Low Energy (BLE) system-on-a-chip (SoC), several passive components, and a chip antenna which can obtain biometric information continuously and easily. The detailed design schematic, manufacturing process, and performance evaluations are discussed in the following sections. The results show that the system, and the method proposed in this paper, can be extended to the development of a high-performance, multiparametric sensor module platform.

2. Method

2.1. Design

A schematic of the epidermal six-axis IMU is shown in Figure 1. The inertial sensor and the signal processing parts used to construct the proposed system include a six-axis inertial sensor (MPU6050; Invensense, San Jose, CA, USA), BLE SoCs (nRF52832; Nordic Semiconductor ASA, Trondheim, Norway), and several passive components. All electronic components are mounted onto the bonding pads, and all components are connected by the serpentine-structured interconnects, allowing the device to elongate under various mechanical strains. The top and bottom layers of the implemented device are covered with a stretchable adhesive film (Tegaderm; 3M, Saint Paul, MN, USA) in order to protect the electronic parts from moisture or dust during attachment to the human body. The device is designed based on a modular concept that allows internal components to be reconfigured, allowing for a miniaturized system that is functionally optimized for a variety of applications. Thus, it can be used as a multiparametric epidermal sensor system, whilst also maintaining the signal processing building block and adding a sensor that can measure electrophysiological signals, strain, or body temperature. As shown in Figure 1, the IMUs can be used for kinematic analysis of human motion by the attachment to various parts of the human body and can be applied to the diagnosis and rehabilitation of degenerative neurological disorders.

Figure 2 shows a block diagram of the wireless epidermal six-axis IMU. The six-axis inertial sensor and the BLE SoC read and send data using the two-wired serial interface (TWI), which uses two bus signals (serial clack (SCL) and serial data (SDA)) to exchange signals. From the six-axis inertial sensor, the acceleration and angular velocity of each of three axes are converted by the embedded 16-bit analog-to-digital converter (ADC). The digitized data are passed through the bus interface unit of BLE SoC via I2C communication. As there is a specific address for each piece of data, there is an address matching unit that processes the data according to the address. In the control unit, the data accumulated in the bus interface unit are processed and the commands are decoded. After the data are processed, they are spread by the universal asynchronous receiver/transmitter (UART) structure in the signal conditioning unit, and through Bluetooth communication the inertial data can be displayed on a mobile device. In the case of the MPU-6050, the accelerometer uses a full-scale range of ±16 g and a sensitivity scale factor of 16,384 LSB/g, where g is gravitational acceleration. For gyroscopes, we used a full-scale range of ±2000°/s sensitivity scale factor of 131 LSB/°/s. The nRF52832 and MPU-6050 use I2C communication with a communication frequency of 400 kHz. The sample rate of the gyroscope and accelerometer of the MPU-6050 is 100 Hz, and the register is read with I2C communication in the nRF52832 and transmitted via Bluetooth in hexadecimal form. Our previous work on this system showed a capability of transmitting wireless inertial signals using BLE SoC (nRF51822) and reported the possibility of real-time human joint angle estimation [20]. In this paper, we use the nRF52832 to extend the functionality of the system. The nRF52832 includes a balun inside the chip, reducing the size and wiring of the entire system and facilitating impedance matching with ease. In addition, the nRF52832 has twice the RAM of the nRF51822, which allows the configuration of a BLE mesh. When constructing a mesh, a large RAM size is required, and sensor data can be collected simultaneously from 17 IMUs which are equipped for the purpose of analyzing the movement of all joints in the human body, enabling the scalability of the human body to a 3D model. Moreover, with BLE 5.0, the transmission speed can be doubled, and the transmission distance can be increased by approximately 2.5 times.

2.2. Manufacturing Process

The implementation process utilized to fabricate the epidermal six-axis IMU on a stretchable adhesive film is illustrated in Figure 3. This process uses a computer-aided design (CAD) program to implement miniaturized devices on substrates with various form factors. The most important consideration in designing this system is determining the limit of elongation, at which point there is no change in the resistance of the metal interconnect and the stress distribution in the system is not concentrated in one place. In our previous paper, various serpentine-structured interconnects were fabricated, and the characteristics of this robust metal interconnect were evaluated [19]. The metal interconnects of the serpentine structure are designed to have a line width of 300 μm, which is mechanically and electrically stable even when the device is applied to a tensile strain up to 30% of its original dimension. After designing the footprints using a CAD program, a programmable cutter (Silhouette Cameo^®, Silhouette America, USA) is used to define the serpentine-structured interconnects and soldering pads.

First, an 18-μm-thick piece of Cu foil (Copper 110 Annealed; Online Metals, Seattle, WA, USA) is laminated on the thermal release tape (TRT), as shown in Figure 3a,b. Then, as shown in Figure 3c, the TRT-laminated Cu foil is cut according to the interconnect and bonding pad pattern as designed by the CAD tool. After patterning by a programmable cutter, the residual area of the TRT-laminated Cu foil, except for the designed patterns, is removed manually using tweezers as depicted in Figure 3d. The opposite layer of the patterned TRT-laminated Cu foil is then bonded with the water-soluble tape (WST) and Kapton film, and the whole sample is bonded onto the glass substrate. As shown in Figure 3e, the bonded substrate is heated on a hot plate at 120 °C for 30 s, and the TRT layer is peeled off. After peeling off this layer, the bonding pad for circuit assembly is revealed. As shown in Figure 3f, the electronic components (six-axis inertial sensor, BLE SoC, chip antenna, passive components, and bridge interconnects) are aligned and fixed onto the bonding pads. Then, as shown in Figure 3g, the Kapton film is removed and the whole device is released from the glass substrate. The stretchable adhesive film is then used to cover the top layer of the circuit for encapsulation, as shown in Figure 3h. As shown in Figure 3i, water is dropped onto the WST layer using a pipette which dissolves the water-soluble compound of the WST. After this, the WST layer is peeled off to leave only the electronic components and the serpentine-structured Cu interconnects. Finally, after covering the bottom layer, the epidermal six-axis IMU on a stretchable adhesive film is ready for implementation.

3. Results

3.1. Implementation Results

The implementation results and mechanical characteristics of the epidermal six-axis IMU are shown in Figure 4. The overall size of the proposed system was approximately 40 × 37 mm², with the thickness being less than 100 μm (excluding the coin cell battery and ICs). In addition, the current consumption was measured to be 46 mA for full-function operation. The stress applied on the Cu interconnects as a result of strain and various deformations can be minimized by the serpentine structures, thus, the elongation limit of the implemented device depends on the stiffness of the stretchable substrate. Figure 4b shows the experimental waterproof results of the implemented device. The Bluetooth signal was continuously acquired, and data were transmitted despite water reaching the device. Owing to the semipermeable characteristics of the Tegaderm, the inner electronic circuitry could be perfectly protected from external intrusion during operation while attached to the skin.

Figure 4c,d presents the flexibility and stretchability experiments on the implemented epidermal device. Even after fully wrapping the device around a 20-mm-radius cylindrical tube or stretching it up to 30%, the acceleration and angular velocity signals sent through Bluetooth communication were maintained at steady levels, indicating that the serpentine-structured interconnects are robust to mechanical deformations. The flexibility and stretchability of the implemented device allow it to be utilized on the human body, including on the wrists, ankles, neck, and abdominal region, which are common sites for monitoring motion for clinical diagnosis.

3.2. Performance Evaluation

Performance evaluation was performed in order to estimate the joint angle using the fabricated device, and the specific signal processing procedure was as follows. First, using the six-axis inertial sensor, raw signals of the three-axis acceleration and the three-axis gyroscope were obtained. Then, the obtained three-axis acceleration signals were normalized and converted into the measured inertial direction. Next, it was necessary to account for any error in the value derived from the transformed measured inertial direction and the estimate of the gravitational direction. Since the error value of each data must be compensated using the acceleration and gyroscope data, the MahonyAHRS (Attitude and Heading Reference System) algorithm [23] was applied. The error value was reduced using the proportional and integral gains. The derived value was reduced through the proportional and integral gains, and then the quaternion was defined by calculating the value with the angular velocities from the three-axis gyroscope.

As mentioned previously, the gold standard for position estimation is to use an optoelectronic system. However, in this paper, the implemented epidermal 6-axis IMU was evaluated through a comparison experiment with Wearnotch (Notch Interfaces, Inc., New York City, NY, USA), a commercial wearable IMU that has previously been used to estimate the position in the ambulatory environment. According to the data-sheet [24], Wearnotch is generally known to have an error value of about 1–2° of angular estimation, and is thus a device that meets the maximum 5° estimation accuracy proposed by the American Medical Association for motion analysis in clinical situations [25].

First, the Euler angle measurement results received from the Wearnotch and epidermal IMU were measured using the rotation plate as shown in Figure 5. Both the implemented epidermal IMU and Wearnotch were positioned at the center of the rotation plate, which was placed horizontally, and the Euler angles were compared according to the rotation which ranged from 0 to 90°. Both devices were assessed by rotating them along the roll, pitch, and yaw axes, and each plot was plotted against the rotating and non-rotating axes. The two devices were measured separately, resulting in an offset in the time step. However, very similar waveforms were measured for the rotation axis, as shown in Figure 5a. The root-mean-square (RMS) errors of the epidermal IMU with respect to the Euler angle received from the Wearnotch were measured to be 1.41°, 1.60°, and 1.84° for the rotating roll, pitch, and yaw axes, respectively. In Figure 5b, the rotation angle estimation experiment was performed at the end of the rotation plate, approximately 30 cm from the center, using each device. This was a preliminary experiment for estimating the actual elbow joint angle, and the experiment was performed with the rotation plate set vertically. Similarly, the RMS error of the epidermal IMU for the Wearnotch was measured, with this experiment performed by setting up the devices in three different axial directions. As a result, the RMS error of the implemented device when rotating in the X-, Y-, and Z-axis directions were measured to be 1.94°, 3.31°, and 2.91°, respectively. The results for the above experiments are summarized in

Table 1. Summarized performance evaluation results.

	Center of Rotation Plate			End of Rotation Plate (30 cm from Center)
Axes	x	y	z	x	y	z
Maximum RMS error	1.41° (X-axis rotation)	1.60° (Y-axis rotation)	1.84° (Z-axis rotation)	1.94° (X-axis rotation)	3.31° (Y-axis rotation)	2.91° (Z-axis rotation)
Maximum angle difference	3.06° (X-axis rotation)	0.78° (X-axis rotation)	0.27° (X-axis rotation)	2.25° (X-axis rotation)	1.10° (X-axis rotation)	1.07° (X-axis rotation)
	0.97° (Y-axis rotation)	2.71° (Y-axis rotation)	1.05° (Y-axis rotation)	1.02° (Y-axis rotation)	3.79° (Y-axis rotation)	1.23° (Y-axis rotation)
	0.67° (Z-axis rotation)	0.79° (Z-axis rotation)	2.39° (Z-axis rotation)	1.63° (Z-axis rotation)	1.53° (Z-axis rotation)	3.38° (Z-axis rotation)

. As shown in Table 1, the maximum angle difference for the non-rotation axis was measured for each experiment to verify the operation of the error compensation algorithm.

As shown in Figure 6, the implemented wireless epidermal six-axis IMU was attached in order to estimate the precise joint angle of the forearm. The experiments were carried out using Wearnotch and the epidermal six-axis IMU using the 3D motion analysis program, with each experiment being performed for five joint angles ranging from 0° to 90°. The estimated joint angles with respect to the rotation axis for each device were compared and showed similar results, which could also be determined in the 3D motion analysis program.

4. Conclusions

This paper presents a wireless epidermal six-axis IMU for applications in wearable biomedical devices. In our previous reports, we have focused on the design of serpentine metal interconnect structures and wireless signal transmission via Bluetooth [19,20]. In this paper, a complementary filter algorithm was applied as a post-signal processing method to improve the performance, relating to issues such as error compensation and high–frequency-component removal. Additionally, mechanical experiments were conducted for applications in real-life use. The proposed device is implemented using conventional circuit components on a soft adhesive substrate, and by firmly attaching it to the target, it is possible to collect accurate data from the human body. The implemented system has dimensions of 40 × 37 mm², and the overall thickness of the system—excluding the battery and ICs—is less than 100 μm. The serpentine-structured interconnects are adopted between the soldered electronics to achieve flexibility and stretchability. The implemented device was mounted on a rotation plate to compare its performance with that of a commercially available device. The filtering of the angular velocity signal, relating to the rotational axis and the removal of noise components in the estimation of the rotational angle, is expected to show results similar to those of commercially available products. An experiment on joint angle estimation was performed by attaching the device to an actual human body. The maximum RMS error against the commercialized Wearnotch device was 3.31°, and the maximum angular displacement was 3.79°. The cause of the error between the two systems is currently being analyzed from multiple angles, with the most likely cause being based on the alignment error of the two systems. At present, a technique for correcting alignment error as well as an experimental method for improving it are currently envisioned. Furthermore, we are planning to analyze the acceleration and angular velocity with respect to the reference acceleration and angular rate through the vibration table and the rate table, and plan to compare the results using an optoelectronic system. The soft, epidermal system introduced in this paper is considered inexpensive and user-friendly, which can provide accurate joint angle estimates in the ambulatory environment. We are also in the process of optimizing the performance of this system for the future, so that it can provide the results of estimating the joint angle with more degrees of freedom, and ultimately provide data for the full body. The preliminary results of the device have potential for the monitoring of people in remote settings for applications in mobile health, HCIs, and wearable robots.