A Low-Cost, Wireless, 3-D-Printed Custom Armband for sEMG Hand Gesture Recognition
Abstract
:1. Introduction
2. Overview of Surface EMG Acquisition Systems
3. The 3DC Armband (Prototype)
3.1. System Overview
- An ICM-20948 low-power 9-axis IMU from InvenSense, USA. This component has a 3-axis gyroscope, a 3-axis accelerometer, and a 3-axis magnetometer.
- An nRF24L01+ low-power wireless transceiver from Nordic Semiconductors, Norway, which sends the sEMG and IMU data to a base station at a 1 Mbps datarate.
- An MSP430F5328 low-power microcontroller unit (MCU) from Texas Instruments, USA. This MCU is mainly used for interfacing the SoC, the IMU, and the wireless transceiver.
- The power management unit (PMU), which includes a 1.9-V low-dropout regulator (LDO) for powering the MCU, the wireless transceiver, and the IMU. The highest voltage in the sEMG sensor is 1.9-V, which is optimized for low-power consumption since it is the smallest viable voltage for powering the MCU, the IMU, and the wireless transceiver, yielding around half the power consumption compared with a typical 3.3-V power supply. The PMU also includes a 1.2-V LDO for powering the SoC, which is the recommend supply voltage for the 0.13-m technology used in the SoC. The system is powered with a 100-mAh LiPo battery.
- The Molex connector (# 0529910308) used for connecting with the Armband and for programming the MCU.
3.2. sEMG Acquisition Interface
3.3. MCU Firmware
3.4. Inertial Measurement Unit
3.5. 3-D Printing Models
4. Comparison Dataset
4.1. Data Acquisition Protocol
4.2. Preprocessing
5. Comparison Methods
5.1. Baseline Method
5.1.1. Hudgins’ Time-Domain Feature Set (H-TD)
- Mean Absolute Value
- Zero Crossing
- Slope Sign Changes
- Waveform Length
5.1.2. Linear Discriminant Analysis
5.2. Raw sEMG Classification
5.3. Time-Frequency Domain Classification
6. Results
7. Discussion
8. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Ethical Statement
Abbreviations
ADC | Analog-to-digital converter |
ANN | Artificial Neural Network |
ASIC | Application-specific integrated circuit |
BML-UL | Biomedical Microsystems Laboratory in Laval University |
CIC | Cascaded integrator-comb |
CMOS | Complementary metal-oxide-semiconductor |
ConvNet | Convolutional Network |
DMA | Direct memory access |
ENOB | Effective number of bits |
H-TD | Hudgins’ Time-Domain Feature Set |
IMU | sample per second |
LDA | Linear Discriminant Analysis |
LDO | Low-dropout regulator |
MARG | Magnetic, Angular Rate, and Gravity |
MCU | Microcontroller unit |
ms | miliseconds |
OSR | Oversampling ratio |
OTA | Operational transconductance amplifier |
sEMG | Surface Electromyography |
SNR | Signal to Noise Ratio |
SoC | System-on-chip |
SPI | serial peripheral interface |
sps | sample per second |
SVM | Support Vector Machine |
PCB | Printed circuit board |
PMU | Power management unit |
PWM | Pulse-width modulation |
USD | United States dollar |
Appendix A. Confusion Matrices
Appendix A.1. LDA Classifier Confusion Matrices
Appendix A.2. Raw ConvNet Classifier Confusion Matrices
Appendix A.3. Spectrogram ConvNet Classifier Confusion Matrices
References
- Hakonen, M.; Piitulainen, H.; Visala, A. Current state of digital signal processing in myoelectric interfaces and related applications. Biomed. Signal Process. Control 2015, 18, 334–359. [Google Scholar] [CrossRef] [Green Version]
- Allard, U.C.; Nougarou, F.; Fall, C.L.; Giguère, P.; Gosselin, C.; Laviolette, F.; Gosselin, B. A convolutional neural network for robotic arm guidance using semg based frequency-features. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 2464–2470. [Google Scholar]
- Janke, M.; Diener, L. Emg-to-speech: Direct generation of speech from facial electromyographic signals. IEEE/ACM Trans. Audio Speech Lang. Process. 2017, 25, 2375–2385. [Google Scholar] [CrossRef]
- Oskoei, M.A.; Hu, H. Myoelectric control systems—A survey. Biomed. Signal Process. Control 2007, 2, 275–294. [Google Scholar] [CrossRef]
- Stegeman, D.F.; Kleine, B.U.; Lapatki, B.G.; Van Dijk, J.P. High-density surface EMG: Techniques and applications at a motor unit level. Biocybern. Biomed. Eng. 2012, 32, 3–27. [Google Scholar]
- Pizzolato, S.; Tagliapietra, L.; Cognolato, M.; Reggiani, M.; Müller, H.; Atzori, M. Comparison of six electromyography acquisition setups on hand movement classification tasks. PLoS ONE 2017, 12, e0186132. [Google Scholar] [CrossRef] [PubMed]
- Phinyomark, A.; N Khushaba, R.; Scheme, E. Feature extraction and selection for myoelectric control based on wearable EMG sensors. Sensors 2018, 18, 1615. [Google Scholar] [CrossRef]
- Fraser, G.D.; Chan, A.D.; Green, J.R.; MacIsaac, D.T. Automated biosignal quality analysis for electromyography using a one-class support vector machine. IEEE Trans. Instrum. Meas. 2014, 63, 2919–2930. [Google Scholar] [CrossRef]
- Noraxon Ultium EMG. Available online: https://www.noraxon.com/our-products/ultium-emg/#1541097779421-89a192e6-7d8d (accessed on 23 April 2019).
- Delsys Trigno Avanti Sensor. Available online: https://www.delsys.com/trigno/research/#trigno-avanti-sensor (accessed on 23 April 2019).
- Biometrics DataLITE Wireless Surface EMG Sensor. Available online: http://www.biometricsltd.com/wireless-sensors.htm#emg (accessed on 23 April 2019).
- Thalmics Labs Myo Armband. Available online: https://support.getmyo.com/hc/en-us (accessed on 23 April 2019).
- Tabor, A.; Bateman, S.; Scheme, E. Evaluation of Myoelectric Control Learning Using Multi-Session Game-Based Training. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 1680–1689. [Google Scholar] [CrossRef]
- O’Shea, H.; Moran, A. To go or not to go? Pupillometry elucidates inhibitory mechanisms in motor imagery. J. Cogn. Psychol. 2018, 30, 466–483. [Google Scholar] [CrossRef]
- Abreu, J.G.; Teixeira, J.M.; Figueiredo, L.S.; Teichrieb, V. Evaluating sign language recognition using the myo armband. In Proceedings of the 2016 XVIII Symposium on Virtual and Augmented Reality (SVR), Gramado, Brazil, 21–24 June 2016; pp. 64–70. [Google Scholar]
- Oymotion G-Force pro. Available online: http://www.oymotion.com/site/ (accessed on 23 April 2019).
- Benatti, S.; Casamassima, F.; Milosevic, B.; Farella, E.; Schönle, P.; Fateh, S.; Burger, T.; Huang, Q.; Benini, L. A versatile embedded platform for EMG acquisition and gesture recognition. IEEE Trans. Biomed. Circuits Syst. 2015, 9, 620–630. [Google Scholar] [CrossRef]
- Chang, K.M.; Liu, S.H.; Wu, X.H. A wireless sEMG recording system and its application to muscle fatigue detection. Sensors 2012, 12, 489–499. [Google Scholar] [CrossRef] [PubMed]
- Ergeneci, M.; Gokcesu, K.; Ertan, E.; Kosmas, P. An embedded, eight channel, noise canceling, wireless, wearable sEMG data acquisition system with adaptive muscle contraction detection. IEEE Trans. Biomed. Circuits Syst. 2018, 12, 68–79. [Google Scholar] [CrossRef] [PubMed]
- Wu, J.; Sun, L.; Jafari, R. A wearable system for recognizing American sign language in real-time using IMU and surface EMG sensors. IEEE J. Biomed. Health Inform. 2016, 20, 1281–1290. [Google Scholar] [CrossRef] [PubMed]
- Fang, Y.; Zhu, X.; Liu, H. Development of a surface emg acquisition system with novel electrodes configuration and signal representation. In Proceedings of the International Conference on Intelligent Robotics and Applications, Busan, Korea, 25–28 September 2013; pp. 405–414. [Google Scholar]
- Gagnon-Turcotte, G.; Ethier, C.; De Köninck, Y.; Gosselin, B. A 0.13-μm CMOS SoC for Simultaneous Multichannel Optogenetics and Electrophysiological Brain Recording. In Proceedings of the IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, CA, USA, 11–15 February 2018; Volume 32, pp. 466–468. [Google Scholar]
- Gagnon-Turcotte, G.; Khiarak, M.N.N.; Ethier, C.; De Koninck, Y.; Gosselin, B. A 0.13-μm CMOS SoC for Simultaneous Multichannel Optogenetics and Neural Recording. IEEE J. Solid-State Circuits 2018, 53, 3087–3100. [Google Scholar] [CrossRef]
- Gagnon-Turcotte, G.; Fall, C.L.; Mascret, Q.; Bielmann, M.; Bouyer, L.; Gosselin, B. A Multichannel Wireless sEMG Sensor Endowing a 0.13 μm CMOS Mixed-Signal SoC. In Proceedings of the IEEE Life Sciences Conference (LSC), Montreal, QC, Canada, 28–30 October 2018; pp. 1–4. [Google Scholar]
- Fall, C.L.; Gagnon-Turcotte, G.; Dube, J.F.; Gagne, J.S.; Delisle, Y.; Campeau-Lecours, A.; Gosselin, C.; Gosselin, B. Wireless sEMG-Based Body–Machine Interface for Assistive Technology Devices. IEEE J. Biomed. Health Inform. 2017, 21, 967–977. [Google Scholar] [CrossRef] [PubMed]
- Madgwick, S.O.; Harrison, A.J.; Vaidyanathan, R. Estimation of IMU and MARG orientation using a gradient descent algorithm. In Proceedings of the 2011 IEEE International Conference on Rehabilitation Robotics, Zurich, Switzerland, 29 June–1 July 2011; pp. 1–7. [Google Scholar]
- Gómez-Espinosa, A.; Espinosa-Castillo, N.; Valdés-Aguirre, B. Foot-Mounted Inertial Measurement Units-Based Device for Ankle Rehabilitation. Appl. Sci. 2018, 8, 2032. [Google Scholar] [CrossRef]
- Georgi, M.; Amma, C.; Schultz, T. Recognizing Hand and Finger Gestures with IMU based Motion and EMG based Muscle Activity Sensing. In Proceedings of the International Conference on Bio-Inspired Systems and Signal Processing (BIOSIGNALS-2015), Lisbon, Portugal, 12–15 January 2015; pp. 99–108. [Google Scholar]
- St-Onge, D.; Côté-Allard, U.; Glette, K.; Gosselin, B.; Beltrame, G. Engaging with Robotic Swarms: Commands from Expressive Motion. ACM Trans. Hum.-Robot Interact. 2019, 8, 1–26. [Google Scholar] [CrossRef]
- Wolf, M.T.; Assad, C.; Stoica, A.; You, K.; Jethani, H.; Vernacchia, M.T.; Fromm, J.; Iwashita, Y. Decoding static and dynamic arm and hand gestures from the JPL BioSleeve. In Proceedings of the 2013 IEEE Aerospace Conference, Big Sky, MT, USA, 2–9 March 2013; pp. 1–9. [Google Scholar]
- Zhang, X.; Chen, X.; Li, Y.; Lantz, V.; Wang, K.; Yang, J. A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2011, 41, 1064–1076. [Google Scholar] [CrossRef]
- Smith, L.H.; Hargrove, L.J.; Lock, B.A.; Kuiken, T.A. Determining the optimal window length for pattern recognition-based myoelectric control: Balancing the competing effects of classification error and controller delay. IEEE Trans. Neural Syst. Rehabil. Eng. 2011, 19, 186–192. [Google Scholar] [CrossRef]
- Englehart, K.; Hudgins, B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 2003, 50, 848–854. [Google Scholar] [CrossRef]
- Scheme, E.; Englehart, K. Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use. J. Rehabil. Res. Dev. 2011, 48, 643–659. [Google Scholar] [CrossRef] [PubMed]
- Phinyomark, A.; Hirunviriya, S.; Limsakul, C.; Phukpattaranont, P. Evaluation of EMG feature extraction for hand movement recognition based on Euclidean distance and standard deviation. In Proceedings of the ECTI-CON2010: The 2010 ECTI International Confernce on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, Chiang Mai, Thailand, 19–21 May 2010; pp. 856–860. [Google Scholar]
- Côté-Allard, U.; Fall, C.L.; Drouin, A.; Campeau-Lecours, A.; Gosselin, C.; Glette, K.; Laviolette, F.; Gosselin, B. Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 760–771. [Google Scholar] [Green Version]
- Jiang, S.; Lv, B.; Guo, W.; Zhang, C.; Wang, H.; Sheng, X.; Shull, P.B. Feasibility of Wrist-Worn, Real-Time Hand, and Surface Gesture Recognition via sEMG and IMU Sensing. IEEE Trans. Ind. Inform. 2018, 14, 3376–3385. [Google Scholar] [CrossRef]
- Zia ur Rehman, M.; Waris, A.; Gilani, S.; Jochumsen, M.; Niazi, I.; Jamil, M.; Farina, D.; Kamavuako, E. Multiday EMG-based classification of hand motions with deep learning techniques. Sensors 2018, 18, 2497. [Google Scholar] [CrossRef] [PubMed]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Cote-Allard, U.; Fall, C.L.; Campeau-Lecours, A.; Gosselin, C.; Laviolette, F.; Gosselin, B. Transfer learning for sEMG hand gestures recognition using convolutional neural networks. In Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada, 5–8 October 2017; pp. 1663–1668. [Google Scholar]
- Demšar, J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 2006, 7, 1–30. [Google Scholar]
- Wilcoxon, F. Individual comparisons by ranking methods. Biom. Bull. 1945, 1, 80–83. [Google Scholar] [CrossRef]
- Phinyomark, A.; Scheme, E. EMG pattern recognition in the era of big data and deep learning. Big Data Cogn. Comput. 2018, 2, 21. [Google Scholar] [CrossRef]
- Muhlstell, J.; Such, O. Dry electrodes for monitoring of vital signs in functional textiles. IEEE Eng. Med. Biol. Soc. 2004, 1, 2212–2215. [Google Scholar]
- Young, A.J.; Hargrove, L.J.; Kuiken, T.A. The effects of electrode size and orientation on the sensitivity of myoelectric pattern recognition systems to electrode shift. IEEE Trans. Biomed. Eng. 2011, 58, 2537–2544. [Google Scholar] [CrossRef]
Delsys Systems Trigno Avanti | Biometrics DataLITE sEMG | Noraxon Ultium EMG | Oymotion gForce-Pro | Thalmic Lab Myo Armband | Hercules | 3DC Armband | |
---|---|---|---|---|---|---|---|
sEMG channels | up to 16 | up to 16 | up to 32 (at 2000 sps) or 16 (at 4000 sps) | 8 | 8 | 8 | 10 |
sEMG ADC * | 16 bits | 13 bits | 16 bits | 8 bits | 8 bits | 12 bits | 10 bits (ENOB *) (data sent on 16 bits) |
sEMG Sampling rate | 1960 sps | 2000 sps | 4000 sps | 1000 sps | 200 sps | 1000 sps | 1000 sps |
Bandwidth or Built-in Filters | 20–450 Hz or 10–850 Hz | 10–490 Hz | 5/10/20– 500/1000/1500 Hz | 20–500 Hz | ∼5–100 Hz | 20–500 Hz | 20–500 Hz |
Contact Dimensions | 5 mm | 78 mm | N.A. | ∼66 mm | 100 mm | 78 mm | 50 mm |
Contact Material | Silver | Stainless Steel | N.A. | Stainless steel silver coated | Stainless Steel | Gold plated Copper | Electroless nickel immersion gold (ENIG) |
Full Scale (Peak to Peak) | +/−11 sps or +/−22 sps | +/−6 sps | +/−24 sps | N.A. | ∼+/−1 sps (measured) | +/−6 sps | +/−3 sps |
Input referred-noise (On system bandwith) | N.A. | <5V | <1 V | N.A. | N.A. | N.A. | 2.2 V |
IMU * sensors | 9-axis Acc, Gyro, Mag | No | 9-axis Acc, Gyro, Mag (if EMG set at 2000 sps or below) | 9-axis Acc, Gyro, Mag | 9-axis Acc, Gyro, Mag | No | 9-axis Acc, Gyro, Mag |
IMU Sampling rate | 24–470 Hz (Acc), 24–360 Hz (Gyro), 50 Hz (Mag) | - | 200 Hz | 50 Hz | 50 Hz | - | 50 Hz |
Transmitter | BLE 4.2 | WiFi | 2.4 GHz | BLE 4.1 | BLE 4.0 | Wi-Fi | Enhanced Shockburst ** |
Autonomy | 4 to 8 h | 8 h | 8 h | N.A. | 16 h | N.A. | 6 h |
Weight | 14 g (per channel) | 17 g (per channel) | 14 g (per channel) | 80 g | 93 g | N.A. | 62 g |
Price | ∼$20,000 USD (for 16 channels) | ∼$17,000 USD (for 16 channels) | ∼$20,000 USD (for 16 channels and free battery replacement) | $1250 USD | $200 USD | N.A. | ∼$150 USD *** |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Côté-Allard, U.; Gagnon-Turcotte, G.; Laviolette, F.; Gosselin, B. A Low-Cost, Wireless, 3-D-Printed Custom Armband for sEMG Hand Gesture Recognition. Sensors 2019, 19, 2811. https://doi.org/10.3390/s19122811
Côté-Allard U, Gagnon-Turcotte G, Laviolette F, Gosselin B. A Low-Cost, Wireless, 3-D-Printed Custom Armband for sEMG Hand Gesture Recognition. Sensors. 2019; 19(12):2811. https://doi.org/10.3390/s19122811
Chicago/Turabian StyleCôté-Allard, Ulysse, Gabriel Gagnon-Turcotte, François Laviolette, and Benoit Gosselin. 2019. "A Low-Cost, Wireless, 3-D-Printed Custom Armband for sEMG Hand Gesture Recognition" Sensors 19, no. 12: 2811. https://doi.org/10.3390/s19122811
APA StyleCôté-Allard, U., Gagnon-Turcotte, G., Laviolette, F., & Gosselin, B. (2019). A Low-Cost, Wireless, 3-D-Printed Custom Armband for sEMG Hand Gesture Recognition. Sensors, 19(12), 2811. https://doi.org/10.3390/s19122811