Ecological Fall Prediction Sensitivity, Specificity, and Accuracy in Patients with Mild Cognitive Impairment at a High Risk of Falls
Abstract
:1. Introduction
2. Materials and Methods
2.1. Participants
2.2. Description of PSD System
2.3. Experimental Procedures
2.3.1. BBS
2.3.2. TUG
2.3.3. ABC
2.4. ROC
2.5. Predicting Fall Risk by PSD
- True positive (TP) = the number of participants correctly identified as fallers.
- False positive (FP) = the number of participants incorrectly identified as fallers.
- True negative (TN) = the number of participants correctly identified as non-fallers.
- False negative (FN) = the number of participants incorrectly identified as non-fallers.
- Sensitivity: Sensitivity is the ability to determine cases correctly with respect to falling.
- Specificity: Specificity is the ability to determine non-faller cases correctly.
- Accuracy: Accuracy is the ability to correctly differentiate non-faller and faller cases.
- Youden’s index: The Youden’s index is one of the indicators that assesses the performance of diagnostic tests.Youden’s index = MAX (sensitivity + specificity − 1)
2.6. The Combination of Complete 360° Tilt Sensing
2.7. Data Processing
2.8. Statistical Analysis
3. Results
3.1. Fall Prediction AUC
3.2. Fall Prediction Sensitivity
3.3. Fall Prediction Specificity
3.4. Fall Prediction Accuracy
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Falls. Available online: https://www.who.int/en/news-room/fact-sheets/detail/falls (accessed on 26 April 2021).
- Szczepańska-Gieracha, J.; Cieślik, B.; Chamela-Bilińska, D.; Kuczyński, M. Postural stability of elderly people with cognitive impairments. Am. J. Alzheimers Dis. Other Demen 2016, 31, 241–246. [Google Scholar] [CrossRef] [PubMed]
- Melo, L.M.; Ansai, J.H.; Ferreira, A.C.V.G.; Silva, D.C.P.; Vale, F.A.C.; Takahashi, A.C.M.; Andrade, L.P. Correlation between changes in Timed Up and Go performance and cognition in older people with mild cognitive impairment: A longitudinal study. Clin. Biomech. 2022, 94, 105620. [Google Scholar] [CrossRef]
- Monika, S.K.; Gupta, A. Relationship between cognitive impairment and postural stability in the elderly population. Sport Sci. Health 2022, 19, 545–551. [Google Scholar] [CrossRef]
- Cedervall, Y.; Stenberg, A.M.; Åhman, H.B.; Giedraitis, V.; Tinmark, F.; Berglund, L.; Halvorsen, K.; Ingelsson, M.; Rosendahl, E.; Åberg, A.C. Timed up-and-go dual-task testing in the assessment of cognitive function: A mixed methods observational study for development of the UDDGait protocol. Int. J. Environ. Res. Public Health 2020, 17, 1715. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Saricaoglu, M.; Yilmaz, N.H.; Ozer, F.F.; Hanoglu, L. The correlation of non-motor symptoms and sleep on balance in Parkinson’s disease patients with normal cognition and mild cognitive impairment. Ir. J. Med. Sci. 2021, 190, 1577–1584. [Google Scholar] [CrossRef] [PubMed]
- Åhman, H.B.; Cedervall, Y.; Kilander, L.; Giedraitis, V.; Berglund, L.; McKee, K.J.; Rosendahl, E.; Ingelsson, M.; Åberg, A.C. Dual-task tests discriminate between dementia, mild cognitive impairment, subjective cognitive impairment, and healthy controls–across-sectional cohort study. BMC Geriatr. 2020, 20, 258. [Google Scholar] [CrossRef]
- Mancioppi, G.; Fiorini, L.; Rovini, E.; Zeghari, R.; Gros, A.; Manera, V.; Robert, P.; Cavallo, F. Innovative motor and cognitive dual-task approaches combining upper and lower limbs may improve dementia early detection. Sci. Rep. 2021, 11, 7449. [Google Scholar] [CrossRef]
- You, J.H.; Shetty, A.; Jones, T.; Shields, K.; Belay, Y.; Brown, D. Effects of dual-task cognitive-gait intervention on memory and gait dynamics in older adults with a history of falls: A preliminary investigation. NeuroRehabilitation 2009, 24, 193–198. [Google Scholar] [CrossRef]
- Johansson, J.; Jarocka, E.; Westling, G.; Nordström, A.; Nordström, P. Predicting incident falls: Relationship between postural sway and limits of stability in older adults. Hum. Mov. Sci. 2019, 66, 117–123. [Google Scholar] [CrossRef]
- Sun, R.; Hsieh, K.L.; Sosnoff, J.J. Fall risk prediction in multiple sclerosis using postural sway measures: A machine learning approach. Sci. Rep. 2019, 9, 16154. [Google Scholar] [CrossRef] [Green Version]
- Dewan, B.M.; James, R.C.; Kumar, N.A.; Sawyer, S.F. Kinematic validation of postural sway measured by biodex biosway (force plate) and SWAY balance (accelerometer) technology. Biomed. Res. Int. 2019, 2019, 8185710. [Google Scholar] [CrossRef] [Green Version]
- Pollind, M.L.; Soangra, R. Mini-Logger-A wearable inertial measurement unit (IMU) for postural sway analysis. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2020, 2020, 4600–4603. [Google Scholar]
- Vandermeeren, S.; Steendam, H. Deep-learning-based step detection and step length estimation with a handheld IMU. IEEE Sens. J. 2022, 22, 24205–24221. [Google Scholar] [CrossRef]
- Ding, W.; Hu, B.; Liu, H.; Wang, X.; Huang, X. Human posture recognition based on multiple features and rule learning. Int. J. Mach. Learn. Cyber 2020, 11, 2529–2540. [Google Scholar] [CrossRef]
- Romijnders, R.; Warmerdam, E.; Hansen, C.; Schmidt, G.; Maetzler, W. A deep learning approach for gait event detection from a single shank-worn IMU: Validation in healthy and neurological cohorts. Sensors 2022, 22, 3859. [Google Scholar] [CrossRef]
- Huang, F.; Wang, Z.; Xing, L.; Gao, C. A MEMS IMU gyroscope calibration method based on deep learning. IEEE Trans. Instrum. Meas. 2022, 71, 1003009. [Google Scholar] [CrossRef]
- Lyons, N.; Santra, A.; Pandey, A. Improved deep representation learning for human activity recognition using IMU sensors. In Proceedings of the 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Pasadena, CA, USA, 13–16 December 2021. [Google Scholar]
- Ghoraani, B.; Boettcher, L.N.; Hssayeni, M.D.; Rosenfeld, A.; Tolea, M.I.; Galvin, J.E. Detection of mild cognitive impairment and Alzheimer’s disease using dual-task gait assessments and machine learning. Biomed. Signal. Process Control 2021, 64, 102249. [Google Scholar] [CrossRef]
- Hillel, I.; Gazit, E.; Nieuwboer, A.; Avanzino, L.; Rochester, L.; Cereatti, A.; Croce, U.D.; Rikkert, M.O.; Bloem, B.R.; Pelosin, E.; et al. Is every-day walking in older adults more analogous to dual-task walking or to usual walking? Elucidating the gaps between gait performance in the lab and during 24/7 monitoring. Eur. Rev. Aging Phys. Act. 2019, 16, 6. [Google Scholar] [CrossRef]
- Ponti, M.; Bet, P.; Oliveira, C.L.; Castro, P.C. Better than counting seconds: Identifying fallers among healthy elderly using fusion of accelerometer features and dual-task Timed Up and Go. PLoS ONE 2017, 12, e0175559. [Google Scholar] [CrossRef] [Green Version]
- Godfrey, A.; Bourke, A.; Del Din, S.; Morris, R.; Hickey, A.; Helbostad, J.L.; Rochester, L. Towards holistic free-living assessment in Parkinson’s disease: Unification of gait and fall algorithms with a single accelerometer. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2016, 2016, 651–654. [Google Scholar]
- D’Angelo, M.G.; Bresolin, N. Cognitive impairment in neuromuscular disorders. Muscle Nerve 2006, 34, 16–33. [Google Scholar] [CrossRef] [PubMed]
- Kim, H.; Park, C.; You, J.S.H. Concurrent validity, test-retest reliability, and sensitivity of a PostureRite system measurement on dynamic postural sway and risk of fall in cerebral palsy. NeuroRehabilitation 2022, 51, 151–159. [Google Scholar] [CrossRef] [PubMed]
- Gonçalves, J.; Ansai, J.H.; Masse, F.A.A.; Vale, F.A.C.; de Medeiros Takahashi, A.C.; de Andrade, L.P. Dual-task as a predictor of falls in older people with mild cognitive impairment and mild Alzheimer’s disease: A prospective cohort study. Braz. J. Phys. Ther. 2018, 22, 417–423. [Google Scholar] [CrossRef] [PubMed]
- Shumway-Cook, A.; Baldwin, M.; Polissar, N.L.; Gruber, W. Predicting the probability for falls in community-dwelling older adults. Phys. Ther. 1997, 77, 812–819. [Google Scholar] [CrossRef] [Green Version]
- Conradsson, M.; Lundin-Olsson, L.; Lindelöf, N.; Littbrand, H.; Malmqvist, L.; Gustafson, Y.; Rosendahl, E. Berg balance scale: Intrarater test-retest reliability among older people dependent in activities of daily living and living in residential care facilities. Phys. Ther. 2007, 87, 1155–1163. [Google Scholar] [CrossRef] [Green Version]
- Time up and Go. Available online: https://www.sralab.org/rehabilitation-measures/timed-and-go (accessed on 6 November 2013).
- Shumway-Cook, A.; Brauer, S.; Woollacott, M. Predicting the probability for falls in community-dwelling older adults using the Timed up & Go Test. Phys. Ther. 2000, 80, 896–903. [Google Scholar]
- Steffen, T.M.; Hacker, T.A.; Mollinger, L. Age-and gender-related test performance in community-dwelling elderly people: Six-minute walk test, Berg balance scale, timed up & go test, and gait speeds. Phys. Ther. 2002, 82, 128–137. [Google Scholar]
- Lajoie, Y.; Gallagher, S.P. Predicting falls within the elderly community: Comparison of postural sway, reaction time, the Berg balance scale, and activities-specific balance confidence (ABC) scale for comparing fallers and non-fallers. Arch. Gerontol. Geriatr. 2004, 38, 11–26. [Google Scholar] [CrossRef]
- Lalkhen, A.G.; McCluskey, A. Clinical tests: Sensitivity and specificity. Contin. Educ. Anaesth. Crit. Care Pain 2008, 8, 221–223. [Google Scholar] [CrossRef] [Green Version]
- Roshdibenam, V.; Jogerst, G.J.; Butler, N.R.; Baek, S. Machine learning prediction of fall risk in older adults using Timed Up and Go test kinematics. Sensors 2021, 21, 3481. [Google Scholar] [CrossRef]
- Baratloo, A.; Hosseini, M.; Negida, A.; El Ashal, G. Part 1: Simple definition and calculation of accuracy, sensitivity and specificity. Emerg 2015, 3, 48–49. [Google Scholar]
- Ghahramani, M.; Stirling, D.; Naghdy, F.; Naghdy, G.; Potter, J. Body postural sway analysis in older people with different fall histories. Med. Biol. Eng. Comput. 2019, 57, 533–542. [Google Scholar] [CrossRef]
- Kelly, D.; Condell, J.; Gillespie, J.; Munoz Esquivel, K.; Barton, J.; Tedesco, S.; Nordstrom, A.; Åkerlund Larsson, M.; Alamäki, A. Improved screening of fall risk using free-living based accelerometer data. J. Biomed. Inform. 2022, 131, 104116. [Google Scholar] [CrossRef]
- Chheda, D.L. Multi-Sensor Architecture Development for Intelligent Systems. Master’s Thesis, The University of Texas at Austin, Austin, TX, USA, May 2014. [Google Scholar]
- Tulipani, L.J.; Meyer, B.; Allen, D.; Solomon, A.J.; McGinnis, R.S. Evaluation of unsupervised 30-second chair stand test performance assessed by wearable sensors to predict fall status in multiple sclerosis. Gait Posture 2022, 94, 19–25. [Google Scholar] [CrossRef]
- Bet, P.; Castro, P.C.; Ponti, M.A. Foreseeing future falls with accelerometer features in active community-dwelling older persons with no recent history of falls. Exp. Gerontol. 2021, 143, 111139. [Google Scholar] [CrossRef]
- Howcroft, J.; Kofman, J.; Lemaire, E.D. Prospective fall-risk prediction models for older adults based on wearable sensors. IEEE Trans. Neural. Syst. Rehabil. Eng. 2017, 25, 1812–1820. [Google Scholar] [CrossRef]
- Delgado-Escaño, R.; Castro, F.M.; Cózar, J.R.; Marín-Jiménez, M.J.; Guil, N.; Casilari, E. A cross-dataset deep learning-based classifier for people fall detection and identification. Comput. Methods Programs Biomed. 2020, 184, 105265. [Google Scholar] [CrossRef]
- Cai, Q.; Song, N.; Yang, G.; Liu, Y. Accelerometer calibration with nonlinear scale factor based on multi-position observation. Meas. Sci. Technol. 2013, 24, 105002. [Google Scholar] [CrossRef]
- Galindo-Tellez, A.; Keefe, K.; Adamek, E.; Brubaker, E.; Crow, B.; Dorrill, R.; Druetzler, A.; Felix, C.J.; Kaneshige, N.; Learned, J.G.; et al. Design and calibration of an optically segmented single volume scatter camera for neutron imaging. J. Instrum. 2021, 16, P04013. [Google Scholar] [CrossRef]
- Fisher, C.J. Using an Accelerometer for Inclination Sensing; AN-1057, Application Note; Analog Devices: Wilmington, MA, USA, 2010; pp. 1–8. [Google Scholar]
- Berg, K.O.; Wood-Dauphinee, S.L.; Williams, J.I.; Maki, B. Measuring balance in the elderly: Validation of an instrument. Can. J. Public Health 1992, 83, S7–S11. [Google Scholar]
- Park, S.H.; Lee, Y.S. The diagnostic accuracy of the Berg balance scale in predicting falls. West J. Nurs. Res. 2017, 39, 1502–1525. [Google Scholar] [CrossRef] [PubMed]
Variable | Healthy Adults (n = 20) | Older Adults with MCI (n = 20) | p-Value | |
---|---|---|---|---|
Age (years), mean ± SD | 25.20 ± 3.19 | 79.00 ± 8.25 | 0.001 | |
Height (cm), mean ± SD | 169.65 ± 9.37 | 157.25 ± 8.45 | 0.001 | |
Weight (kg), mean ± SD | 67.70 ± 13.83 | 59.40 ± 7.30 | 0.024 | |
Male (%) | 50 | 50 | - | |
Female (%) | 50 | 50 | - | |
BBS | Clinical tests (score) mean ± SD | 54.61 ± 1.33 | 39 ± 5.88 | 0.001 |
Posture sway data * mean ± SD | 0.16 ± 0.20 | 0.41 ± 0.12 | 0.001 | |
TUG | Clinical tests (second) mean ± SD | 9.42 ± 0.25 | 14.11 ± 1.37 | 0.001 |
Posture sway data * mean ± SD | 0.36 ± 0.08 | 0.47 ± 0.12 | 0.01 |
Subscales | AUC | SE | 95% Confidence Interval | p-Value | ||
---|---|---|---|---|---|---|
Lower | Upper | |||||
BBS | AP | 0.91 | 0.50 | 0.82 | 0.99 | 0.001 |
ML | 0.94 | 0.40 | 0.87 | 0.99 | 0.001 | |
V | 0.91 | 0.50 | 0.81 | 0.99 | 0.001 | |
Mean * | 0.92 | 0.47 | 0.84 | 0.99 | 0.001 | |
TUG | AP | 0.83 | 0.68 | 0.69 | 0.96 | 0.002 |
ML | 0.82 | 0.73 | 0.68 | 0.96 | 0.002 | |
V | 0.86 | 0.62 | 0.74 | 0.98 | 0.001 | |
Mean * | 0.84 | 0.68 | 0.70 | 0.97 | 0.002 |
Subscales | Cut-Off Point | Sensitivity | Specificity | Accuracy | |
---|---|---|---|---|---|
BBS | AP | 0.39 | 0.88 | 0.83 | 0.81 |
ML | 0.34 | 0.88 | 0.90 | 0.76 | |
V | 0.34 | 0.99 | 0.76 | 0.76 | |
Mean * | 0.36 | 0.92 | 0.83 | 0.78 | |
TUG | AP | 0.50 | 0.82 | 0.73 | 0.53 |
ML | 0.39 | 0.91 | 0.69 | 0.58 | |
V | 0.49 | 0.91 | 0.69 | 0.53 | |
Mean * | 0.46 | 0.88 | 0.70 | 0.55 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kim, C.; Park, H.; You, J. Ecological Fall Prediction Sensitivity, Specificity, and Accuracy in Patients with Mild Cognitive Impairment at a High Risk of Falls. Sensors 2023, 23, 6977. https://doi.org/10.3390/s23156977
Kim C, Park H, You J. Ecological Fall Prediction Sensitivity, Specificity, and Accuracy in Patients with Mild Cognitive Impairment at a High Risk of Falls. Sensors. 2023; 23(15):6977. https://doi.org/10.3390/s23156977
Chicago/Turabian StyleKim, Chaesu, Haeun Park, and Joshua (Sung) You. 2023. "Ecological Fall Prediction Sensitivity, Specificity, and Accuracy in Patients with Mild Cognitive Impairment at a High Risk of Falls" Sensors 23, no. 15: 6977. https://doi.org/10.3390/s23156977
APA StyleKim, C., Park, H., & You, J. (2023). Ecological Fall Prediction Sensitivity, Specificity, and Accuracy in Patients with Mild Cognitive Impairment at a High Risk of Falls. Sensors, 23(15), 6977. https://doi.org/10.3390/s23156977