Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Participants
2.2. Design and Procedure
2.3. Measures
2.3.1. Sleep Diary
2.3.2. Three-Night Self-Applied Polysomnography
2.3.3. Insomnia Severity Index (ISI)
2.3.4. Epworth Sleepiness Scale (ESS)
2.3.5. Depression Anxiety and Stress Scale 21 (DASS-21)
2.3.6. Covariates
2.4. Statistical Analytics
3. Results
3.1. Descriptive Statistics
3.2. Overall Compliance
3.2.1. Compliance by Individual Differences
3.2.2. Compliance on Weekdays and Weekend Days
3.3. Total Sleep Time
4. Discussion
Strengths and Limitations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Walker, M.P. Sleep essentialism. Brain 2021, 144, 697–699. [Google Scholar] [CrossRef]
- Benjafield, A.V.; Ayas, N.T.; Eastwood, P.R.; Heinzer, R.; Ip, M.S.; Morrell, M.J.; Nunez, C.M.; Patel, S.R.; Penzel, T.; Pépin, J.L.; et al. Estimation of the global prevalence and burden of obstructive sleep apnoea: A literature-based analysis. Lancet Respir. Med. 2019, 7, 687–698. [Google Scholar] [CrossRef] [PubMed]
- O’Reilly, J. Costs of Treatment Non-adherence in Obstructive Sleep Apnoea. In CPAP Adherence: Factors and Perspectives; Springer: Berlin/Heidelberg, Germany, 2022; pp. 125–140. [Google Scholar]
- Filip, I.; Tidman, M.; Saheba, N.; Bennett, H.; Wick, B.; Rouse, N.; Patriche, D.; Radfar, A. Public health burden of sleep disorders: Underreported problem. J. Public Health 2017, 25, 243–248. [Google Scholar] [CrossRef]
- Hillman, D.R.; Lack, L.C. Public health implications of sleep loss: The community burden. Med. J. Aust. 2013, 199, S7–S10. [Google Scholar] [CrossRef]
- Buysse, D.J.; Ancoli-Israel, S.; Edinger, J.D.; Lichstein, K.L.; Morin, C.M. Recommendations for a standard research assessment of insomnia. Sleep 2006, 29, 1155–1173. [Google Scholar] [CrossRef] [PubMed]
- Arnardottir, E.S.; Islind, A.S.; Óskarsdóttir, M.; Ólafsdóttir, K.A.; August, E.; Jónasdóttir, L.; Hrubos-Strøm, H.; Saavedra, J.M.; Grote, L.; Hedner, J.; et al. The Sleep Revolution project: The concept and objectives. J. Sleep Res. 2022, 31, e13630. [Google Scholar] [CrossRef] [PubMed]
- Guo, H.; Wei, M.; Ding, W. Changes in cognitive function in patients with primary insomnia. Shanghai Arch. Psychiatry 2017, 29, 137. [Google Scholar] [PubMed]
- Leng, Y.; McEvoy, C.T.; Allen, I.E.; Yaffe, K. Association of sleep-disordered breathing with cognitive function and risk of cognitive impairment: A systematic review and meta-analysis. JAMA Neurol. 2017, 74, 1237–1245. [Google Scholar] [CrossRef]
- Caporale, M.; Palmeri, R.; Corallo, F.; Muscarà, N.; Romeo, L.; Bramanti, A.; Marino, S.; Lo Buono, V. Cognitive impairment in obstructive sleep apnea syndrome: A descriptive review. Sleep Breath. 2021, 25, 29–40. [Google Scholar] [CrossRef] [PubMed]
- Palagini, L.; Bastien, C.H.; Marazziti, D.; Ellis, J.G.; Riemann, D. The key role of insomnia and sleep loss in the dysregulation of multiple systems involved in mood disorders: A proposed model. J. Sleep Res. 2019, 28, e12841. [Google Scholar] [CrossRef]
- Tai, X.Y.; Chen, C.; Manohar, S.; Husain, M. Impact of sleep duration on executive function and brain structure. Commun. Biol. 2022, 5, 201. [Google Scholar] [CrossRef] [PubMed]
- Thurman, S.M.; Wasylyshyn, N.; Roy, H.; Lieberman, G.; Garcia, J.O.; Asturias, A.; Okafor, G.N.; Elliott, J.C.; Giesbrecht, B.; Grafton, S.T.; et al. Individual differences in compliance and agreement for sleep logs and wrist actigraphy: A longitudinal study of naturalistic sleep in healthy adults. PLoS ONE 2018, 13, e0191883. [Google Scholar] [CrossRef] [PubMed]
- Irwin, M.R.; Olmstead, R.; Carroll, J.E. Sleep disturbance, sleep duration, and inflammation: A systematic review and meta-analysis of cohort studies and experimental sleep deprivation. Biol. Psychiatry 2016, 80, 40–52. [Google Scholar] [CrossRef]
- Deering, S.; Pratap, A.; Suver, C.; Borelli Jr, A.J.; Amdur, A.; Headapohl, W.; Stepnowsky, C.J. Real-world longitudinal data collected from the SleepHealth mobile app study. Sci. Data 2020, 7, 418. [Google Scholar] [CrossRef] [PubMed]
- Arnardottir, E.S.; Islind, A.S.; Óskarsdóttir, M. The future of sleep measurements: A review and perspective. Sleep Med. Clin. 2021, 16, 447–464. [Google Scholar] [CrossRef] [PubMed]
- Silva, G.E.; Goodwin, J.L.; Sherrill, D.L.; Arnold, J.L.; Bootzin, R.R.; Smith, T.; Walsleben, J.A.; Baldwin, C.M.; Quan, S.F. Relationship between reported and measured sleep times: The sleep heart health study (SHHS). J. Clin. Sleep Med. 2007, 3, 622–630. [Google Scholar] [CrossRef] [PubMed]
- Yap, Y.; Slavish, D.C.; Taylor, D.J.; Bei, B.; Wiley, J.F. Bi-directional relations between stress and self-reported and actigraphy-assessed sleep: A daily intensive longitudinal study. Sleep 2020, 43, zsz250. [Google Scholar] [CrossRef] [PubMed]
- Sadeh, A. The role and validity of actigraphy in sleep medicine: An update. Sleep Med. Rev. 2011, 15, 259–267. [Google Scholar] [CrossRef]
- Oakes, D.J.; Pearce, H.A.; Roberts, C.; Gehrman, P.G.; Lewis, C.; Jones, I.; Lewis, K.J. Associations between comorbid anxiety and sleep disturbance in people with bipolar disorder: Findings from actigraphy and subjective sleep measures. J. Affect. Disord. 2022, 309, 165–171. [Google Scholar] [CrossRef]
- Carney, C.E.; Buysse, D.J.; Ancoli-Israel, S.; Edinger, J.D.; Krystal, A.D.; Lichstein, K.L.; Morin, C.M. The consensus sleep diary: Standardizing prospective sleep self-monitoring. Sleep 2012, 35, 287–302. [Google Scholar] [CrossRef]
- Riemann, D.; Baglioni, C.; Bassetti, C.; Bjorvatn, B.; Dolenc Groselj, L.; Ellis, J.G.; Espie, C.A.; Garcia-Borreguero, D.; Gjerstad, M.; Gonçalves, M.; et al. European guideline for the diagnosis and treatment of insomnia. J. Sleep Res. 2017, 26, 675–700. [Google Scholar] [CrossRef] [PubMed]
- Vallo Hult, H.; Islind, A.S.; Rydenman, K.; Hällsjö Wekell, P. Decreased memory bias via a mobile application: A symptom tracker to monitor children’s periodic fever. In Challenges of Trustable AI and Added-Value on Health; IOS Press: Amsterdam, The Netherlands, 2022; pp. 915–919. [Google Scholar]
- Schmitz, L.; Sveinbjarnarson, B.F.; Gunnarsson, G.N.; Davíðsson, Ó.A.; Davíðsson, Þ.B.; Arnardottir, E.S.; Óskarsdóttir, M.; Islind, A.S. Towards a Digital Sleep Diary Standard. In Proceedings of the Americas Conference on Information Systems, Minneapolis, MN, USA, 10–14 August 2022. [Google Scholar]
- Cancela, J.; Charlafti, I.; Colloud, S.; Wu, C. Digital health in the era of personalized healthcare: Opportunities and challenges for bringing research and patient care to a new level. Digit. Health 2021, 7–31. [Google Scholar] [CrossRef]
- Golinelli, D.; Boetto, E.; Carullo, G.; Nuzzolese, A.G.; Landini, M.P.; Fantini, M.P. Adoption of digital technologies in health care during the COVID-19 pandemic: Systematic review of early scientific literature. J. Med. Internet Res. 2020, 22, e22280. [Google Scholar] [CrossRef] [PubMed]
- Islind, A.S.; Lindroth, T.; Lundin, J.; Steineck, G. Co-designing a digital platform with boundary objects: Bringing together heterogeneous users in healthcare. Health Technol. 2019, 9, 425–438. [Google Scholar] [CrossRef]
- Germine, L.; Reinecke, K.; Chaytor, N.S. Digital neuropsychology: Challenges and opportunities at the intersection of science and software. Clin. Neuropsychol. 2019, 33, 271–286. [Google Scholar] [CrossRef] [PubMed]
- Dietch, J.R.; Taylor, D.J. Evaluation of the Consensus Sleep Diary in a community sample: Comparison with single-channel electroencephalography, actigraphy, and retrospective questionnaire. J. Clin. Sleep Med. 2021, 17, 1389–1399. [Google Scholar] [CrossRef] [PubMed]
- Ibáñez, V.; Silva, J.; Cauli, O. A survey on sleep questionnaires and diaries. Sleep Med. 2018, 42, 90–96. [Google Scholar] [CrossRef]
- Short, M.A.; Arora, T.; Gradisar, M.; Taheri, S.; Carskadon, M.A. How many sleep diary entries are needed to reliably estimate adolescent sleep? Sleep 2017, 40, zsx006. [Google Scholar] [CrossRef]
- De Alcantara Borba, D.; Reis, R.S.; de Melo Lima, P.H.T.; Facundo, L.A.; Narciso, F.V.; Silva, A.; de Mello, M.T. How many days are needed for a reliable assessment by the Sleep Diary? Sleep Sci. 2020, 13, 49. [Google Scholar]
- Islind, A.S. Co-Design as a Driver of Change. Perform. Paradig. 2022, 17, 166–180. [Google Scholar]
- Islind, A.S.; Lundh Snis, U. From co-design to co-care: Designing a collaborative practice in care. Syst. Signs Actions 2018, 11, 1–24. [Google Scholar]
- Punjabi, N.M.; Brown, T.; Aurora, R.N.; Patel, S.R.; Stosor, V.; Cho, J.H.J.; Helgadóttir, H.; Ágústsson, J.S.; D’Souza, G.; Margolick, J.B. Methods for home-based self-applied polysomnography: The Multicenter AIDS Cohort Study. Sleep Adv. 2022, 3, zpac011. [Google Scholar] [CrossRef] [PubMed]
- Kainulainen, S.; Korkalainen, H.; SigurÐardóttir, S.; Myllymaa, S.; Serwatko, M.; SigurÐardóttir, S.Þ.; Clausen, M.; Leppänen, T.; Arnardóttir, E.S. Comparison of EEG signal characteristics between polysomnography and self applied somnography setup in a pediatric cohort. IEEE Access 2021, 9, 110916–110926. [Google Scholar] [CrossRef]
- Punjabi, N.M.; Patil, S.; Crainiceanu, C.; Aurora, R.N. Variability and misclassification of sleep apnea severity based on multi-night testing. Chest 2020, 158, 365–373. [Google Scholar] [CrossRef] [PubMed]
- Berry, R.B.; Brooks, R.; Gamaldo, C.E.; Harding, S.M.; Marcus, C.; Vaughn, B.V. The AASM Manual for the Scoring of Sleep and Associated Events. Rules, Terminology and Technical Specifications; American Academy of Sleep Medicine: Darien, IL, USA, 2012; Volume 176, p. 2012. [Google Scholar]
- Bastien, C.H.; Vallières, A.; Morin, C.M. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med. 2001, 2, 297–307. [Google Scholar] [CrossRef] [PubMed]
- Hagell, P.; BROMAN, J.E. Measurement properties and hierarchical item structure of the Epworth Sleepiness Scale in Parkinson’s disease. J. Sleep Res. 2007, 16, 102–109. [Google Scholar] [CrossRef] [PubMed]
- Lovibond, P.F.; Lovibond, S.H. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav. Res. Ther. 1995, 33, 335–343. [Google Scholar] [CrossRef] [PubMed]
- Freeman, D.; Sheaves, B.; Waite, F.; Harvey, A.G.; Harrison, P.J. Sleep disturbance and psychiatric disorders. Lancet Psychiatry 2020, 7, 628–637. [Google Scholar] [CrossRef]
- Konjarski, M.; Murray, G.; Lee, V.V.; Jackson, M.L. Reciprocal relationships between daily sleep and mood: A systematic review of naturalistic prospective studies. Sleep Med. Rev. 2018, 42, 47–58. [Google Scholar] [CrossRef]
- Wohlgemuth, W.K.; Edinger, J.D.; Fins, A.I.; Sullivan, R.J. How many nights are enough? The short-term stability of sleep parameters in elderly insomniacs and normal sleepers. Psychophysiology 1999, 36, 233–244. [Google Scholar] [CrossRef]
Variable | Total (N = 45) M ± SD 1/% |
---|---|
Demographics and lifestyle | |
Age, M (SD) | 50.3 ± 11.7 |
Sex, Female | 55.6 |
Education, high | 66.7 |
Work status, employed | 66.7 |
Marital status, married/cohabitating | 91.1 |
Weekly exercise, M (SD) | 2.8 ± 2.00 |
Sleep measures | |
AHI (≥15 events per hour) 2 | 26.2 |
ISI score (≥15) 3 | 31.1 |
ESS score (>10) 4 | 42.2 |
Mental health (DASS-21 5) | |
Depression score (≥10) | 20.5 |
Anxiety score (≥8) | 13.6 |
Stress score (≥15) | 20.5 |
Morning Diary Entries | Evening Diary Entries | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Variables | R2 | Un-Standardized Coefficients | Standardized Coefficients | t | p | R2 | Un-Standardized Coefficients | Standardized Coefficients | t | p | ||
B | SE | Beta (β) | B | SE | Beta (β) | |||||||
Demographics | ||||||||||||
Age | 0.087 | 0.594 | 0.298 | 0.294 | 1.995 | 0.053 | 0.097 | 0.630 | 0.297 | 0.311 | 2.118 | 0.040 |
Sex, female | 0.016 | 5.570 | 6.645 | 0.127 | 0.838 | 0.497 | 0.041 | 8.930 | 6.594 | 0.202 | 1.354 | 0.183 |
Marital status, cohabitating | 0.000 | 1.640 | 11.695 | 0.021 | 5.263 | 0.889 | 0.000 | 1.220 | 11.756 | 0.016 | 0.104 | 0.918 |
Education, higher | 0.023 | −7.033 | 6.980 | −0.152 | −1.008 | 0.319 | 0.020 | −6.633 | 7.025 | −0.143 | −0.944 | 0.350 |
Working | 0.007 | −3.767 | 7.038 | −081 | −0.535 | 0.595 | 0.008 | −4.067 | 7.071 | −0.087 | −0.575 | 0.568 |
Weekly exercises | 0.008 | 4.499 | 7.851 | 0.090 | 0.573 | 0.570 | 0.016 | −1.399 | 1.679 | −0.126 | −0.833 | 0.409 |
Sleep Measures | ||||||||||||
AHI (≥15) 1 | 0.011 | −1.175 | 1.675 | −0.106 | −0.702 | 0.487 | 0.003 | 2.924 | 7.953 | 0.058 | 0.368 | 0.715 |
ISI (≥15) 2 | 0.006 | −3.776 | 7.167 | −0.080 | −0.527 | 0.601 | 0.024 | −7.316 | 7.141 | −0.154 | −1.024 | 0.311 |
ESS (>10) 3 | 0.009 | 4.223 | 6.709 | 0.096 | 0.629 | 0.532 | 0.002 | 2.176 | 6.766 | 0.49 | 0.322 | 0.749 |
Mental Health (DASS-21 4) | ||||||||||||
Depression (≥10) | 0.031 | −9.492 | 8.228 | −0.175 | −1.154 | 0.255 | 0.059 | −13.175 | 8.132 | −0.242 | −1.620 | 0.113 |
Anxiety (≥8) | 0.007 | −5.333 | 9.789 | −0.084 | −0.545 | 0.589 | 0.022 | −9.368 | 9.446 | −0.147 | −0.961 | 0.342 |
Stress (≥15) | 0.006 | 4.337 | 8.331 | 0.080 | 0.521 | 0.605 | 0.000 | 0.095 | 8.382 | 0.002 | 0.011 | 0.991 |
Weekday Compliance (%) (n = 2892) | Weekend Compliance (%) (n = 1158) | p-Value | |
---|---|---|---|
Morning diary | 64.4 | 66.3 | 0.25 |
Evening diary | 56.8 | 55.7 | 0.52 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kristbergsdottir, H.; Schmitz, L.; Arnardottir, E.S.; Islind, A.S. Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary. Diagnostics 2023, 13, 2883. https://doi.org/10.3390/diagnostics13182883
Kristbergsdottir H, Schmitz L, Arnardottir ES, Islind AS. Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary. Diagnostics. 2023; 13(18):2883. https://doi.org/10.3390/diagnostics13182883
Chicago/Turabian StyleKristbergsdottir, Hlín, Lisa Schmitz, Erna Sif Arnardottir, and Anna Sigridur Islind. 2023. "Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary" Diagnostics 13, no. 18: 2883. https://doi.org/10.3390/diagnostics13182883
APA StyleKristbergsdottir, H., Schmitz, L., Arnardottir, E. S., & Islind, A. S. (2023). Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary. Diagnostics, 13(18), 2883. https://doi.org/10.3390/diagnostics13182883