Next Article in Journal
Unmet Needs in the Assessment of Right Ventricular Function for Severe Tricuspid Regurgitation
Next Article in Special Issue
Prediction of the Sleep Apnea Severity Using 2D-Convolutional Neural Networks and Respiratory Effort Signals
Previous Article in Journal
How Does Anxiety and Depression Affect the Outcome after Periradicular Infiltration Therapy?—A Retrospective Analysis of Patients Undergoing CT-Guided Single-Level Nerve Root Infiltration Due to Chronic Monoradicular Pain
Previous Article in Special Issue
Heart Rate Variability as a Surrogate Marker of Severe Chronic Coronary Syndrome in Patients with Obstructive Sleep Apnea
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary

by
Hlín Kristbergsdottir
1,2,*,†,
Lisa Schmitz
2,3,*,†,
Erna Sif Arnardottir
2 and
Anna Sigridur Islind
2,3
1
Department of Psychology, Reykjavik University, 102 Reykjavík, Iceland
2
Reykjavik University Sleep Institute, 102 Reykjavík, Iceland
3
Department of Computer Science, Reykjavik University, 102 Reykjavík, Iceland
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Diagnostics 2023, 13(18), 2883; https://doi.org/10.3390/diagnostics13182883
Submission received: 22 June 2023 / Revised: 31 August 2023 / Accepted: 6 September 2023 / Published: 8 September 2023
(This article belongs to the Special Issue Moving beyond Current Diagnosis of Sleep-Disordered Breathing)

Abstract

:
Sleep diaries are the gold standard for subjective assessment of sleep variables in clinical practice. Digitization of sleep diaries is needed, as paper versions are prone to human error, memory bias, and difficulties monitoring compliance. Methods: 45 healthy eligible participants (Mage = 50.3 years, range 23–74, 56% female) were asked to use a sleep diary mobile app for 90 consecutive days. Univariate and bivariate analysis was used for group comparison and linear regression for analyzing reporting trends and compliance over time. Results: Overall compliance was high in the first two study months but tended to decrease over time (p < 0.001). Morning and evening diary entries were highly correlated (r = 0.932, p < 0.001) and participants significantly answered on average 4.1 days (95% CI [1.7, 6.6]) more often in the morning (M = 60.2, sd = 22.1) than evening ((M = 56.1, sd = 22.2), p < 0.001). Conclusion: Using a daily diary assessment in a longitudinal sleep study with a sleep diary delivered through a mobile application was feasible, and compliance in this study was satisfactory.

1. Introduction

Sleep is one of the fundamental pillars of good health. It impacts a range of physiological functions, such as the hormonal system, cardiovascular system, immune system and brain health [1]. Sleep disorders cause a major socioeconomic burden on healthcare systems worldwide, e.g., using the current diagnostic criteria, almost 1 billion people between the ages of 30 and 69 are affected by obstructive sleep apnea (OSA), the most common type of sleep-disordered breathing [2]. This manifests in billions of USD annually in lost productivity, cost of accidents, and other downstream health sequelae [3]. There is profound evidence that sleep disorders are associated with a range of physical health problems, such as diabetes, cardiovascular disease, obesity, and hypertension, that lead to increased mortality and morbidity rates [2,4,5,6]. Individuals with untreated sleep disorders are also at higher risk of deteriorating cognitive functioning that may impact occupational performance and social participation, thus compromising the quality of life and individuals’ socioeconomic status [7,8,9]. Sleep disorders are strongly associated with cognitive decline, attention loss, memory impairment, issues with mood regulation, and mood disorders [8,10,11]. Therefore, sleep disorders are one of the leading health challenges today and a serious threat to public health if left undiagnosed and untreated.
Our understanding of sleep disorders and possible underlying mechanisms is challenging. We still face several limitations in sleep research, i.e., expensive and time-consuming diagnostic procedures, small and homogeneous sample sizes, lack of long-term studies on sleep disorders, and daily fluctuations in sleep [7,12,13]. Hence, while the impact of short-term and acute sleep deprivation has been well-documented, our understanding of long-term sleep disorders with daily fluctuations in a naturalistic setting (i.e., home environment) needs to be clarified [7,14,15].
Although polysomnography (PSG) is considered the gold standard for sleep measurements, it is time-consuming and expensive. Moreover, it does not include subjective feedback from the patient, which can be pivotal for aspects like sleep hygiene and insomnia diagnosis. Additionally, PSGs require manual scoring, and the queues for studies tend to be lengthy and the overall resources are scarce. Continuous measurement for long-term studies of sleep in a naturalistic setting would be revolutionary [16]. Furthermore, a combination of subjective and objective measurements would be beneficial as both provide their own unique sleep features [17,18]. When it comes to subjective sleep measurements, sleep diaries and self-report measurements are more compatible with long-term and continuous measurements, where the long-term effect and compliance over time are studied [16]. Sleep questionnaires and sleep diaries outline the gold standard for subjective self-reported assessment of sleep variables in clinical practice. Sleep diaries are traditionally delivered on paper and have shown a high correlation with actigraphy measures on sleep onset and offset times, making them an essential support tool for diagnosing and treating sleep hygiene and insomnia [19,20]. However, although sleep diaries in pen-and-paper format are the conventional method for self-report sleep assessment, they are increasingly considered outdated and are prone to memory bias and difficulties in monitoring compliance [6,21,22,23]. Despite the rapid increase in digitization across different sectors, the consumer adoption of mobile apps and wearables, and their feasibility for delivering digital health solutions, there remain significant challenges in the design and development of digital health solutions for sleep, in particular in digital sleep diaries [16,24].
The adoption of digital health solutions and the rapid change towards more personalized and participatory healthcare is promising [25,26]. It enables more intensive and larger data collection than is possible with traditional data collection in sleep research. However, digitization of healthcare in general and for sleep, in particular, has been lagging significantly [7], although it is evident that digital health solutions outline an important factor for the future [26]. With the development of digital health solutions, there is great potential for longitudinal subjective assessment of sleep health, particularly the ability to monitor in real time and follow up continuously with subjects between more extensive examinations. Healthcare providers and researchers can be updated about key indicators of clinically significant changes such as the subjects’ sleep and lifestyle habits, cognitive functioning, mood, and physical activity [25,27]. The benefits of digital health solutions include that they are cost-beneficial, improve accessibility, enable at-home monitoring for both clinical populations and the general population, and may provide new information in clinical evaluation and diagnostics that are currently not being captured [7,13,25,28]. For example, it is possible to measure symptoms more frequently and for longer periods by using digital health solutions for momentary assessments and, thus, to capture new domains of sleep that traditional in-laboratory assessments do not. A fundamental shift towards this new horizon of digital sleep health is imperative.
A mobile sleep diary has several benefits, including detailed real-time data on night-to-night variability in sleep and sleep habits [16,29]. A mobile sleep diary offers the possibility of reporting daily sleep measures close to the reported event, increasing data accuracy while minimizing the risk of recall bias [30]. Digital sleep diaries enable both patients and healthcare providers to evaluate their sleep measures and follow their progress in real time through informative data visualizations. However, the main concern and possible limitation of using sleep diaries (either on paper or digital), especially when used for a long consecutive period, is compliance [16,30]. Few studies have examined the usage of a digital sleep diary for prolonged periods (>7 nights) [13,31,32], and we found only one study, Thurman et al. [13], that investigated compliance specifically over a 16-week period. In that study, compliance decreased linearly over time.
The primary aim of this study was to examine the feasibility and provide a data-driven analysis of a longitudinal repeated mobile sleep diary assessment. Our mobile app includes a sleep diary which was co-designed and developed as a part of the Sleep Revolution mobile application (app), based on the extended paper-and-pen Consensus Sleep Diary [21,24]. We examined the compliance in terms of reporting in the app for 90 consecutive days without any active intervention for a group of healthy sleepers and subjects with reported sleep problems. We hypothesized that compliance would decrease significantly over time according to a linear function. As a secondary exploratory aim, we examined individual differences in compliance across different types of sleep measures and demographic groups. We hypothesized that individuals with sleep apnea or insomnia were more likely to show higher levels of compliance and that compliance would not vary among different demographic groups. As a third exploratory aim, we examined if compliance would differ between weekdays and weekends, i.e., we hypothesized that individuals would be less compliant during weekends than on weekdays.

2. Materials and Methods

2.1. Study Participants

Fifty-nine participants were recruited by word of mouth and local advertisement in Iceland (within Reykjavik University, newspapers, radio, etc.) for this three-month pilot study. Inclusion criteria were that participants had to be of age 18 or older, own a smartphone and be willing to download the Sleep Revolution app, and not shift workers. Participants were asked to (i) complete an online questionnaire on background information and self-report sleep and lifestyle measures, (ii) complete a three-night self-applied polysomnography, and (iii) complete a sleep diary twice a day. A total of 45 participants (mean age = 50.3 years, range 23–74, 25 females, 20 Males) completed all three tasks and were included in the analysis. Those not included dropped out of the study at the beginning and did not complete the questionnaires or the three-night self-applied polysomnography. Participants were not paid or compensated in any form for their study participation.

2.2. Design and Procedure

Prior to participation in the main sleep study, participants completed questionnaires on demographics, lifestyle, and self-reported sleep measures as well as screening lists. Participants were then subjected to a three-night self-applied polysomnography and asked to keep a daily sleep diary in the Sleep Revolution app. Automated reminders were sent to participants to enhance compliance on a regular basis. Participants were only allowed to answer on the current day to ensure they reported their real-time experience. Participants signed an informed consent prior to their participation and the study was approved by the Icelandic National Bioethics Committee (ref. no. 21-070).

2.3. Measures

2.3.1. Sleep Diary

The app was co-designed and developed through extensive user engagement within the Sleep Revolution project. The co-design approach outlines a novel way of approaching the design, as we have elicited the needs of the participants iteratively throughout the design process [33,34]. The design matured through multiple iterations and eventually, the app was ready for inclusion of participants. The results of this study outline the first cohort using the app in a longitudinal manner. In this study, the participants were instructed to complete the sleep diary twice a day for 90 consecutive days in an app developed by the Sleep Revolution [7,24]. The sleep diary in the mobile app is adapted from the Consensus Sleep Diary [21] and collects a subjective evaluation of sleep measures. In addition, the Sleep Revolution sleep diary also gathers information on lifestyle factors that might impact their sleep, i.e., mood and stress levels, number of naps, number of caffeine and alcohol units consumed, and duration of exercise conducted that day.

2.3.2. Three-Night Self-Applied Polysomnography

A self-applied polysomnography [35,36,37] was used at baseline over three consecutive nights and the apnea–hypopnea index (AHI) was derived. AHI was defined as the number of respiratory events (apneas or hypopneas) per hour of sleep. For adult subjects, AHI is traditionally classified as normal (AHI < 5/h), mild (AHI between 5 and 14.9 events per hour), moderate (AHI between 15 and 29.9/h), and severe (AHI ≥ 30/h). Due to this study’s relatively small sample size, an AHI score of ≥15 was considered the threshold criteria for having OSA for covariate analysis [14].
The recordings were annotated by expert sleep technologists in accordance with the AASM manual, version 2.6 [38]. To do this, the Noxturnal version 6.2.2 software (Nox Medical, Reykjavik, Iceland) was used. The N3 scoring rules were updated for the self-applied setup in accordance with Kainulainen et al. [36] and the recommended AASM rules for hypopnea scoring were used.

2.3.3. Insomnia Severity Index (ISI)

The ISI is a 7-item self-report instrument designed to assess the severity of both nighttime and daytime components of insomnia. The recall period typically consists of the most recent month. The tool evaluates the severity of sleep onset, sleep maintenance, and early morning awakening problems, sleep dissatisfaction, interference of sleep difficulties with daytime functioning, noticeability of sleep problems by others, and distress caused by the sleep difficulties. Each of these items is rated on a 5-point Likert scale (e.g., 0 = no problem; 4 = very severe problem), with a total score ranging from 0 to 28. The interpretation for the total score is as follows: absence of insomnia (0–7); sub-threshold insomnia (8–14); moderate insomnia (15–21); and severe insomnia (22–28). In this study, a cut-off of ≥15 was used to indicate a significant risk of insomnia. The ISI is available in three versions—a version each for patients, clinicians, and significant others—but this paper focuses on the patient version only [39].

2.3.4. Epworth Sleepiness Scale (ESS)

The ESS is an 8-item self-report instrument that assesses the likelihood of dozing or falling asleep in common situations of daily living. A total score is calculated and ranges from 0 to 24, where higher scores indicate a higher likelihood of dozing or falling asleep and scores of >10 indicate a significant risk of excessive sleepiness [40].

2.3.5. Depression Anxiety and Stress Scale 21 (DASS-21)

The DASS-21 is a 21-item self-report instrument designed to measure the negative emotional states of depression, anxiety, and stress. Each subscale includes 14 items with the following cut-off scores as positive indicators for symptoms of depression (≥10), anxiety (≥8), and stress (≥15) [41].

2.3.6. Covariates

Based on previous research, the following covariates that may relate to sleep and compliance were included in the analysis: age (years), sex (female/male), education (high/low), employment status (working/not working), weekly exercise (count), marital status of married or cohabitating (yes/no).

2.4. Statistical Analytics

Statistics were calculated using IBM SPSS Statistics 28 and Python. Background data are presented descriptively, categorical data by frequencies (n) and proportions (%) and continuous data with means and standard deviations (S.D.s). The perceived total sleep time refers to the answer to the question “In total, how long did you sleep?”. Participants had the option to have this value automatically calculated from the answers to the questions “What time did you try to go to sleep? ”, “How long did it take you to fall asleep?”, “In total, how long did these awakenings last?”, and “What time was your final awakening?”. The result of the calculated value is referred to as the estimated total sleep time. To compare the mean count of morning and evening diary entries, a paired sample T-test was applied and multiple linear regression was used to examine individual differences in morning and evening diary entries by demographics, sleep measures, and relevant lifestyle factors. To account for multiple comparisons, the Bonferroni correction test was used and the significance level was adjusted (p-value of 0.05/2 = 0.025). Simple linear regression was used to explore linear trends in morning and evening diary entries over time. The resulting linear regression estimate (ß) can be interpreted as the average percentage point change in compliance per day.

3. Results

3.1. Descriptive Statistics

Table 1 shows the descriptive sociodemographic profile of participants in the study.

3.2. Overall Compliance

Overall compliance was high over the study period. Participants answered a total of 2711 times out of the 4049 possible morning diary entries (67%) and 2525 times out of the 4050 possible evening diary entries (62%). Simple linear regression revealed a significant linear trend of compliance tending to decrease over time for both mornings (b = −0.675, t = −8.581, p < 0.001, R2 = 0.456) and evenings (b = −0.745, t = −10.464, p < 0.001, R2 = 0.554).
Compliance was high at the beginning of the study and stayed considerably stable through the first two months of the study. At the beginning of the study, compliance was 98% for the morning diary and 78% for the evening diary. In the first month, average compliance was high, at 80% (morning) and 67% (evening), and then slightly dropped between the first and second months, to 70% (morning) and 60% (evening). During the last month, compliance started to decline and dropped to 31% (morning) and 26% (evening) in the last days of the study (see Figure 1). Figure 2 and Figure 3 provide a visual presentation of compliance by each participant for the morning (see Figure 2) and evening (see Figure 3) sleep diaries.
Morning and evening diary entries were highly correlated (r = 0.93, p < 0.001), and results from paired sample t-test show a significant average difference (t = 3.38, df = 44, p < 0.002). On average, participants answered 4.1 (95% CI [1.7, 6.6]) days more often in the morning (M = 60.2, sd = 22.1) than in the evening (M = 56.1, sd = 22.2).

3.2.1. Compliance by Individual Differences

We examined compliance for the morning and evening diaries by individual differences with simple linear regression, i.e., demographic profile, sleep measures, and mental health. For demographics, objective and subjective sleep measures, and symptoms of depression, anxiety, and stress, a significant mean difference in compliance was not found (see Table 2).

3.2.2. Compliance on Weekdays and Weekend Days

We compared the overall compliance for weekdays and weekend days for the morning and evening diaries. Table 3 shows the average compliance rate for all participants during the study period consisting of 2892 weekdays and 1158 weekend days. The compliance rates show no significant difference between weekdays and weekends for the morning and evening diaries.

3.3. Total Sleep Time

The morning diary collects various data points regarding the participants’ sleep the previous night. We evaluated the gathered data and found discrepancies regarding bedtimes and the corresponding total sleep time. The implementation of the application allowed participants to enter what time they went to sleep and when they woke up in an am/pm format. A button to estimate the total sleep time based on the previously entered data was provided to the participants in the app view for entering the total sleep time.
We examined the deviation of the participants’ perceived total sleep time deviation from the estimated total sleep time. The data distribution is symmetrical around the mean, with most perceived total sleep time entries showing no deviation from the estimated total sleep time. However, a smaller second bell curve can be found around the −12 h deviation mark. A possible explanation for this observation could be that incorrect time entries cause this deviation due to the am/pm format they were entered in.
Therefore, we also examined the distribution after correcting for the outliers. We corrected the data by adding 12 h to those deviation data points that show a negative correction greater than 8 h and subtracting 12 h from those that show a positive correction greater than 8 h. This eliminates the majority of outliers and results in a bell curve with fewer outliers, as can be seen in Figure 4.
The data (see Figure 4) show that over 90% of the total sleep time entries did not deviate more than an hour from the estimated duration.

4. Discussion

The current study examined the feasibility of using a sleep diary in an app for 90 consecutive days and individual differences in sustaining compliance. To our knowledge, a study of this magnitude has not been performed previously, and our findings provide a novel addition to the existing literature on digital measures in sleep research.
Our main findings showed high compliance rates, especially for the first two months of the study (98–62%), and that the digital sleep diary was accessible to all, regardless of demographic profile, possible sleep problems, or symptoms of depression, anxiety, and stress. However, compliance did significantly decline over time, as expected. At the beginning of the study, approximately 98% of study participants completed their morning sleep diary entries, and 78% completed their evening diary entries. By the end of the study, 52–46% of participants remained compliant. For the last days of the study, compliance rates dropped to 26–31%, indicating participants experienced a higher respondent burden. The decrease in compliance could be attributed to participants’ fatigue or a lack of motivation to continue reporting over an extended period. Another explanation could be that participants were asked to attend a final interview at the end of the study and were able to choose a date that fit them well. Some participants chose a date before they had finished their 90 days and likely stopped using the app after the interview. This is a limitation to the study and the interpretation of compliance, as only 50% of the total sample used the app for the 90-day study period. As there was no special endpoint or treatment being monitored in the study, this could have affected compliance. Nevertheless, we find the results interesting and novel despite this limitation as, to our knowledge, findings on compliance in a digital sleep diary for a three-month study period have not been reported.
Despite the immense knowledge of short-term and acute sleep problems and their effects on physical and mental health [2,42], previously, studies have not focused on daily fluctuations in sleep in a naturalistic setting for prolonged periods of time [13,15], e.g., in a recent systematic review of the relationship between daily sleep and mood, results show that the majority of studies lend support to a short-term dynamic association between sleep and daily mood, but studies of the long-term effects are few and still remain to be fully investigated [43].
Studies using a digital sleep diary have mostly reported findings on compliance for up to 14 days [30]. One study conducted by Thurman et al. [13] that examined compliance in using a digital sleep diary for 16 consecutive weeks showed compliance significantly decreased with time, consistent with our findings. While our sleep diary was designed with the goal of fostering engagement through notification reminders and data visualizations, additional measures should be taken to ensure high compliance for longer time periods. The app version that was used during the study period lacked actionable feedback, but another study in the Sleep Revolution is currently ongoing and investigates participant compliance and habit change through the use of goals as part of the gamification of the digital sleep diary. Studies have shown that the reliability of data collected by sleep diaries increases with the number of days used and is therefore important to support participants’ motivation in answering the sleep diary for prolonged periods [31,32,44].
Our findings also revealed a high correlation of compliance between morning and evening diary entries, indicating consistency in participants’ reporting. However, participants answered the morning diary more frequently than the evening diary, despite the presence of notification reminders in the app for both morning and evening diaries. This disparity may be attributed to the circumstance that the morning reminders were received at a more convenient time. It is conceivable that participants were more likely to be at home in the morning and, thus, had easier access to the diary, while in the evening, they might have been occupied by various activities. To our knowledge, such findings of a correlation between compliance in the morning and evening entries of a digital sleep diary have not been reported elsewhere.
Interestingly our findings also revealed that compliance on weekdays and weekends showed no significant differences for either morning or evening diaries, suggesting that participants maintained consistent reporting habits regardless of the day of the week. The expectation was that compliance would decrease during the weekend, as people’s schedules tend to be more irregular on those days than they are during weekdays. A possible explanation for the consistent compliance is the notification reminders that helped people to remember to fill in the diary during weekdays and weekends alike. Our findings suggest that a digital sleep diary supports data reliability. Previous studies have mainly focused on the accuracy of data collection between weekdays and weekends, suggesting a need for a minimum of 7–10 days of entries in order to obtain reliable data, especially on weekends [31,32]. However, findings on the difference in compliance during weekdays and weekends have not been reported elsewhere to our knowledge.
Lastly, our findings revealed discrepancies between participants’ perceived total sleep time and the estimated total sleep time. The distribution of the deviation showed a symmetrical curve, indicating that most participants accurately estimated their total sleep time. However, a smaller portion of participants showed significant deviations, likely due to errors in entering sleep and wake times or confusion related to the am/pm format, which is generally not used in Iceland. Correcting for these outliers resulted in a more normalized distribution, suggesting that providing clearer instructions or formats for time entry could improve accuracy in estimating total sleep time.

Strengths and Limitations

The study has several strengths, including addressing gaps in the literature on digital solutions for sleep research. To our knowledge, this is the first study that investigates participant compliance with a sleep diary app in an intensive longitudinal setting. The results highlight the potential of digital health solutions in improving longitudinal sleep assessment and indicate that a digital sleep diary can foster compliance and, thus, overcome the limitations and drawbacks associated with the traditional pen-and-paper format [6,21,22,23]. The study emphasizes that the integration of longitudinal subjective measurements in digital health solutions opens up new possibilities for capturing comprehensive sleep information and contributing to meaningful changes in daily habits, mood, and overall well-being [8,10,11,16].
Furthermore, the study has addressed the lack of long-term studies in naturalistic settings [7,12,13,16] by evaluating digital solutions. By providing more accessible and cost-effective methods of sleep assessment, digital health solutions have the potential to mitigate the socioeconomic burden of sleep disorders on healthcare systems and improve overall public health outcomes [2,3,4,5].
Overall, the findings from this study provide valuable insights into compliance and sleep patterns among participants using a morning and evening sleep diary app. However, there are limitations to consider. First, the sample size in this study was relatively small, and our findings need to be interpreted with caution, as generalizability in such samples is limited. Future studies with larger and more diverse participant samples could provide a more comprehensive understanding of compliance and sleep duration patterns. Second, compliance in the study dramatically decreased in the last days of the study period, which could impact the reliability of the data. The fact that some participants attended a final interview before they had finished the study might explain the dramatic drop in compliance in the last 10 days of the study. In future studies within an intensive longitudinal setting, it might be beneficial to provide more frequent reminders and positive feedback to participants to promote their participation and promote the quality of data received. In addition, a final interview should occur only after participants have finished the study to ensure compliance throughout the study period. Lastly, the accuracy and quality of data collected in a sleep diary mobile application need further testing. In our study, a small portion of participants showed significant deviations when entering sleep and wake times. This is likely due to the format being am/pm, but further testing on the reliability of the data being collected overall in a mobile application would be beneficial.

5. Conclusions

By providing more accessible and cost-effective methods of sleep assessment, digital health solutions have the potential to mitigate the socioeconomic burden of sleep disorders on healthcare systems and improve overall public health outcomes [2,3,4,5]. Thus, the findings from this study provide valuable insights into compliance and sleep duration patterns among participants using a morning and evening sleep diary app.
In conclusion, this study has demonstrated the potential of digital sleep diaries in revolutionizing sleep assessment and management. The integration of subjective measurements in an app holds promise for enhancing diagnostic procedures, designing targeted interventions, and ultimately improving the quality of life for individuals affected by sleep disorders. Ergo, using daily diary assessment in a longitudinal sleep study with a sleep diary delivered through an app is feasible, and compliance in this study showed to be satisfactory. Moreover, this paper contributes a compliance assessment of the novel sleep diary delivered through a mobile application. The results show that the app holds high potential both for the future of data collection in sleep medicine and clinical assessment.

Author Contributions

Conceptualization, A.S.I. and E.S.A.; methodology, A.S.I. and E.S.A.; software design, L.S. and A.S.I.; validation, H.K.; formal analysis, H.K. and L.S.; investigation, H.K. and L.S.; resources, H.K. and L.S.; data curation, H.K. and L.S.; writing—original draft preparation, H.K. and L.S.; writing—review and editing, A.S.I. and E.S.A.; visualization, H.K. and L.S.; supervision, A.S.I. and E.S.A.; project administration, A.S.I. and E.S.A.; funding acquisition, A.S.I. and E.S.A. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the European Union’s Horizon 2020 Research and Innovation Programme under Grant 965417.

Institutional Review Board Statement

This study was granted ethical approval by the National Bioethics Committee of Iceland (under the reference number VSN-21-070) and each participant of the study has signed an informed consent.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patient(s) to publish this paper.

Data Availability Statement

The data analyzed for the purpose of this paper are not publicly available due to privacy and ethical restrictions.

Acknowledgments

This work is part of the Sleep Revolution project, supported by the European Union’s Horizon 2020 Research and Innovation Programme under Grant 965417. We would like to thank the participants in the study wholeheartedly for their invaluable contributions to science.

Conflicts of Interest

Erna S. Arnardottir discloses lecture fees from Nox Medical, Philips, ResMed, Vistor, Apnimed, Jazz Pharmaceuticals, Linde Healthcare, Alcoa Fjardaral, and Wink Sleep. Erna S. Arnardottir is also a member of the Philips Sleep Medicine and Innovation Medical Advisory Board.

References

  1. Walker, M.P. Sleep essentialism. Brain 2021, 144, 697–699. [Google Scholar] [CrossRef]
  2. Benjafield, A.V.; Ayas, N.T.; Eastwood, P.R.; Heinzer, R.; Ip, M.S.; Morrell, M.J.; Nunez, C.M.; Patel, S.R.; Penzel, T.; Pépin, J.L.; et al. Estimation of the global prevalence and burden of obstructive sleep apnoea: A literature-based analysis. Lancet Respir. Med. 2019, 7, 687–698. [Google Scholar] [CrossRef] [PubMed]
  3. O’Reilly, J. Costs of Treatment Non-adherence in Obstructive Sleep Apnoea. In CPAP Adherence: Factors and Perspectives; Springer: Berlin/Heidelberg, Germany, 2022; pp. 125–140. [Google Scholar]
  4. Filip, I.; Tidman, M.; Saheba, N.; Bennett, H.; Wick, B.; Rouse, N.; Patriche, D.; Radfar, A. Public health burden of sleep disorders: Underreported problem. J. Public Health 2017, 25, 243–248. [Google Scholar] [CrossRef]
  5. Hillman, D.R.; Lack, L.C. Public health implications of sleep loss: The community burden. Med. J. Aust. 2013, 199, S7–S10. [Google Scholar] [CrossRef]
  6. Buysse, D.J.; Ancoli-Israel, S.; Edinger, J.D.; Lichstein, K.L.; Morin, C.M. Recommendations for a standard research assessment of insomnia. Sleep 2006, 29, 1155–1173. [Google Scholar] [CrossRef] [PubMed]
  7. Arnardottir, E.S.; Islind, A.S.; Óskarsdóttir, M.; Ólafsdóttir, K.A.; August, E.; Jónasdóttir, L.; Hrubos-Strøm, H.; Saavedra, J.M.; Grote, L.; Hedner, J.; et al. The Sleep Revolution project: The concept and objectives. J. Sleep Res. 2022, 31, e13630. [Google Scholar] [CrossRef] [PubMed]
  8. Guo, H.; Wei, M.; Ding, W. Changes in cognitive function in patients with primary insomnia. Shanghai Arch. Psychiatry 2017, 29, 137. [Google Scholar] [PubMed]
  9. Leng, Y.; McEvoy, C.T.; Allen, I.E.; Yaffe, K. Association of sleep-disordered breathing with cognitive function and risk of cognitive impairment: A systematic review and meta-analysis. JAMA Neurol. 2017, 74, 1237–1245. [Google Scholar] [CrossRef]
  10. Caporale, M.; Palmeri, R.; Corallo, F.; Muscarà, N.; Romeo, L.; Bramanti, A.; Marino, S.; Lo Buono, V. Cognitive impairment in obstructive sleep apnea syndrome: A descriptive review. Sleep Breath. 2021, 25, 29–40. [Google Scholar] [CrossRef] [PubMed]
  11. Palagini, L.; Bastien, C.H.; Marazziti, D.; Ellis, J.G.; Riemann, D. The key role of insomnia and sleep loss in the dysregulation of multiple systems involved in mood disorders: A proposed model. J. Sleep Res. 2019, 28, e12841. [Google Scholar] [CrossRef]
  12. Tai, X.Y.; Chen, C.; Manohar, S.; Husain, M. Impact of sleep duration on executive function and brain structure. Commun. Biol. 2022, 5, 201. [Google Scholar] [CrossRef] [PubMed]
  13. Thurman, S.M.; Wasylyshyn, N.; Roy, H.; Lieberman, G.; Garcia, J.O.; Asturias, A.; Okafor, G.N.; Elliott, J.C.; Giesbrecht, B.; Grafton, S.T.; et al. Individual differences in compliance and agreement for sleep logs and wrist actigraphy: A longitudinal study of naturalistic sleep in healthy adults. PLoS ONE 2018, 13, e0191883. [Google Scholar] [CrossRef] [PubMed]
  14. Irwin, M.R.; Olmstead, R.; Carroll, J.E. Sleep disturbance, sleep duration, and inflammation: A systematic review and meta-analysis of cohort studies and experimental sleep deprivation. Biol. Psychiatry 2016, 80, 40–52. [Google Scholar] [CrossRef]
  15. Deering, S.; Pratap, A.; Suver, C.; Borelli Jr, A.J.; Amdur, A.; Headapohl, W.; Stepnowsky, C.J. Real-world longitudinal data collected from the SleepHealth mobile app study. Sci. Data 2020, 7, 418. [Google Scholar] [CrossRef] [PubMed]
  16. Arnardottir, E.S.; Islind, A.S.; Óskarsdóttir, M. The future of sleep measurements: A review and perspective. Sleep Med. Clin. 2021, 16, 447–464. [Google Scholar] [CrossRef] [PubMed]
  17. Silva, G.E.; Goodwin, J.L.; Sherrill, D.L.; Arnold, J.L.; Bootzin, R.R.; Smith, T.; Walsleben, J.A.; Baldwin, C.M.; Quan, S.F. Relationship between reported and measured sleep times: The sleep heart health study (SHHS). J. Clin. Sleep Med. 2007, 3, 622–630. [Google Scholar] [CrossRef] [PubMed]
  18. Yap, Y.; Slavish, D.C.; Taylor, D.J.; Bei, B.; Wiley, J.F. Bi-directional relations between stress and self-reported and actigraphy-assessed sleep: A daily intensive longitudinal study. Sleep 2020, 43, zsz250. [Google Scholar] [CrossRef] [PubMed]
  19. Sadeh, A. The role and validity of actigraphy in sleep medicine: An update. Sleep Med. Rev. 2011, 15, 259–267. [Google Scholar] [CrossRef]
  20. Oakes, D.J.; Pearce, H.A.; Roberts, C.; Gehrman, P.G.; Lewis, C.; Jones, I.; Lewis, K.J. Associations between comorbid anxiety and sleep disturbance in people with bipolar disorder: Findings from actigraphy and subjective sleep measures. J. Affect. Disord. 2022, 309, 165–171. [Google Scholar] [CrossRef]
  21. Carney, C.E.; Buysse, D.J.; Ancoli-Israel, S.; Edinger, J.D.; Krystal, A.D.; Lichstein, K.L.; Morin, C.M. The consensus sleep diary: Standardizing prospective sleep self-monitoring. Sleep 2012, 35, 287–302. [Google Scholar] [CrossRef]
  22. Riemann, D.; Baglioni, C.; Bassetti, C.; Bjorvatn, B.; Dolenc Groselj, L.; Ellis, J.G.; Espie, C.A.; Garcia-Borreguero, D.; Gjerstad, M.; Gonçalves, M.; et al. European guideline for the diagnosis and treatment of insomnia. J. Sleep Res. 2017, 26, 675–700. [Google Scholar] [CrossRef] [PubMed]
  23. Vallo Hult, H.; Islind, A.S.; Rydenman, K.; Hällsjö Wekell, P. Decreased memory bias via a mobile application: A symptom tracker to monitor children’s periodic fever. In Challenges of Trustable AI and Added-Value on Health; IOS Press: Amsterdam, The Netherlands, 2022; pp. 915–919. [Google Scholar]
  24. Schmitz, L.; Sveinbjarnarson, B.F.; Gunnarsson, G.N.; Davíðsson, Ó.A.; Davíðsson, Þ.B.; Arnardottir, E.S.; Óskarsdóttir, M.; Islind, A.S. Towards a Digital Sleep Diary Standard. In Proceedings of the Americas Conference on Information Systems, Minneapolis, MN, USA, 10–14 August 2022. [Google Scholar]
  25. Cancela, J.; Charlafti, I.; Colloud, S.; Wu, C. Digital health in the era of personalized healthcare: Opportunities and challenges for bringing research and patient care to a new level. Digit. Health 2021, 7–31. [Google Scholar] [CrossRef]
  26. Golinelli, D.; Boetto, E.; Carullo, G.; Nuzzolese, A.G.; Landini, M.P.; Fantini, M.P. Adoption of digital technologies in health care during the COVID-19 pandemic: Systematic review of early scientific literature. J. Med. Internet Res. 2020, 22, e22280. [Google Scholar] [CrossRef] [PubMed]
  27. Islind, A.S.; Lindroth, T.; Lundin, J.; Steineck, G. Co-designing a digital platform with boundary objects: Bringing together heterogeneous users in healthcare. Health Technol. 2019, 9, 425–438. [Google Scholar] [CrossRef]
  28. Germine, L.; Reinecke, K.; Chaytor, N.S. Digital neuropsychology: Challenges and opportunities at the intersection of science and software. Clin. Neuropsychol. 2019, 33, 271–286. [Google Scholar] [CrossRef] [PubMed]
  29. Dietch, J.R.; Taylor, D.J. Evaluation of the Consensus Sleep Diary in a community sample: Comparison with single-channel electroencephalography, actigraphy, and retrospective questionnaire. J. Clin. Sleep Med. 2021, 17, 1389–1399. [Google Scholar] [CrossRef] [PubMed]
  30. Ibáñez, V.; Silva, J.; Cauli, O. A survey on sleep questionnaires and diaries. Sleep Med. 2018, 42, 90–96. [Google Scholar] [CrossRef]
  31. Short, M.A.; Arora, T.; Gradisar, M.; Taheri, S.; Carskadon, M.A. How many sleep diary entries are needed to reliably estimate adolescent sleep? Sleep 2017, 40, zsx006. [Google Scholar] [CrossRef]
  32. De Alcantara Borba, D.; Reis, R.S.; de Melo Lima, P.H.T.; Facundo, L.A.; Narciso, F.V.; Silva, A.; de Mello, M.T. How many days are needed for a reliable assessment by the Sleep Diary? Sleep Sci. 2020, 13, 49. [Google Scholar]
  33. Islind, A.S. Co-Design as a Driver of Change. Perform. Paradig. 2022, 17, 166–180. [Google Scholar]
  34. Islind, A.S.; Lundh Snis, U. From co-design to co-care: Designing a collaborative practice in care. Syst. Signs Actions 2018, 11, 1–24. [Google Scholar]
  35. Punjabi, N.M.; Brown, T.; Aurora, R.N.; Patel, S.R.; Stosor, V.; Cho, J.H.J.; Helgadóttir, H.; Ágústsson, J.S.; D’Souza, G.; Margolick, J.B. Methods for home-based self-applied polysomnography: The Multicenter AIDS Cohort Study. Sleep Adv. 2022, 3, zpac011. [Google Scholar] [CrossRef] [PubMed]
  36. Kainulainen, S.; Korkalainen, H.; SigurÐardóttir, S.; Myllymaa, S.; Serwatko, M.; SigurÐardóttir, S.Þ.; Clausen, M.; Leppänen, T.; Arnardóttir, E.S. Comparison of EEG signal characteristics between polysomnography and self applied somnography setup in a pediatric cohort. IEEE Access 2021, 9, 110916–110926. [Google Scholar] [CrossRef]
  37. Punjabi, N.M.; Patil, S.; Crainiceanu, C.; Aurora, R.N. Variability and misclassification of sleep apnea severity based on multi-night testing. Chest 2020, 158, 365–373. [Google Scholar] [CrossRef] [PubMed]
  38. Berry, R.B.; Brooks, R.; Gamaldo, C.E.; Harding, S.M.; Marcus, C.; Vaughn, B.V. The AASM Manual for the Scoring of Sleep and Associated Events. Rules, Terminology and Technical Specifications; American Academy of Sleep Medicine: Darien, IL, USA, 2012; Volume 176, p. 2012. [Google Scholar]
  39. Bastien, C.H.; Vallières, A.; Morin, C.M. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med. 2001, 2, 297–307. [Google Scholar] [CrossRef] [PubMed]
  40. Hagell, P.; BROMAN, J.E. Measurement properties and hierarchical item structure of the Epworth Sleepiness Scale in Parkinson’s disease. J. Sleep Res. 2007, 16, 102–109. [Google Scholar] [CrossRef] [PubMed]
  41. Lovibond, P.F.; Lovibond, S.H. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav. Res. Ther. 1995, 33, 335–343. [Google Scholar] [CrossRef] [PubMed]
  42. Freeman, D.; Sheaves, B.; Waite, F.; Harvey, A.G.; Harrison, P.J. Sleep disturbance and psychiatric disorders. Lancet Psychiatry 2020, 7, 628–637. [Google Scholar] [CrossRef]
  43. Konjarski, M.; Murray, G.; Lee, V.V.; Jackson, M.L. Reciprocal relationships between daily sleep and mood: A systematic review of naturalistic prospective studies. Sleep Med. Rev. 2018, 42, 47–58. [Google Scholar] [CrossRef]
  44. Wohlgemuth, W.K.; Edinger, J.D.; Fins, A.I.; Sullivan, R.J. How many nights are enough? The short-term stability of sleep parameters in elderly insomniacs and normal sleepers. Psychophysiology 1999, 36, 233–244. [Google Scholar] [CrossRef]
Figure 1. Daily compliance for morning and evening diaries for the study period.
Figure 1. Daily compliance for morning and evening diaries for the study period.
Diagnostics 13 02883 g001
Figure 2. Visual presentation of individual compliance for the morning sleep diary. The blue bar marks the days the participants filled out the sleep diary. The red bar marks the end of the individual participant’s study period.
Figure 2. Visual presentation of individual compliance for the morning sleep diary. The blue bar marks the days the participants filled out the sleep diary. The red bar marks the end of the individual participant’s study period.
Diagnostics 13 02883 g002
Figure 3. Visual presentation of individual compliance for the evening sleep diary. The blue bar marks the days the participants filled out the sleep diary. The red bar marks the end of the individual participant’s study period.
Figure 3. Visual presentation of individual compliance for the evening sleep diary. The blue bar marks the days the participants filled out the sleep diary. The red bar marks the end of the individual participant’s study period.
Diagnostics 13 02883 g003
Figure 4. Histograms of the unaltered and corrected deviation distributions of the perceived total sleep time from the estimated total sleep time.
Figure 4. Histograms of the unaltered and corrected deviation distributions of the perceived total sleep time from the estimated total sleep time.
Diagnostics 13 02883 g004
Table 1. Measurements in morning and evening sleep diary app.
Table 1. Measurements in morning and evening sleep diary app.
VariableTotal
(N = 45)
M ± SD 1/%
Demographics and lifestyle
   Age, M (SD)50.3 ± 11.7
   Sex, Female55.6
   Education, high66.7
   Work status, employed66.7
   Marital status, married/cohabitating91.1
   Weekly exercise, M (SD)2.8 ± 2.00
Sleep measures
   AHI (≥15 events per hour) 226.2
   ISI score (≥15) 331.1
   ESS score (>10) 442.2
Mental health (DASS-21 5)
   Depression score (≥10)20.5
   Anxiety score (≥8)13.6
   Stress score (≥15)20.5
1 Mean and standard deviation, 2 Apnea–hypopnea index, 3 Insomnia Severity Index, 4 Epworth Sleepiness Scale 5 The Depression, Anxiety and Stress Scale—21 Items.
Table 2. Simple linear regression results for morning and evening diary entries.
Table 2. Simple linear regression results for morning and evening diary entries.
Morning Diary EntriesEvening Diary Entries
Variables R2 Un-Standardized Coefficients Standardized Coefficients t p R2 Un-Standardized Coefficients Standardized Coefficients t p
B SE Beta (β) B SE Beta (β)
Demographics
Age0.0870.5940.2980.2941.9950.0530.0970.6300.2970.3112.1180.040
Sex, female0.0165.5706.6450.1270.8380.4970.0418.9306.5940.2021.3540.183
Marital status, cohabitating0.0001.64011.6950.0215.2630.8890.0001.22011.7560.0160.1040.918
Education, higher0.023−7.0336.980−0.152−1.0080.3190.020−6.6337.025−0.143−0.9440.350
Working0.007−3.7677.038−081−0.5350.5950.008−4.0677.071−0.087−0.5750.568
Weekly exercises0.0084.4997.8510.0900.5730.5700.016−1.3991.679−0.126−0.8330.409
Sleep Measures
AHI (≥15) 10.011−1.1751.675−0.106−0.7020.4870.0032.9247.9530.0580.3680.715
ISI (≥15) 20.006−3.7767.167−0.080−0.5270.6010.024−7.3167.141−0.154−1.0240.311
ESS (>10) 30.0094.2236.7090.0960.6290.5320.0022.1766.7660.490.3220.749
Mental Health (DASS-21 4)
Depression (≥10)0.031−9.4928.228−0.175−1.1540.2550.059−13.1758.132−0.242−1.6200.113
Anxiety (≥8)0.007−5.3339.789−0.084−0.5450.5890.022−9.3689.446−0.147−0.9610.342
Stress (≥15)0.0064.3378.3310.0800.5210.6050.0000.0958.3820.0020.0110.991
1 Apnea–hypopnea index, 2 Insomnia severity index, 3 Epworth sleepiness scale, 4 The Depression, Anxiety and Stress Scale—21 Items.
Table 3. The average compliance rate for the morning and evening diaries on weekends and weekdays.
Table 3. The average compliance rate for the morning and evening diaries on weekends and weekdays.
Weekday
Compliance (%)
(n = 2892)
Weekend
Compliance (%)
(n = 1158)
p-Value
Morning diary64.466.30.25
Evening diary56.855.70.52
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kristbergsdottir, H.; Schmitz, L.; Arnardottir, E.S.; Islind, A.S. Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary. Diagnostics 2023, 13, 2883. https://doi.org/10.3390/diagnostics13182883

AMA Style

Kristbergsdottir H, Schmitz L, Arnardottir ES, Islind AS. Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary. Diagnostics. 2023; 13(18):2883. https://doi.org/10.3390/diagnostics13182883

Chicago/Turabian Style

Kristbergsdottir, Hlín, Lisa Schmitz, Erna Sif Arnardottir, and Anna Sigridur Islind. 2023. "Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary" Diagnostics 13, no. 18: 2883. https://doi.org/10.3390/diagnostics13182883

APA Style

Kristbergsdottir, H., Schmitz, L., Arnardottir, E. S., & Islind, A. S. (2023). Evaluating User Compliance in Mobile Health Apps: Insights from a 90-Day Study Using a Digital Sleep Diary. Diagnostics, 13(18), 2883. https://doi.org/10.3390/diagnostics13182883

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop