Implementing Technologies: Assessment of Telemedicine Experiments in the Paris Region: Reasons for Success or Failure of the Evaluations and of the Deployment of the Projects
Abstract
:1. Introduction
2. Materials and Methods
2.1. Quantitative Analysis: Interventions, Study Protocols, Endpoints
2.2. Implementation of the Evaluation and Deployment of the Projects
- Have you experienced technical difficulties which may affect the quality of care delivered by the telemedicine service?
- Have you experienced difficulties in your collaboration with other professional groups in relation to the telemedicine service?
- Have you experienced difficulties in your collaboration with the staff at other institutions in relation to the telemedicine service?
- How would you describe the usability of the telemedicine application for you?
- Has the use of the telemedicine application had any effect on your use of time?
- Has the use of the telemedicine application had any effects on your tasks?
- Has the use of the telemedicine application had any effects on the communication within your institution?
- Has the use of the telemedicine application had effects on the communication with other institutions?
- Would you like to continue to use the telemedicine service?
- How would you describe your overall satisfaction with the use of the telemedicine service?
3. Results
3.1. Quantitative Analysis
3.2. Implementation of the Evaluation and Deployment of the Projects
4. Discussion
4.1. Concerning the Implementation of TLM
- (a)
- Technical difficulties in the deployment of TLM
- (b)
- Stakeholder involvement
- (c)
- The deployment of telemedicine
4.2. Concerning the Research Protocols
- (d)
- Evidentiary requirements for resource use
- (e)
- Evidence for policy decisions
- (f)
- Dealing with time constraints
5. Recommendations
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Available online: https://clinicaltrials.gov/ct2/results?cond=&term=telemedicinen.d (accessed on 31 January 2023).
- Telemedicine[Title]—Search Results—PubMed (nih.gov). Available online: https://pubmed.ncbi.nlm.nih.gov/?term=telemedicine%5BTitle%5D&sort=date (accessed on 31 January 2023).
- Al Quran, H.A.; Khader, Y.S.; Ellauzi, Z.M.; Shdaifat, A. Effect of real-time teledermatology on diagnosis, treatment and clinical improvement. J. Telemed. Telecare 2015, 21, 93–99. [Google Scholar] [CrossRef] [PubMed]
- Blozik, E.; Wildeisen, I.E.; Fueglistaler, P.; von Overbeck, J. Telemedicine can help to ensure that patients receive timely medical care. J. Telemed. Telecare 2012, 18, 119–121. [Google Scholar] [CrossRef] [PubMed]
- Driessen, J.; Bonhomme, A.; Chang, W.; Nace, D.A.; Kavalieratos, D.; Perera, S.; Handler, S.M. Nursing Home Provider Perceptions of Telemedicine for Reducing Potentially Avoidable Hospitalizations. J. Am. Med. Dir. Assoc. 2016, 17, 519–524. [Google Scholar] [CrossRef] [PubMed]
- Totten, A.M.; Womack, D.M.; Eden, K.B.; McDonagh, M.S.; Griffin, J.C.; Grusing, S.; Hersh, W.R. Telehealth: Mapping the Evidence for Patient Outcomes from Systematic Reviews; Agency for Healthcare Research and Quality (US): Rockville, MD, USA, 2016.
- McFarland, S.; Coufopolous, A.; Lycett, D. The effect of telehealth versus usual care for home-care patients with long-term conditions: A systematic review, meta-analysis and qualitative synthesis. J. Telemed. Telecare 2021, 27, 69–87. [Google Scholar] [CrossRef] [PubMed]
- Al-Durra, M.; Nolan, R.P.; Seto, E.; Cafazzo, J.A.; Eysenbach, G. Nonpublication Rates and Characteristics of Registered Randomized Clinical Trials in Digital Health: Cross-Sectional Analysis. J. Med. Internet Res. 2018, 20, e1192. [Google Scholar] [CrossRef] [PubMed]
- Paleari, L.; Malini, V.; Paoli, G.; Scillieri, S.; Bighin, C.; Blobel, B.; Giacomini, M. EU-Funded Telemedicine Projects—Assessment of, and Lessons Learned From, in the Light of the SARS-CoV-2 Pandemic. Front. Med. 2022, 9, 849998. [Google Scholar] [CrossRef] [PubMed]
- Zulfiqar, A.A.; Hajjam, A.; Andrès, E. Focus on the Different Projects of Telemedicine Centered on the Elderly in France. Curr. Aging Sci. 2019, 11, 202–215. [Google Scholar] [CrossRef] [PubMed]
- Moitry, M.; Zarca, K.; Granier, M.; Aubelle, M.S.; Charrier, N.; Vacherot, B.; Caputo, G.; Mimouni, M.; Jarreau, P.H.; Durand-Zaleski, I. Effectiveness and efficiency of tele-expertise for improving access to retinopathy screening among 351 neonates in a secondary care center: An observational, controlled before-after study. PLoS ONE 2018, 13, e0206375. [Google Scholar] [CrossRef] [PubMed]
- Zarca, K.; Charrier, N.; Mahé, E.; Guibal, F.; Carton, B.; Moreau, F.; Durand-Zaleski, I. Tele-expertise for diagnosis of skin lesions is cost-effective in a prison setting: A retrospective cohort study of 450 patients. PLoS ONE 2018, 13, e0204545. [Google Scholar] [CrossRef] [PubMed]
- Charrier, N.; Zarca, K.; Durand-Zaleski, I.; Calinaud, C.; the ARS Ile de France telemedicine group. Efficacy and cost effectiveness of telemedicine for improving access to care in the Paris region: Study protocols for eight trials. BMC Health Serv. Res. 2016, 16, 45. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Greenhalgh, T.; Russell, J. Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Med. 2010, 7, e1000360. [Google Scholar] [CrossRef] [PubMed]
- Stratton, S. Quasi-Experimental Design (Pre-Test and Post-Test Studies) in Prehospital and Disaster Research. Prehospital Disaster Med. 2019, 34, 573–574. [Google Scholar] [CrossRef] [PubMed]
- Sanson-Fisher, R.W.; D’Este, C.A.; Carey, M.L.; Noble, N.; Paul, C.L. Evaluation of systems-oriented public health interventions: Alternative research designs. Annu. Rev. Public Health 2014, 35, 9–27. [Google Scholar] [CrossRef] [PubMed]
- Kidholm, K.; Ekeland, A.G.; Jensen, L.K.; Rasmussen, J.; Pedersen, C.D.; Bowes, A.; Flottorp, S.A.; Bech, M. A model for assessment of telemedicine applications: MAST. Int. J. Technol. Assess. Health Care. 2012, 28, 44–51. [Google Scholar] [CrossRef] [PubMed]
- Bower, P.; Cartwright, M.; Hirani, S.P.; Barlow, J.; Hendy, J.; Knapp, M.; Henderson, C.; Rogers, A.; Sanders, C.; Bardsley, M.; et al. A comprehensive evaluation of the impact of telemonitoring in patients with long-term conditions and social care needs: Protocol for the whole systems demonstrator cluster randomised trial. BMC Health Serv. Res. 2011, 11, 184. [Google Scholar] [CrossRef] [PubMed]
- OSRM—Open Source Routing Machine. 2015. Available online: http://project-osrm.org/ (accessed on 13 October 2017).
- Wagner, A.K.; Soumerai, S.B.; Zhang, F.; Ross-Degnan, D. Segmented Regression Analysis of Interrupted Time Series Studies in Medication Use Research. J. Clin. Pharm. Ther. 2002, 27, 299–309. [Google Scholar] [CrossRef] [PubMed]
- Craig, P.; Dieppe, P.; Macintyre, S.; Michie, S.; Nazareth, I.; Petticrew, M. Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ 2008, 337, a1655. [Google Scholar] [CrossRef] [PubMed]
- Economides, M.; Bolton, H.; Male, R.; Cavanagh, K. Feasibility and Preliminary Efficacy of Web-Based and Mobile Interventions for Common Mental Health Problems in Working Adults: Multi-Arm Randomized Pilot Trial. JMIR Form. Res. 2022, 6, e34032. [Google Scholar] [CrossRef] [PubMed]
- Koehler, F.; Koehler, K.; Deckwart, O.; Prescher, S.; Wegscheider, K.; Kirwan, B.-A.; Winkler, S.; Vettorazzi, E.; Bruch, L.; Oeff, M.; et al. Efficacy of telemedical interventional management in patients with heart failure (TIM-HF2): A randomised, controlled, parallel-group, unmasked trial. Lancet 2018, 392, 1047–1057. [Google Scholar] [CrossRef] [PubMed]
Project and Objectives | Population and Trial Registration | Design | Intervention | Comparator | Outcomes |
---|---|---|---|---|---|
Teleconsultations: The elderly population is often exposed to unnecessary hospitalizations, which can cause deterioration in their health status. | Dependent polymorbid nursing home residents (TLM-TMG91- NCT02164747 and TLM-E-VLINE-NCT02157740) without on-site access to primary or secondary care. TMG: TeleMedicine Geriatrics; 91 is the department code E-VELINE is a play on words with the department’s name, Yvelines | Controlled before after. We compared two periods with a segmented regression analysis: before telemedicine (January 2013–December 2013 for TLM-TMG91 and January 2013–December 2014 for TLM-E-VLINE) and during routine use of telemedicine period (January 2015–December 2015 for TMG91 and January 2016-December 2016 for TLM-E-VLINE). | Programmed teleconsultations, either open to all medical specialties or with a psychiatrist or emergency teleconsultation with a doctor on call. 20 nursing homes. | Same number of nursing homes without telemedicine matched with propensity scores computed from variables identified by the Regional Health Agency and geriatricians: the number of private and public hospitals located within 20 min from the nursing home—computed using Open Source Routing Machine (OSRM)—the proportion of residents over 90 years old, the average level of dependence of residents, a global indicator of health care, the mean number of transportations and consultations. | Number of unplanned hospitalizations by nursing home and by month. Data was extracted from the national claims database, and aggregated at the nursing home level |
Teleconsultations: Because of their severe disabilities and medical shortage, access to specialized consultations and preventive follow-up is limited. | Autistic children and adolescents (TLM-PROMETTED-NCT02996708) PROMETTED: (PROgramme MEdical de Télépsychiatrie) | Controlled before after (January 2014–December 2014) and after during routine use of TLM (January 2017–December 2017) comparison. | Programmed teleconsultation with a pediatric neurologist or a psychiatrist in five institutions benefiting from telemedicine | Three structures with no access to telemedicine | Proportion of patients who had taken the Autism Diagnostic Interview-Revised (ADI-R), a structured interview and rating score, at least once during the study, and at least one reassessment per year of the Childhood Autism Rating Scale (CARS) which rates items indicative of autism spectrum disorder after direct observation, and Vineland Adaptive Behavior scale, a measure of adaptive behavior skills for children and adolescents. |
children and adolescents with multiple handicaps (TLM-POLYHANDICAP) living in institutions | Before after in eight institutions treating children and adolescents with multiple handicaps. before telemedicine (January 2015–December 2015) and during routine use of TLM period (January 2017–December 2017). | In these institutions, the patients could have access to teleconsultation and/or consultation. All those who received at least one teleconsultation were included in the intervention group | The control group consisted of patients who had only received consultations, without benefiting from telemedicine. | The primary endpoint was the average number of neuropediatric visits per child. | |
Tele-expertise. Neonatology services need access to expert professionals in brain imaging, whether in the context of ongoing care or when a second opinion is required. Some decisions in neonatal resuscitation cannot be made without an expert in ethics and medico-legal aspects. | Newborns hospitalized in a neonatal intensive care unit with severe brain disorders (TLM-MATRIX NEONAT). MATIX: MAgneTic Resonance teleXpertise | Before after. Before telemedicine (December 2014–September 2015) and during routine use of telemedicine (June 2016–December 2016). | MRI image transfer for a second opinion from a pediatric neuroradiologist. The requests for expertise from six neonatal intensive care units were analyzed. | Before telemedicine | The primary endpoint was the time between the date of MRI and the date of decision; the decision could be a withdrawal of resuscitation, continuation of care, or the request for another MRI. |
Second opinion from a pathologist specialist | Frozen section and images (TLM-Pathology Expertise-NCT02374697) | Before after. Before telemedicine (September 2013–December 2013) and during routine use of telemedicine (January 2015–June 2015). | Transfer of digital slides uploaded to a webserver for a second opinion from specialized pathologists for complex pathological diagnoses. The requests for expertise from 16 pathology units were analyzed. | Before telemedicine | The primary endpoint was the average response time to receive the results, i.e., the time between the dispatch of the digital slides to the reception of the report giving the second opinion of the remote pathologist, |
Medical tele-assistance for intraoperative consultations during a surgical procedure; in some community hospitals, no on-site pathologist is available. Offsite analysis of images of intraoperative frozen sections had the goal of obtaining the same diagnostic accuracy as the original glass slide interpretation. The objective was to verify that the implementation of telemedicine did not extend delays for the result from the expected time of 30 min. | (TLM-Pathology Frozen Section –NCT02368769). | Before after. Before telemedicine (January 2013–June 2013) and during routine use of telemedicine (January 2015–June 2015). | Intraoperative frozen sections from a two-site academic department of pathology were analyzed. | Before telemedicine | The primary endpoint was the period of time between the specimen’s time of arrival to the pathological anatomy and cytology unit and the time of result’s transmission to the surgeon. |
Projects | Primary Endpoints | Design | With TLM | Without TLM | p Value | ||||
---|---|---|---|---|---|---|---|---|---|
N | Cost/Patient | Effectiveness | N | Cost/Patient | Effectiveness | ||||
TLM-TMG91 | Number of unplanned hospitalizations by nursing home and by month. | Controled before after | Control: 366 | €1579 | See Figure 1 | Control: 292 | 331 | See Figure 1 | 0.39 |
Intervention: 397 | Intervention: 341 | ||||||||
TLM-E-VLINE | Number of unplanned hospitalizations by nursing home and by month. | Controled before after | Control: 976 | €1489 | See Figure 1 | Control: 1145 | 238 | See Figure 1 | 0.45 |
Intervention: 1087 | Intervention: 1232 | ||||||||
TLM-POLYHANDICAP | Number of neuropediatric visits per child | Case control | 31 | 984 | 1.3 (Before period) | 27 | 372 | 1.3 (Before period) | 0.99 (Before period) |
2.0 (After period) | 2.0 (After period) | 0.93 (After period) | |||||||
TLM-MATRIX NEONAT | Time between the date of MRI and the date of decision | Case control | 40 | 377 | 4.4 days | 30 | 220 | 5.9 days | 0.35 |
TLM-Pathology Expertise | Response time to the requesting pathologist | Before after | 100 | 357 | 6.9 days | 134 | 88 | 24.9 days | p < 0.001 |
TLM-Pathology Frozen Sections | Proportion of time less than 30 min between the time of arrival of the specimen in the pathological anatomy and cytology unit and the time of transmission of the result to the surgeon | Before after | 98 | 181 | 0.48 | 89 | 148 | 0.81 | 0.33 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Le Bras, A.; Zarca, K.; Mimouni, M.; Durand-Zaleski, I.; ARS Ile de France Telemedicine Group. Implementing Technologies: Assessment of Telemedicine Experiments in the Paris Region: Reasons for Success or Failure of the Evaluations and of the Deployment of the Projects. Int. J. Environ. Res. Public Health 2023, 20, 3031. https://doi.org/10.3390/ijerph20043031
Le Bras A, Zarca K, Mimouni M, Durand-Zaleski I, ARS Ile de France Telemedicine Group. Implementing Technologies: Assessment of Telemedicine Experiments in the Paris Region: Reasons for Success or Failure of the Evaluations and of the Deployment of the Projects. International Journal of Environmental Research and Public Health. 2023; 20(4):3031. https://doi.org/10.3390/ijerph20043031
Chicago/Turabian StyleLe Bras, Alicia, Kevin Zarca, Maroua Mimouni, Isabelle Durand-Zaleski, and ARS Ile de France Telemedicine Group. 2023. "Implementing Technologies: Assessment of Telemedicine Experiments in the Paris Region: Reasons for Success or Failure of the Evaluations and of the Deployment of the Projects" International Journal of Environmental Research and Public Health 20, no. 4: 3031. https://doi.org/10.3390/ijerph20043031
APA StyleLe Bras, A., Zarca, K., Mimouni, M., Durand-Zaleski, I., & ARS Ile de France Telemedicine Group. (2023). Implementing Technologies: Assessment of Telemedicine Experiments in the Paris Region: Reasons for Success or Failure of the Evaluations and of the Deployment of the Projects. International Journal of Environmental Research and Public Health, 20(4), 3031. https://doi.org/10.3390/ijerph20043031