Improving Quality of Life in Chronic Patients: A Pilot Study on the Effectiveness of a Health Recommender System and Its Usability

Abstract

The TeNDER project aims to improve the quality of life (QoL) of chronic patients through an integrated care ecosystem. This study evaluates the health recommender system (HRS) developed for the project, which offers personalized recommendations based on data collected from a set of monitoring devices. The list of notifications covered different areas of daily life such as physical activity, nutrition, and sleep. We conducted this case study to evaluate the effectiveness and usability of the HRS in providing accurate and relevant recommendations to users. Evaluation process consisted on survey administration for QoL assessment and the satisfaction and usability of the HRS. The four-week pilot study involved several patients and caregivers and demonstrated that the HRS was perceived as user-friendly, consistent, and helpful, with a positive impact on patients’ QoL. However, the study highlights the need for improvement in terms of personalization of recommendations.

1. Introduction

Chronic diseases have become a major health problem in Europe. Around 75% of healthcare expenditure in Europe is spent on the management and treatment of chronic diseases. It is striking that almost two-thirds of health expenditure comes from elderly people with multiple chronic diseases [1,2].

The projected global population outlook estimates that the population of older people (defined as those aged 65 and over) will increase significantly, from 727 million at the beginning of 2020 to 11.5 billion in 2050 [3]. During this period, the number of people aged 75–84 in the EU is projected to increase by 56.1%, and the number of people aged 65–74 is expected to increase by 16.6%, while it is expected that by then there will be 13.5% fewer people under 55 years of age living in the EU [4].

Among the most prominent chronic age-related diseases, Alzheimer’s disease (AD), Parkinson’s disease (PD), and cardiovascular diseases (CVD) [5] are the main ones. AD is a neurodegenerative pathology that produces cognitive and behavioral changes and is reflected as dementia [6]. AD is responsible for the largest burden of disease: Alzheimer’s and related disorders affect more than 6.1 million people in Europe, and this figure is expected to double every 20 years as the population ages [7,8]. Currently, caring for people with dementia across Europe costs approximately EUR 130 billion a year [9].

PD is the second most prevalent neurodegenerative disease. Globally, the burden of PD has doubled as a result of the increase in the number of older people. The prevalence of PD is estimated to be 6.2 million people worldwide, although the European Parkinson’s Disease Association (EPDA) suggests that the figure should be higher, at around 10 million people, due to under-diagnosis [10]. Some studies suggest that this figure could double by 2040 to 12 million [11]. The estimated annual European cost of Parkinson’s disease is EUR 13.9 billion, representing EUR 1600 per patient [12].

People with CVD have one or more risk factors, such as high blood pressure, diabetes, or hyperlipidemia. Early detection of disease aggravation and prompt treatment is essential [13]. CVD represented 31% of all global deaths in 2016, and it is deemed as the leading cause of premature death (37% of all deaths under the age of 70) and disability in Europe and worldwide [14].

The complications of chronic diseases can be influenced by several factors, some of which are modifiable and preventable if strategies are promoted among the population to improve health habits [15]. Lifestyle factors can influence the onset and intensity of chronic diseases [16,17,18]. People who lead a healthy lifestyle, such as eating a balanced diet, exercising regularly, having good sleep habits, and avoiding toxic intake, have many health benefits, especially in avoiding complications of chronic diseases [19,20]. Providing patients with chronic diseases with a tool to help them manage their self-care may have important individual, clinical, and public health implications in view of the rapidly increasing trends in the prevalence of multimorbidity.

Therefore, it is crucial to find new and innovative solutions to improve their quality of life (QoL), increase autonomy, and promote self-care with the ultimate goal of reducing the burden on European healthcare systems. The TeNDER project [21] is positioned to address this issue. The project aimed to create an integrated care ecosystem for people with chronic diseases, which adapted to their needs through a multisensorial system while preserving privacy and ensuring data protection. The TeNDER project was conducted in four different European countries and involved over 1500 users, including patients, health professionals, social workers, caregivers, and other staff. The project used affect-based micro tools to match clinical and clerical patient information and provided tailored integrated care services to promote well-being and health recovery. In addition, interactive communication and social services strengthened elderly support, extending their autonomy and care supply chain.

One of the key components of the TeNDER project is the health recommender system (HRS), which plays a critical role in personalizing the care experience for patients with chronic diseases. The HRS analyzes patient data and generates personalized recommendations that match the person’s needs. These recommendations are integrated into the mobile app, providing users with updates and information. This helps improve the overall QoL for patients with chronic diseases while ensuring that they receive the care and support they need.

In this article, we provide a comprehensive overview of the HRS that was developed for the TeNDER project. We will discuss the process used to make the data useful for defining personalization, generating recommendations, and sending the information to the mobile app. We will also provide an evaluation of the HRS, highlighting its performance and potential for improvement. By doing so, we hope to contribute to the development of innovative solutions for chronic diseases and to improve the QoL for those affected by these conditions.

1.1. Literature Review

An overview of the concept of recommender systems (RS) in general topics can inform about the current state of research and guide future directions for health research. In a recent study [22], a comprehensive analysis of various RS applications and their algorithms was presented. The study established a taxonomy of components required for effective RS development and evaluated the datasets, simulation platforms, and performance metrics employed in different contributions. Furthermore, a conducted study [23] of over 1000 research papers on RS published between 2011 and the first quarter of 2017 identified research trends in the field and highlighted potential areas for future research.

As general research in RS, an investigation [24] provided an overview of the field of RS and discussed the current generation of recommendation methods. The authors identified both the limitations and advantages of RS, emphasizing that they are considered a subset of information filtering systems that suggest content to users based on their needs. RS are widely employed by popular e-commerce websites to recommend a variety of items, including news, music, research articles, books, and products.

As part of the era of artificial intelligence (AI) and big data, specialized issues have been published to explore complex data, advanced AI, and combinations with RSs. A special issue [25] aimed to complement these discussions and inspire readers to tackle existing and emerging challenges in building RSs mostly focused on AI techniques.

Focusing on health topics, HRSs have the potential to support individuals in making informed decisions about their health by providing personalized recommendations based on observed data. These systems have been developed and studied in a variety of contexts, including wearable devices, electronic health records, and other data sources. These systems offer a rich source of data for HRSs.

They have gained increasing attention as a means of supporting individuals in making informed decisions about their health and well-being [26,27,28]. Some systems used machine learning algorithms to analyze data collected from various sources, such as wearable devices and electronic health records, and provided personalized recommendations to users based on their individual characteristics and needs. Sensorial ecosystems, which consist of a network of interconnected devices that continuously monitor and collect data about an individual’s health, offer a rich source of data for HRSs [29,30].

A review of the existing literature on HRSs reveals a number of key challenges and limitations [31,32]. One of the main issues is the quality and reliability of the data used to generate recommendations. Inaccurate or incomplete data can lead to inappropriate or irrelevant recommendations, which may negatively impact user acceptance and adherence. Another challenge is the need for personalization, as different individuals have different health needs and preferences. Developing algorithms that can effectively tailor recommendations to individual characteristics and needs is a key challenge in the field. An example is the work performed [33] on the development of trustworthy recommender systems (TRS). TRSs not only prioritize accuracy but also aim to be transparent, unbiased, fair, and robust. They proposed a novel framework for the construction of TRSs.

Despite these challenges, previous research has demonstrated the potential benefits of using these systems to improve health outcomes. For example, in a specific RS work [34], they reviewed the current landscape of RS research and identified trends and future directions in this field.

A study on HRS [35], examined the potential of these systems to support and enhance computer-tailored digital health interventions, identifying their effects on efficiency, effectiveness, trustworthiness, and enjoyment of digital health programs. Another related work [36], discussed the use of artificial intelligence in the development of RS to improve prediction accuracy. The authors mentioned the specific techniques and approaches used, such as fuzzy techniques, transfer learning, and neural networks.

Sensorial ecosystems typically generate large volumes of sensitive personal data, which must be carefully protected to ensure compliance with privacy regulations and to prevent unauthorized access or misuse. Another challenge is to ensure compatibility and integration among these systems. This work [37] conducted a thorough analysis of compatibility, security and privacy issues in the context of IoT-based health care including security requirements, threat models, and attack taxonomies from the healthcare perspective. An additional study [38] regarding sensorial ecosystems discussed the use of classification methods for proactive monitoring of patients and the recommendation of personalized wearable devices. Unfortunately, they did not evaluate the effectiveness of the use of the devices. By contrast, another study [39], filled this research gap by evaluating the effectiveness and impact of a wearable fitness tracker as part of a greater healthcare ecosystem for driving user behavior from the perspective of value co-creation.

In summary, the existing literature on HRSs suggests that these systems have the potential to support individuals in managing their health and improving their overall well-being. However, further research is needed to address the challenges and limitations of these systems in real-world settings applied to patients with chronic diseases. In particular, our work applies to find evidences on the positive effects of HRS in contrast to the previously cited work [26], where the authors did not find enough evidences for the positive effects of HRSs in terms of cost-effectiveness and patient health outcomes.

1.2. Research Question and Objectives

The primary objective of this study is to investigate the effectiveness and usability of a HRS for improving the QoL of chronic patients, caregivers, and healthcare professionals. To achieve this objective, the study is designed to address the following specific research objectives:

• To study the effectiveness of the HRS in increasing adherence to self-care.
• To study the usability of the tool from the perspective of chronic patients, carers, and healthcare professionals.
• To study satisfaction with the tool from the perspective of chronic patients, carers, and healthcare professionals.

By pursuing these research objectives, we aimed to provide valuable insights into the potential benefits and challenges of implementing a HRS-based technological system in the management and treatment of chronic diseases. The findings of this study could inform the development of future interventions and strategies.

2. Materials and Methods

This section presents the research methodology applied, including the population sample and the design of the protocols. Additionally, the developed HRS is explained in the context of the TeNDER ecosystem. It includes essential information regarding the technical and clinical aspects of the study.

2.1. TeNDER Sensorial Ecosystem

The large-scale TeNDER project was piloted in four European countries with a sample of more than 1500 patients with chronic diseases. It involved piloting a sensory ecosystem designed to provide a more complete and accurate picture of patient health and well-being by capturing data from a variety of sources. The collected data were processed and analyzed using advanced techniques to provide healthcare professionals and patients with actionable information to improve their health status.

The TeNDER tool consisted of an integrated system consisting of a set of devices designed to capture data from various sources related to patient health and well-being. The devices included a sleep sensor, a wristband for fall detection and step tracking, a location sensor, and other sensors, such as binary and environmental sensors. The architecture of the TeNDER project consisted of three main levels: the Low-Level Subsystem (LLS), the High-Level Subsystem (HLS) and the TeNDER services. The LLS level was responsible for the acquisition of data from the different sensors and consisted of the multisensor capture module, which was in charge of the deployment of the sensors and their connectivity. The HLS layer processed the information extracted by the LLS layer and provided data aggregation and pre-processing. The TeNDER services layer offered services aimed at improving the QoL of patients and providing better guidance to healthcare experts on the patient’s health status. At this level was placed the HRS for extracting data and providing notifications to patients, where Figure 1 depicts the data workflow of different modules focused on feeding data to the HRS.

2.2. Description of the Study Design and Sample

In order to ensure a more personalized experience and accurate data collection, a subsample of patients was extracted from the total population to participate in the HRS pilot. This pilot took place over a four-week period, where patients received daily notifications on the mobile app. The total number of participants in the pilot was 37, including 17 patients, 10 caregivers, and 10 healthcare professionals. This allowed for a more hands-on approach to survey administration and ensured that all patients were able to provide their feedback in a consistent and reliable manner. By selecting a subsample, it was also possible to gain a more detailed understanding of the impact of the TeNDER system on a smaller group of patients.

Table 1. Number of participants and devices used in the pilot study.

Group	Number of Patients	Group	Number
Alzheimer	8	Caregivers	10
Parkinson	3	Professionals	10
Cardiovascular	6	Devices	28
Total patients	17	Total	48

summarizes all data related to the pilot, including the number of patients, caregivers, and healthcare professionals involved in the study. The study population was composed of patients with chronic diseases, considered as main diseases if PD, AD, or CVD.

The scope of the study was the health consortium partner centers: SERMAS (primary care centers of the Madrid Health Service, Madrid, Spain), APM (Parkinson’s Association of Madrid, Madrid, Spain), UNITOV (Hospital of the University of Rome Tor Vergata, Rome, Italy), SKBA (Neurological Hospital, Schoen Clinic, Prien, Germany), Spomincica (Department of Neurology, University Medical Center, Ljubljana, Slovenia).

This study made a specific selection criteria for patients, caregivers, and health professionals to be included in the pilot. Patients had to be over 60 years and able to understand the local language. They could not be unwilling to use the technologies or have a life expectancy less than 6 months. Caregivers were required to have knowledge of technology and provide support to the patient. Health professionals had to be socio-health professionals and have at least one year of professional experience. In

Table 2. Criteria for participants in the project.

Category	Inclusion Criteria	Exclusion Criteria
Patients	Over 60 years Understand local language Has a reference person or caregiver in case of dependency	Unwilling to use technology in the project Unable to follow the requirements of the study Life expectancy less than 6 months Alcoholism or drug addiction
Parkinson Disease Patients	Confirmed diagnosis	Multiple system atrophy Progressive supranuclear palsy Corticobasal syndrome Dementia with Lewy bodies
Alzheimer Disease Patients	MMSE between 19 and 28 points	GDS over 6–7
Cardiovascular Disease Patients	Heart failure with a NYHA grade II–III HF Stable coronary artery disease or artery coronary disease with or without ST elevation Atrial fibrillation, pacemaker, and stroke	Arterial coronary disease (less than 4 weeks ago) Severe aortic stenosis
Caregivers	Current care for a participant patient Ability to use a smartphone and have basic tech knowledge In case not living with the patient, provide logistical support or be a paid carer or directly employed	Do not understand the local language Do not agree to participate in the study Do not sign the informed consent form
Health Professionals	Socio-health professionals Carrying out professional activity in a PCHC participating in the study At least one-year professional experience Ability to provide continuity of care for the patient during the intervention	Do not agree to participate in the study Do not sign the informed consent form

, further details are referred to for the selection criteria. All of the participants in the study had to accept and understand their participation and sign the informed consent form.

However, it is important to note that the data collected in this study may be subject to certain biases. For example, self-report measures may be subject to recall bias, and the data collected by the sensorial ecosystem devices may be influenced by factors such as device accuracy and user adherence. These limitations should be taken into account when interpreting the results of the research.

2.3. Data Collection and Evaluation

Data collection included self-reporting measures. The self-report included data obtained through patient interviews and questionnaires. This information from patients included demographic information (gender, age, and main disease), usability and satisfaction (using a validated System Usability Scale (SUS) questionnaire [40]), and health management attitudes (using an ad hoc Likert-type scale from 1 to 5 that asks questions on self-care, information, autonomy, safety and QoL. It was scored from “1—Completely disagree” to “5—Completely agree”).

The participants were contacted by research teams via telephone or in-person to conduct follow-up interviews regarding their experience with the HRS. The patients were asked about their weekly QoL related to the system. At the end of the pilot, a post-survey was administered to assess several aspectes of the patients with the HRS. The post-survey consisted of the QoL questionnaire, SUS questionnaire, and a final questionnaire assessing the level of satisfaction with the use of the HRS. The specific questions included in each of the questionnaires and their objectives are outlined below:

• QP: QoL Proxy. The measure of effectiveness for the TeNDER system was the QoL Proxy surveys, which were used to assess the impact of the system on participants’ well-being. The QoL proxy questions included in the survey were carefully designed to capture important aspects of daily life, such as physical activity, self-care, social engagement, and emotional well-being. This measure was collected weekly to assess any changes in QoL that may have resulted from using the system and the HRS. The surveys were the following:

  -

  QP1: The TeNDER system has helped me to have information about my health. (1–5)

  -

  QP2: The TeNDER system has helped to improve my autonomy. (1–5)

  -

  QP3: The TeNDER system is a support in my daily life. (1–5)

  -

  QP4: The TeNDER system helps me to improve my self–care (food, physical activity, sleep and rest…). (1–5)

  -

  QP5: The TeNDER system helps me to feel safer and more secure. (1–5)

  -

  QP6: I believe that using the TeNDER system regularly could increase my QoL. (1–5)

Additionally, at the end of the four weeks, a final QoL assessment was performed in the context of the use of the RS, which included the following questions:

• Q: Quality of Life. Specifically tackled QoL surveys, but focused on the overall system of the project.

  -

  Q1: How would you rate your QoL today, in the context of using the TeNDER system? (0–10)

  -

  Q2: Since you have been using the TeNDER system, you believe that your QoL of life has improved? (Improved/Maintained/Worsened)

  -

  Q3: According to your experience of using the TeNDER System, how does its use influence your QoL? (Open)

  -

  Q4: Would you recommend the use of the TeNDER system to a friend or family member? (Yes/No)

  -

  Q4.2: If NO, ask why? (Open)

• SUS: System Usability Scale. The SUS surveys [40] were designed to evaluate the usability of the HRS and the overall experience of its patients. The surveys included ten questions that were answered on a Likert scale of 1 to 5, with 1 being the most negative and 5 being the most positive response. The SUS questionnaire was evaluated at the end of the four-week pilot period along with the other post-pilot interview questionnaires.

  -

  SUS1: I think that I would like to use the recommender frequently. (1–5)

  -

  SUS2: I found the recommender unnecessarily complex. (1–5)

  -

  SUS3: I thought the recommender was easy to use. (1–5)

  -

  SUS4: I think that I would need the support of a technical person to be able to use this recommender. (1–5)

  -

  SUS5: I found the various functions in the recommender were well integrated. (1–5)

  -

  SUS6: I thought there was too much inconsistency in the recommender. (1–5)

  -

  SUS7: I would imagine that most people would learn to use the recommender very quickly. (1–5)

  -

  SUS8: I found the recommender very cumbersome to use. (1–5)

  -

  SUS9: I felt very confident using the recommender. (1–5)

  -

  SUS10: I needed to learn a lot of things before I could get going with the recommender. (1–5)

• R: Recommender Satisfaction Feedback. The objective of this questionnaire was to obtain an assessment from the participants on the degree of satisfaction achieved with the use of RS, as well as to obtain key information that allowed to identify weaknesses in the design and implement improvements in subsequent developments. To this end, two types of questions were prepared, some based on a Likert scale 1–5 that allowed easy assessment of the degree of satisfaction together with other open questions where participants could freely express their feelings, opinions, and comments. The questionnaire consisted of the following questions.

  -

  R1: After your experience using the Recommender, how would you rate your satisfaction with it? (1–5)

  -

  R2: Do you find this service helpful in your daily life? (Yes/No).

  -

  R2.2: Why? (Open).

  -

  R3: What benefits have you found using the recommender? (Open)

  -

  R4: What would you improve about the recommender? (Open)

  -

  R5: What didn’t you like about the recommender? (Open)

  -

  R6: Would you recommend this service to others? (Yes/No)

  -

  R6.1: If NO, ask why. (Open)

2.4. Health Recommender System

The HRS leveraged data collected from patient questionnaires (examples in Appendix A 

Table A1. Sample list of patient questionnaires.

Area	Message
Social	How often do you feel that you lack companionship: Never (0), Hardly ever (1), some of the time (2), or often (3)? How often do you feel isolated from others? (Never, hardly ever, some of the time, or often
Sleep	How often do you have difficulty getting to sleep when you go to bed? Never (0); Some days (1); Most days (2); Every Day (3) How often do you feel rested when you wake up in the morning?
Nutritional	I have had an illness or condition that has caused me to change the type and/or amount of food I eat. Yes (1), No (0) I eat little fruit, vegetables or dairy products.Yes (1), No (0)
Physical Activity	Have you experienced an incident that has caused you to reduce your usual activity? Yes (1), No (0)

) and sensory devices to identify trigger events specific to each patient, which in turn triggered personalized recommendations. The lower-level sensory data was used to generate patient-specific recommendations. Figure 1 illustrates the detailed interconnection between modules on the TeNDER ecosystem.

The system-generated recommendations were aimed at improving various aspects of patients’ daily life, such as social interactions, sleep, nutrition, environment, emotions, physical activity, and medication adherence (refer to Appendix B 

Table A2. Sample list of patient recommender messages.

Area	Description	Message
Social	The patient feels alone	Web link to a social service explaining isolation and give possiblites to groups with other people.
	The patient needs help with house-work	There are public resources that could be useful to receive help with household chores. If you need it, consult your social worker.
Sleep	Irregular sleep time for a few days	Your sleep score was low for the last days. We recommend to go to sleep more regular.
	General recommendation	It is good to expose yourself to natural light during the day, and avoid staying indoors for long periods of time.
Nutritional	Loss of weight unintentionally	You should contact with your health professional to explain the weight loss.
	General recommendation	Meat should be prepared by removing all visible fat and choosing the leanest parts.
Environmental	Room temperature too high/low for a few days	The room temperature is detected not to be comfortable for the last days. Please check the heating system or ventilate the apartment if necessary.
	Binary sensors have detected that he forgets to close the main gate at night	Remember to check the status of the door before going to bed.
Emotional	The patient is found to be sad or apathetic most of the day	Did you know that staying active increases serotonin levels and is one of the most effective ways to fight apathy?
Physical Activity	Low activity time for several days	Your activity level was low durring the last days. We recommend you to take a walk for a half an hour.
	General recommendation	Did you know that daily walking can help improve your mood and your quality of sleep?
Others	General recommendation	To improve the results of your pharmacological treatment, keep fixed schedules for taking your medication.
	General recommendation	Remember to report any adverse effects of the medication to your referring doctor.

for some examples). These recommendations were activated based on the static observations collected from devices worn by the patients and were selected using predefined rules. The recommendations were linked to the area of recommendation based on the following connections:

• Social. Fall detection + social questionnaires.
• Sleep. Sleep tracker + sleep questionnaires.
• Nutritional. Localization sensor + nutrition questionnaires.
• Environmental. Environmental sensor (temperature and humidity).
• Emotional. No specific trigger.
• Physical activity. Wristband data (steps) + physical activity questionnaires.
• Other. Included medication reminders.

The HRS played a crucial role in supporting patient health by providing tailored recommendations based on a patient’s specific needs and evolution. To do this, it relied on the electronic health record (EHR) [41] microservice to access the unique claim key for a specific individual and their associated health records (observations). This interaction was facilitated through the use of a RESTful Application Programming Interface (REST API) developed, to exchange information and allowed for seamless communication between the HRS and the EHR. EHRs were intended to improve the efficiency and accuracy of healthcare by providing healthcare providers and researchers with immediate access to a patient’s medical information [42].

Regarding the methodology employed to set up the HRS, we developed it based on a user-centered approach which is outlined in the following steps:

• Requirement gathering: We identified the needs and requirements of the target users (chronic patients). This phase was intended for the creation of the different recommendation areas.
• Design: We designed the architecture of the HRS, and the integration with other services, including the data collection (from EHR), notification system (from RabbitMQ), and user interfaces (mobile app).
• Development: We developed the HRS using Python language [43], a high-level programming language known for its versatility and functionality. Python’s capabilities for data manipulation, exploration, and analysis were especially valuable in creating a system that was optimized for personalized recommendations.
• Testing and evaluation: We conducted several rounds of testing and evaluation to ensure the quality and integration of the HRS with other componentes of the project.

2.4.1. Mobile Questionnaires

The main purpose of including the questionnaires in the HRS was to complement the information collected by the TeNDER system through the different devices and sensors. The questionnaires helped us get a clearer picture of specific aspects of the patients participating in the pilot.

Prior to developing the questionnaires, a comprehensive analysis of the data collected by the TeNDER system was conducted to identify potential gaps and enhance the personalization and appropriateness of the recommendations sent.

Questionnaires were an important component of the project, providing valuable insights into the daily lives of our participants. These questionnaires were sent by the socio-health professionals to the patients, who then filled them out using the mobile app (workflow example in Figure 2). The questionnaires covered four key areas of focus: social, sleep, nutritional, and physical activity. These questionnaires helped to better understand the experiences and needs of our participants.

The completion of these questionnaires was vital to the success of the HRS as it helped to personalize the recommendations for each individual participant. The data collected from these questionnaires was used to define the messages to meet the specific needs and preferences of each person, resulting in a more meaningful and effective experience. A sample of the prepared questionnaires is present in Appendix A Table A1.

2.4.2. Recommendation Messages

The HRS used daily analysis of a week’s worth of patient data to identify patterns and changes over time, enabling it to provide targeted recommendations based on a patient’s evolution. For instance, if a patient had irregular bedtimes during the week, the system may suggest maintaining a more consistent schedule. Additionally, if the patient reported sleeping difficulties through questionnaires (workflow example in Figure 2), the system could offer recommendations to address this specific issue. The use of a week’s worth of data with the combination of the results on the questionnaires allowed for a more comprehensive assessment of the patient’s needs, leading to the generation of more targeted recommendations. Appendix B Table A2 includes additional messages samples of the different areas of the HRS.

A meticulous co-design process was undertaken with patients, caregivers, and professionals to determine areas of interest and potential triggers, as part of developing the recommendation messages. The messages were further assessed for their comprehensibility, utilizing language that was fully tailored to the end-users.

The RabbitMQ [44] configuration microservice was responsible for handling notifications and events for the HRS. It was integrated with the RabbitMQ broker server, which created a queue and sent notifications and events to the appropriate UI client using a unique notification identifier. This design allowed the HRS to easily access the RabbitMQ configuration to generate notifications or events. A notification example received in the mobile app is observed in Figure 3.

3. Results

In this section, we present the results of our study on the effectiveness and usability of the proposed HRS aimed at improving the QoL on users with chronic diseases. The study focused on a specific population of users who were recruited from different geographic region. The patients were monitored by the help of caregivers and healthcare professionals. The study was conducted over a period of four weeks, during which participants were asked to use the given devices and the mobile application, where they received the notifications from the HRS.

The results of QoL proxy about the study are presented in Figure 4 using heatmaps. Each heatmap represents the weekly counts of responses for a specific question. The heatmap graphic provides a visual representation of the distribution of responses over time, allowing for a quick overview of the data. Any changes in the distribution of responses can be easily identified, highlighting potential areas of response.

Results showed that the patients had positive responses towards the TeNDER system. The majority of the responses for QP1 (easy access to health information), QP3 (daily support), QP4 (improving self-care), and QP6 (QoL increasing) fell within the range of 4 (agree) to 5 (completely agree). However, there were two questions, namely QP2 (improving autonomy), and QP5 (safer and security), where the responses were more varied. These questions received lower scores than the other questions, indicating that participants may not have experienced the same level of improvement in autonomy and feelings of safety and security as they did in other areas.

At the end of the four weeks, post-interviews were conducted to gather feedback, as well as surveys to assess user satisfaction with the HRS. Figure 5 depicts the results of the QoL post-survey. In terms of evaluation of QoL for the patients, over 50% of the patients rated their QoL below 5 points, while the remaining patients were distributed across the higher scale. There was no reported decline in QoL for the patients, with mixed responses indicating either a maintained or improved QoL. Furthermore, regarding recommending the HRS to other friends or family members, most patients expressed a positive willingness to recommend the system to a friend or family member.

The SUS surveys conducted after the pilot showed that the HRS was appropiate in terms of usability. The first set of surveys is presented in Figure 6. The system was well received by the patients, with SUS1 indicating a stable trend and positive responses. The system was not considered complex by the patients, as shown by the mostly low values (1 and 2) in SUS2. SUS3 and SUS5 showed that the system was intuitive and well-integrated as most responses were positive and high. The results for SUS4 (need for technical support) were mixed, indicating that some patients required technical support while others did not.

The results of the second set are shown in Figure 7. Based on SUS6, they did not show any noticeable inconsistency in the system. SUS7 (learn to use quickly) had mixed results, which implied that some patients had difficulty learning to use the system quickly. SUS8 (too cumbersome) had more responses in the lower range (1 and 2), indicating that some patients found it somewhat complicated to use. SUS9 (confident using the tool) had mixed results, suggesting that some patients felt confident while others did not. Lastly, SUS10 (learn a lot before using the tool) showed results in the middle of the table, suggesting that patients did not need to learn a lot before using the tool.

The results of the recommender satisfaction and feedback surveys were generally positive, as reported in Figure 8. The satisfaction rate was mixed on the positive side, with no negative responses. Around 70% of the patients found the service helpful in their daily lives and stated that they would recommend the service to others. In terms of feedback, the responses were open-ended and provided valuable insights for improving the HRS on future work.

4. Discussion

The study findings offer valuable insights into the effectiveness and influence of a HRS on patients with chronic diseases. While the weekly administration of the TeNDER system surveys yielded positive results, there was no evidence of improvement over the course of the pilot.

The majority of patients rated the HRS as user-friendly and easy to use, with positive responses mostly observed for high values. Patients also reported that the system’s functionalities were integrated and consistent and did not require further technical support.

They were generally satisfied, with a majority finding the service helpful in their daily life and willing to recommend it to others. However, there is still some feedback to be considered, especially regarding the personalization of recommendations and the possibility of including deeper and more complex ones.

Finally, the QoL survey showed that the HRS had a positive impact on the patients’ QoL, with no patient reporting that their QoL had worsened due to the system. The majority of patients reported either maintained or improved QoL.

5. Conclusions

The pilot study demonstrated that the HRS had a positive impact on patients’ QoL and increased adherence to self-care among chronic patients. The study also found that the HRS was highly usable for different participant groups, including caregivers and healthcare professionals. While the study revealed that the system was generally perceived as consistent and helpful, there is still room for improvement in terms of personalization of recommendations and the inclusion of more complex and deeper recommendations.

To further advance the understanding of HRSs, future research should aim to explore the potential benefits and limitations of different types of recommendations and the real-time monitoring of patient vitals via wristband data. It is also crucial to expand the sample size and include different pilots and control groups to ensure the validity of the findings. Patient feedback should be incorporated into the system to iteratively improve the user experience.

Despite the limitations of the study, the findings have important implications for the design and implementation of HRSs in sensorial ecosystems. Future research should aim to explore the long-term effects of the system on patients’ QoL and identify potential factors that may influence these effects. By doing so, we can gain a better understanding of the potential benefits of HRSs and their applications in different contexts and populations.