Voice Assistant Application for Avoiding Sedentarism in Elderly People Based on IoT Technologies

Abstract

The rise in the use of virtual assistants such as Siri, Google Assistant, or Alexa among different sectors of society is facilitating access to information and services that were previously inconceivable due to the existing digital divide due to age. This situation allows especially the elderly to perform tasks much more easily and to access applications and services that could be a challenge for them with other digital user interfaces. With this in mind, the EMERITI project aims to improve the lives of the elderly through the use of virtual assistants in different case studies. In this sense, virtual voice assistants along with the use of Internet of Things (IoT) technologies can contribute to avoid sedentarism in the elderly; however, it is necessary to address the problem of proactivity presented by the virtual assistants available in the market. This article presents a solution that, through the use of activity monitoring smart bracelets, IoT devices and virtual voice assistants allow the elderly to monitor their daily physical activity simply by using their voice and therefore prevent them from sedentary patterns. Finally, this study presents the technical results obtained after the deployment of the proposed system and discusses the main advantages and the current challenges of the use of virtual assistants in applications to prevent sedentary lifestyles in the elderly.

1. Introduction

The European Commission’s “Ageing Europe” [1] report shows that 20% of the European population is currently 65 years or older and predicts that this percentage will increase to 23.8% by 2030. In addition, most older adults wish to stay in their homes and to age in place [2]. These facts have led to more and more researchers aiming to improve the ageing process in the older adult home and ease the access to Information and Communications Technologies (ICT) by this population.

On the one hand, many studies have focused on improving the home environment of the older adults by using home automation devices that make everyday tasks easier and include ambient intelligence concepts such as adaptation to the user’s context and the use of Internet of Things (IoT) devices. However, employing these systems can be challenging for this population for different reasons that are, in many cases, associated with ageing, such as the lack of ICT knowledge or health problems.

On the other hand, the large majority of this population presents difficulties in accessing the newest technologies, which is the motivation for many studies focussing on creating new ways of access to new technologies to make them accessible to this growing sector of the population.

In parallel, the field of Voice Assistants (VAs) has undergone a radical change in recent years due to improvements in the fields of Natural Language Processing and tasks such as voice processing and synthesis. Big technology companies such as Google, Amazon, and Apple as well as other open-source projects have proposed their own voice assistants (Google Assistant, Alexa, Siri, etc.), providing a totally natural communication interface for the user. In addition, many of current voice assistants allow developers to create applications (“skills” or “actions”) that can interact with the user in addition to elements of their environment.

This user interface has a great potential for older adults [3] due to its ease of use and short learning curve. Moreover, it offers the possibility of using it as a novel user interface with home automation devices and other Internet-based services (such as news, multimedia, and e-commerce) in a seamless way.

It is at this point that the need to research and develop new solutions integrating VAs into the daily lives of the older adults arises, to serve as a tool to help in their daily life and the process of ageing.

Improvement of the aging process of the elderly in their own home is one of the points on which the scientific literature has focused. As a result, it is possible to highlight multiple works in the scientific literature that focus on home care and remote medical assistance systems [4,5]. Along these lines, other systems that seek to promote healthy habits in the elderly have been addressed in the literature in works such as [6] or [7], which address the use of mobile technology for active, healthy ageing, but these studies do not take into account the possible digital divide of the elderly. One of the challenges in the aging process is the lack of physical activity in the elderly. Studies such as the one by [8] seek to quantify Physical Activity (PA), using different sensors such as smart bracelets in order to validate their use for this purpose.

Likewise, works such as that of [9] conclude that activity trackers, for example, smart bracelets and IoT technologies, may be an effective technology to encourage physical activity among older adults. Moreover, works such as the one by [10] highlight the importance of promoting moderate exercise practice among the elderly, thus allowing physical fitness maintenance.

However, the use of smart bracelets alone for activity tracking is not enough for avoiding sedentarism, at this point the engagement and motivation of the user is needed. Studies such as [11] show how gamification of physical activity in older adults achieves greater engagement and interest on performing physical activities.

For these reasons, this work proposes a system that combines the accessibility of a novel user interface such as VAs in order to avoid the digital divide due to age, the use of gamification to promote PA in the elderly, and the use of Ambient Intelligence and IoT devices such as smart bracelets, which allow to monitor that PA. Therefore, the main objective of this system is to improve the physical habits of the elderly, seeking to avoid sedentary lifestyles.

This work is framed in the EMERITI project, which aims to integrate the VAs in the daily life of the elderly by trying to improve the interaction with their environment and making easier the access to the technology via the VAs themselves.

The article has the following structure: Section 2 presents a summary of the state of the art regarding the use of voice assistants to address the digital divide problem and a summary of the main concepts and technologies used in this work, such as Ambient Intelligence, smart bracelets for sedentarism tracking, and the gamification techniques for promoting sports activities on older adults. Section 3 describes the proposed system and as well as its main elements. Section 4 presents the case study developed in this work: the application to monitor the physical activity of the elderly and the results obtained after its deployment. Finally, Section 5 details the conclusions of the work and points out the main lines of open investigation along with possible future work.

2. Background

The study deals with different topics such as the digital divide, voice assistants, and ambient intelligence. This section explains these topics and their relationship with the research shown in this article.

2.1. Digital Divide

The digital divide is a term that has affected society almost since the beginning of the digital age. All scientific articles define it similarly [12,13,14] as a gap between people who do and do not have access to forms of information and communication technologies. Nevertheless, for some authors [15,16] the digital divide is categorized into several levels. The first-level digital divide defines a binary status, to have (or not) Internet access; the second-level digital divide is related to Internet skills and use; the third-level digital divide defines tangible outcomes of the Internet use.

The digital divide is related to and caused by several factors. These factors are usually linked [17,18]. The most important aspects are mentioned below:

• Economic gap: It is related to the difference in the incomes of the population. At the beginning of the Internet, digital devices were extremely expensive and not all citizens were able to afford those kinds of devices. Those first years could have been the cause for the digital divide in some sectors of society. However, the economic gap is still a problem and the cause of the digital divide in many countries.
• Racial gap: The structural barriers and discrimination that different races suffer could be a cause of the digital divide that exists between them.
• Disability gap: Many homes with a disability member in the family have problems accessing the Internet since many of these people need special equipment, which is expensive and, in some cases, hard to find it. Furthermore, those people who do not have access to these special types of equipment need the help of another person to access the Internet.
• Gender gap: Technology in the engineered world is mainly populated by men. There is a huge difference in the number of women vs. men in technical careers. Additionally, the traditional education patterns guide more women to nontechnical fields. These circumstances along with some policies in some countries lead to a digital divide.
• LGBT gap: This community usually suffers similar consequences as already explained in racial and gender gaps. The digital divide caused by the LGBT gap can be huge especially in underdeveloped countries.
• Age gap: Many elderly people have problems accessing the Internet and developing Internet skills. Many of them are considered digital immigrants since they were not influenced by digital devices and the digital age during their young years. Even so, some of them have developed digital skills and can do some basic searches on the Internet. Nonetheless, most elderly people do not have these abilities; this may be due to security concerns, motivation, memory loss, cost, lack of support, among others. Furthermore, some of them live in areas with none or poor quality of Internet network.

This last gap is the most important one for the present study since we are proposing a new system to avoid sedentarism in elderly people. The proposed system makes use of Internet services and devices connected to the Internet. As a result, our system has to avoid and deal with the main problems that elderly people have when they are trying to establish a connection with an Internet device.

Because of that, our system must use a user interface in which elderly people do not need a huge training process and something that they are also used to. Hence, we have selected monitoring smart bracelets and virtual voice assistants to monitor and communicate with elderly people.

2.2. Voice Assistants

A voice assistant (VA), also known as a kind of intelligent personal assistant, is an end-user interface with verbal skills. A VA can talk to and understand users by using different kinds of artificial intelligence techniques such as speech recognition, natural language processing, dialogue systems, and speech synthesis. The communication between the VA and the user is bidirectional; however, the user is the one that starts the conversation by asking the VA to perform an action. The VA uses artificial intelligence techniques to understand the received information and perform the action if possible. The action usually implies a response (answer) from the VA. According to the study developed by De Barcelos et al. [19], VAs have a general architecture, which is summarized in the Figure 1. This shows the functionality and different techniques used by VAs in a normal conversation.

The communication process starts with the user pressing a button of the VA or verbally via the pronunciation of a keyword that wakes up the VA device. The VA uses speech recognition techniques to identify the keyword and begins the conversation by waiting to receive an action or command from the user.

Once the conversation starts, the user pronounces a phrase containing the command s/he wants to obtain from the VA. Speech recognition techniques are used by the VA to transform speech to text; this text is analyzed with Natural Language Processing (NLP) techniques. This phase of the process tries to identify in the user’s sentence the action requested; the techniques also use the current context of the conversation to increase its accuracy. If the action requested is not available or if the semantics of the phrase have not been extracted correctly, the VA will communicate the result to the user using text-to-speech techniques (TTS). On the other hand, if the semantic interpretation of the phrase matches any available action, the VA will execute that action and will answer the user by TTS.

Actions that cannot be performed by the basic functionality of the VA can be performed by the use of multimedia services or external calls, invoking third-party software agents (e.g., Google Actions or Alexa Skills), and even interacting with home automation and Internet of Things (IoT) platforms by acting on the devices of those platforms or retrieving information from them.

There are a huge variety of actions that the VA can perform under request. This factor has led to their increasingly widespread use [20]. The scientific world has increased its interest in the application of VAs in older adults’ home environments in recent years; therefore, some of the more recent works on VAs are described below.

Several studies like Kowalski et al. [3] or the study developed by the Front Porch Center for Innovation and Wellbeing (FPCIW) [21] measure the satisfaction of elderly users who have used Google Home and Amazon Echo via surveys. Other works, such as Reis et al. [22], examined the main functionalities of several assistants to assist older people; they conclude that there is still work to do to improve the communication between VAs and the elderly.

Hence, it is crucial to research how to integrate VAs in the daily life and routines of elderly people, transforming the VAs into useful tools and introducing digital devices and ICTs in this kind of population sector.

It is worth mentioning how the digital divide issue related to the lack of digital skills can be addressed with VAs [23]. Nevertheless, these technologies require an ICT infrastructure in rural areas to function; without these structures, the citizens cannot take full advantage of VAs. Consequently, initiatives such as WiFi4EU [24] and local policies to provide infrastructure in these areas are essential.

2.3. Ambient Intelligence

In order to overcome the obstacle of proactivity on voice assistants and to create innovative applications for VAs and elderly people, the use of Ambient Intelligence (AmI) [25] and concepts becomes a suitable solution to the problem.

The inclusion of sensors at older adults’ homes and/or the integration of wearables able to collect data relative to their health or current conditions is an area of high interest in the scientific literature nowadays.

The approaches that can be found in the literature are divided into those that are aimed to sense the older adults’ households and monitor their environment, and those that use wearable devices to continuously monitor the users’ conditions or a combination of the above. Both approaches have detractors due to the level of intrusiveness that these systems may be subjected to.

Supporters of the first approach argue that the proper functioning of the system requires the older adult to remember to put a certain wearable on. Advocates of the second approach argue the high cost of the complete sensing set of devices for the houses and the level of invasiveness of the older person privacy, especially in camera-based approaches such as the one of Ramanujam et al. [26].

There are multiple studies in the literature of both approaches, such as the work of Grgurić et al. [27], which shows the use of sensors in the elderly homes to develop a system that analyses daily habits and identifies anomalies, notifying a family member or caregiver if necessary.

On the other hand, other works such as that of Lashmi et al. [28] propose an AmI system based on a video surveillance and an anomaly detection system like the previous one. The work of Almeida et al. [29] proposes a similar system but using low-cost sensors of temperature and humidity as well as door and window sensors to analyze people behavior. In this work, they employ an approach closer to Edge Computing.

However, none of these works deal with the ease of interaction of the older adults that suffer digital divide, and they do not take into account the ease of use of the system. The use of voice assistants for this purpose is a research area which requires new solutions to validate their effectiveness in the everyday older adults’ life.

2.4. Smart Bracelets for Sedentarism Tracking

According to a study developed by the World Health Organization [30], chronic diseases will account for almost three-quarters of all deaths worldwide. These diseases could be prevented through a healthy diet, avoidance of unhealthy products like alcohol and/or tobacco, and regular physical activity [31]. However, many people tend to have a sedentary life [32]. This situation is worrying especially in elderly people.

Therefore, it is important to monitor elderly people to promote physical activity to those who have a more sedentary life. There are several solutions to achieve the monitoring task, from video monitoring [33] and smart home monitoring [34] to microsensors. The microsensors solution provides a less intrusive solution and is much cheaper than the other solutions. These microsensors, like smart bracelets, can be placed anywhere. Additionally, this solution has already been tested to analyze body movements in the elderly [35]. Furthermore, the use of a smart bracelet does not suppose a big challenge to the elderly; it is like a watch and it does not require any action from the user. Thus, it seems like a perfect solution for monitoring.

Smart bracelets allow to measure and control several variables. Hereafter, we mention some of these attributes:

• Fall detection and gait/balance analysis: Falling is a common issue that elderly people must deal with. The time that a person remains on the floor after a tough fall can be crucial to his/her life, and many older people have difficulties standing up when they fall. Hence, and thanks to smart bracelets, it is possible to detect fall detection and gait/balance analysis. Several studies have been carried out during the last years [36,37,38]. These systems can help elderly people and reduce the secondary effects of many falls’.
• Sleep Analysis: It is possible to monitor sleep. Many elderly people develop dysfunctional sleep patterns that can cause problems in their daily life. Smart bracelets can detect and notify users about their sleeping activities [39,40].
• Sedentary/activity monitoring: Smart bracelets can measure the number of steps that the user performs during the day [41,42]. They can measure the activity and sedentarism. This monitoring also allows measuring the energy expenditure of the user during the day [43]. Activity and energy expenditure monitoring allow the user to be aware of their daily life’s rhythm so they can avoid diseases like obesity and type 2 diabetes mellitus.
• Location Awareness: Indoor and outdoor locations can be obtained through different techniques, such as GPS, wearable cameras, and other wireless communication technologies (Wi-Fi, BLE, RFID, etc.) [44]. This information can be also useful to monitor the activity of the user or to detect if they are at home or not.

Our work focuses on the last two points to deal with sedentarism in elderly people. A voice assistant interface notifies the user in the case where less activity was performed than was expected. All the previous works do not provide this novel interface that requires no interaction at all with the smart bracelet. Therefore, our work proposes these new notification alerts for elderly people.

2.5. Gamification on Adults

Gamification consists of the use of techniques and dynamics typical of game design in contexts that are not necessarily recreational, in order to obtain motivation, reinforce positive behaviors, or lead, in a comfortable and nonintrusive way, a user towards an objective. Despite the fact that gamification is an increasingly popular concept, there is a canonical dearth in research and application. Klock et al. [45] describe the state of the art of tailored gamification through a systematic literature review.

Gamification techniques have been used in several domains, focusing on both youth people and adults. For example, a study related to the technological gap in older people is that of Kevin Browne et al. [46], where a gamification process was designed that focused on tablet applications to teach literacy to nonliterate adults. They conclude that the integration of game design elements increases learner engagement, moreover, they detected user behaviors that suggest greater emotional involvement. Attending to how to approach gamification techniques focusing on the different groups, Van der Lubbe et al. [47] provide an overview of the field, focusing on how serious gaming and gamification can empower vulnerable target groups, subdividing those target groups according to age, physical health status, mental health status, cultural background, and socio-economic status, where the age target group—the most diverse—has the most different types of risks and empowerment methods.

In the field of physical activity, a positive impact has been found when applying gamification. Parra-González et al. [48] study and contrast the efficacy of two methodologies, gamification and flipped learning, in the area of physical education in childhood and adolescence: primary and secondary education. In another study, Ashley et al. [49] find that those game mechanics engage and motivate groups of people to reduce their physical inactivity.

Finally, works such as those of [11,50,51], focused on the study of motivation for physical activity in the elderly, conclude several guidelines that can improve engagement, such as (1) elements of intrinsic motivation, such as the choice of exercise intensity or the difficulty of challenge; (2) extrinsic motivation elements such as valuing effort, showing progression, or highlighting achievements, and (3) feedback elements that show a graphical representation of progress and encouragement through motivational phrases.

In view of these studies, we consider that gamification mechanisms are important for our study. Therefore, we include a gamification system together with ambient intelligence devices with the aim of motivating the elderly in the development of physical activity in a nonintrusive way.

3. Proposed System

This section describes the architecture of the proposed system and each of its component elements and together with their role in the system.

When developing the system, one of the most important points taken into consideration is that of reactivity, seeking to make it capable of interacting in real-time with the end user in such a way that the feedback is as natural as possible, since the end user must be provided with the simplest and clearest possible use.

In this way, the system should allow the elderly to set a sportive challenge through the VA, to be carried out throughout the day, which consists of completing a certain number of steps to help them maintain their physical activity. In addition, as the day goes by, the system will provide feedback through voice messages, following basic gamification guidelines, until the set goal is achieved, at which time the user will be congratulated for the achievement. Therefore, the user will set the goal by voice, and, through a smart bracelet capable of storing his/her activity, progress will be monitored throughout the day. The data from the smart bracelet will be collected by an Edge Computing Device (ECD), which is a device with limited computing capabilities, and transmitted to remote cloud services to know the user’s status when s/he is at home.

In the following subsections, we will go into the details of the elements that make up the proposed system.

3.1. Architecture

The architecture developed, shown in Figure 2, is divided between the elements that act in the local area network (the so-called edge), composed of those with which the end user interacts directly, and on the other hand, the services that run in the cloud, which will resolve the requests made by the different users that make use of the system.

• Intelligent voice assistant: A device with an integrated intelligent assistant, additionally, it could have an LCD screen on which informative images can be displayed. It constitutes the main communication interface between the user and the system, allowing all interaction between the user and the software to be carried out by voice, making it as transparent and fluid as possible. This assistant communicates with the cloud services in order to complete the requested actions carried out by the user.
• Edge computing device: It consists of an SBC (Single Board Computer) that will be located in the home of the senior, next to a VA. It is a device with enough computational capabilities to run the programs responsible for monitoring the smart bracelet (explained below) via Bluetooth Low Energy (BLE) and allowing interaction with the user through the VA when it detects that this is close. It is able to communicate with cloud services through the Internet.
• BLE smart bracelet: Bluetooth smart bracelet to be worn by the user, with the capability to monitor the physical activity performed throughout the day. This will serve both to monitor in a nonintrusive way if the user is close to the ECD and the VA, and to periodically collect information on the user’s physical activity (number of steps, calories burned, heart rate, oxygen saturation, etc.). In this hardware, it will be necessary to take into account the power consumption of the device since some of these sensors have a high energy consumption.
• Cloud services: Services that will allow communication between the VA and the ECD through the Internet. They consist of conversational agents and an NoSQL real-time database. This service will be responsible, on the one hand, for monitoring the gamification tasks related to the progress of the user and challenges, and on the other hand, for interacting with the ECD, which will proactively notify the user’s progress or the achievement of the challenge. Moreover, it allows the storage of a history of the user’s physical activity for further analysis.
• Conversational agents: Software agent implemented as an application for VAs—under the name EMERITI—which, through the intelligent VA, allows users to set the challenges they consider appropriate and check, when requested, their daily progress through a conversation.

3.2. Proactivity of Current Intelligent Voice Assistants

The most popular intelligent voice assistants, including Google Assistant, Amazon’s Alexa, and Apple’s Siri, are not proactive. This means that, apart from certain alerts, reminders, or other elements provided by the operating system, they only allow dialogs and applications to be initiated by the user, so as not to interfere with the user’s day-to-day conversations. This respects the privacy and prevents these devices from running third-party programs in the background that can be a drawback from a security point of view.

Therefore, for the development of systems that require proactivity, alternative methods must be sought to interact with the user without the user initiating the dialogue. In our case study, this is achieved through the interaction between the VA and the ECD using the Chromecast CastV2 protocol [52]. In this client–server protocol, an application on a receiving device listens for cast requests, and, when another sending device sends a cast request, a certain code is executed, or certain information is displayed, on the receiving device. This allows, for example, certain links to be loaded remotely or audio to be arbitrarily streamed to run at a time of the developers’ choosing. Many of today’s VAs implement this protocol (particularly those of Google Assistant).

In the proposed architecture, this protocol will be used to solve this issue and achieve proactivity on the part of the system, since, through the detection of the smart bracelet, the ECD can interact with the user without having to first run an application on the VA.

This proactivity is fundamental for the gamification of the user’s physical activity, as it allows providing feedback without the need for the user to deliberately communicate with the VA.

3.3. Developed Software for the Edge Computing Device

An application has been developed that will run as a background service. It has two main functionalities that run simultaneously through two independent threads of execution, which will be described in depth in the following subsections. The first, called tracker thread, is in charge of managing the Bluetooth to interact with the smart bracelet and detect the user. This point is essential in the system since the detection of the user’s presence is vital to establish and maintain an interaction with the user and therefore enable the system to be proactive.

The second one, called mailbox thread, is in charge of the management of messages generated in the cloud services to interact with the elder.

3.3.1. Tracker Thread: Tracking Smart BLE Smart Bracelet

This thread is in charge of managing the Bluetooth BLE connection of the ECD periodically, detecting nearby smart bracelets. It has two responsibilities: on the one hand to detect the presence of the user, so that s/he can be interacted with correctly through the VA, on the other hand, to obtain the data from his/her smart bracelet so that his/her daily progression can be processed.

To fulfil these two responsibilities, the tracker thread periodically scans for Bluetooth devices trying to find the smart bracelet associated with the user, filtering those with a Bluetooth BLE service related to the smart bracelet used, to subsequently to check if among the devices found is the one associated with the user linked to the ECD. Once the desired smart bracelet is found, the tracker thread connects to it in order to obtain the information of the physical activity performed by the user until that moment to be able to upload the user’s updated data to the real-time database. At that moment, the changes produced in the database are monitored by a function as a service (FaaS) that is in charge of processing them. Finally, after processing the data, this function will establish through a message whether it is necessary to transmit an encouragement message to the user based on the Gamification strategy implemented in the system. This message will be transmitted and processed by the mailbox thread, which will be detailed later in Section 3.3.2.

In the smart bracelet detection algorithm, described in the Algorithm 1, the connection only occurs when the distance based on the Received Signal Strength Indicator (RSSI) exceeds a certain distance threshold. Different types of filtering have been studied for calculating the distance and inferring the presence of the device: (1) basic connection based on the distance from the RSSI without the use of a filter, (2) connection after applying a filter based on the mean, (3) connection after applying a filter based on the median, and (4) application of the 1D Kalman filter.

Algorithm 1. Smart bracelet tracking algorithm.

Inputs: distanceThreshold

filter

userBracelet

deviceAdvertisments <- scanDeviceAdvertismentPackets(userBracelet)

rssi <- applyRssiFilter(deviceAdvertisments, filter)

distance <- calculateDistance(rssi)

if distance < distanceThreshold then

connectToDevice()

braceletInfo <- getDataFromDevice()

sendInfoToDB(braceletInfo)

end if

Outputs: distance

braceletInfo

The RSSI metric was processed by obtaining the distance as a reference measure. The Log-distance path loss model, Equation (1) (Patri and Prasanna [53]), has been used to calculate the distance between the smart bracelet and the ECD (Equation (2)), where RSSI(d) is the signal strength at a distance d in dBm, RSS_d0 the signal strength at a reference distance d₀ in dBm, n the path loss index of shadow fading whose approximate values were extracted from Röbesaat et al. [54] and can be seen in the

Table 1. Path loss exponent for different environments.

Environment	Path Loss Exponent n
Free space	2
Urban area cellular radio	2.7–3.5
Shadowed urban cellular radio	3–5
Line-of-sight in building	1.6–1.8
Obstruction in building	4–6
Obstruction in factories	2–3

, and X₀ is the Gaussian noise with zero mean, which is assumed to be zero.

$$RSSI{\left(d\right)}_{dbm}=RS{S}_{d_0}-10n{\log }_{10}\left(\frac{d}{d_0}\right)+X_0$$

(1)

$$distance=d_0·{10}^{\frac{RS{S}_{d_0}-RSSI\left(d\right)+X_0}{10\times n}}$$

(2)

The average and median filters adjust the RSSI received signal by taking into consideration the last n measurements, also called the frame size parameter. To obtain the estimated RSSI, the average RSSI of these n measurements is calculated in the first filter or the median value in the second one is chosen. The value of n should be small in cases where a short reaction time is desired or the elicitation times between measurements are long, otherwise too much delay will be added to the estimated measurement due to the low density of samples over time.

A Kalman filter is an optimal estimation algorithm used to estimate measurements that cannot be obtained directly and correctly. Yoon et al. [55] detail the problems in obtaining accurate positional measurements on indoor Bluetooth BLE devices and how to minimize the error through these filters. Röbesaat et al. [54] also address this problem through an experimental study employing this type of filter for device positioning through triangularization using low-energy Bluetooth modules, thus achieving more accurate measurements.

In our study, a Kalman filter for one-dimensional data is used to reduce noise, taking the one used by Bulten et al. [56]. It takes into consideration the last n RSSI measurements obtained, in order to reduce the error, generally produced due to the presence of walls, people and/or objects, multi-path reflections, or radio differences.

The main problem that can arise with the use of these filters is the reaction time, which becomes much longer even though the measurements are optimal when the user is still standing. In the case study, where the VA is required to respond in real-time when the user is nearby, it is essential to minimize the response time.

3.3.2. Mailbox Thread: Communication with Cloud Services

The second thread is in charge of listening for messages from the platform, which can be generated from two sources: on the one hand, after the communication of the user with the application through the VA, on the other hand, through the function as a service once they process the data from the smart bracelet that are introduced in the NoSQL database in real time. Depending on the type of message, the device will act accordingly, these can be of three types:

• Assign smart bracelet request: Starts the process of assigning the smart bracelet in the next scan of the Tracker Thread. When the application receives this order to pair a new smart bracelet with the user, it starts the process of associating a nearby smart bracelet. This process is necessary for the user to be able to make use of the system, which will be described later in the conversational agent use cases. In this procedure, the ECD connects to a nearby smart bracelet compatible with the system and, once this connection is achieved, it will send a notification invocation to the smart bracelet, waiting for the user to respond by pressing the button on the smart bracelet. If the user presses the button, it will be assigned to the user as their property and linked to the ECD. If it is not pressed and a certain threshold of attempts is exceeded, then no smart bracelet will be assigned.
• Tell: This type of message will be the most used by the application. It is used to make voice notifications to the user using the CastV2 protocol, informing the user through the VA about the status of his/her challenge. The messages will be those established by the functions and may be related to the user’s progress or the completion of the challenge set. In this way, proactivity in the VA is achieved without the need for an explicit invocation by the user; this proactivity keeps the user aware of his/her situation without the user having to actively check it. Additionally, it reminds the elder to charge the smart bracelet when it is low on battery.
• Show progress: This message is generated through a lambda function when the user requests, through the VA, to display the progress graphically. This graph is generated by the daemon and is transmitted to the VA by using the CastV2 protocol. It is important to mention that this message is only available when the VA has a display.

In addition to message resolution, this thread is responsible for connecting to the configuration assigned to a user’s ECD and watching for updates to keep the system consistent.

3.4. EMERITI Fitness VA Application

The purpose of the EMERITI fitness action is to act as an intermediary between the system and elderly people to facilitate as much as possible the communication. In this way, a user will not need to have prior knowledge or familiarity with any technology, except for a few guidelines to be able to carry out a fluid conversation with the VA and fulfil the objectives for which the action was designed. The use cases for this action are described below.

3.4.1. Pair New Smart Bracelet

This use case describes the process of registering the smart bracelet that the elderly person will normally use. When the user invokes the pairing for the first time by talking to EMERITI, the user will already have an ECD associated with it but will not yet have the smart bracelet associated with it.

Therefore, when the user interacts with the action for the first time and does not have an associated smart bracelet, they will be warned about it, and they will be informed that to configure one they must press the button on the smart bracelet when it starts to vibrate. If the user presses the button and the pairing process is successful, the user will be notified by the VA. If not, the user will have to pair the smart bracelet the next time s/he talks to EMERITI. The sequence diagram of this use case is shown in Figure 3. Smart bracelet register sequence diagram.

3.4.2. Gamifications Use Cases

The main use case of the application is to set a daily challenge of steps to achieve. This challenge can be set through the voice assistant. When the user invokes EMERITI and has an associated ECD and smart bracelet associated, the action will provide the different available possibilities.

Once EMERITI is presented, the elderly person will have to say a phrase that is related to “step challenge” to trigger this use case. When triggering the use case, it will first check whether a step challenge is already stored in the real-time database for that user. If there is already one, the VA will ask if s/he want to change it. If not, it will ask him/her how many steps s/he wants to achieve each day. This same question will be asked if s/he wants to change the current daily challenge, and if s/he does not want to change it, the conversation will end.

In case of modification or creation of a step challenge, the VA will be waiting for the user to provide a number of steps, and if s/he provides a number that is feasible, it will be saved in the database. Subsequently, EMERITI will ask how often the user wants encouragement messages to be sent. The user can choose between three frequencies: never being alerted, being alerted a few times (every 5 h), or being alerted many times (every 2 h). After choosing the frequency, the conversation will end.

Finally, if the number of steps provided is unfeasible because it is too high, the user will be asked to repeat the number of steps. The sequence diagram of this use case is shown in Figure 4.

The second use case related to gamification corresponds to the encouragement notifications generated by the device depending on the percentage of steps the user has taken so far that day. The frequency of these notifications will depend on the choice made by the user during the previous use case. This use case consists of a simple sequence where the ECD sends the corresponding encouragement message to the VA and the VA plays it back when the user is nearby. It is important to mention that no early morning announcements are generated in the time slot defined by the user in order to avoid being disturbed.

The last use case defined in this section, which focuses on the user being able to see the results obtained during a day, corresponds to the use case of showing progress. This functionality of the system corresponds to the physical activity history that can be queried to the voice assistant asking for the current progress to be displayed. Ultimately, this use case consists of a very simple sequence, where the adult asks EMERITI to show his/her daily progress of a day and EMERITI shows him/her a chart with this data.

4. Case Study and Results

This section describes the results obtained after conducting a case study of this application with a group of elderly people volunteers in the age range of 70 to 90 years for one month. The following sections describe each of the devices used for the implementation of the proposed system in this case study and the results of the different use cases.

4.1. Hardware Used

This section describes the hardware used, consisting of the following three elements of the architecture: (1) intelligent voice assistant, (2) Edge Computing Device, and (3) Bluetooth BLE smart bracelet.

4.1.1. Intelligent Voice Assistant

It should be noted that any device that has the Google Assistant integrated and implements the Google CastV2 protocol can also be used to carry out the main functionalities of the system, even if it does not have a display.

On the other hand, there are other technologies such as Amazon’s Alexa Voice Assistant, but it was decided to use the Google Assistant option because it was more convenient to develop and publish the action, as Google provides clearer documentation for development, and the Google CastV2 protocol makes it possible to add the proactivity we want to the system in a simple way.

The Smart Display Lenovo with Google VA was used for this purpose, as it allowed the user to view the progress charts. Furthermore, one of the simplest and cheapest devices with the Google VA, the Google Nest Mini, was used for testing.

Returning to the Smart Display Lenovo, this hardware device will be the communication interface between the elderly and the exercise-monitoring application. It should be located close to the ECD, as the latter is responsible for monitoring the user’s proximity and transmitting the feedback corresponding to the user’s situation.

4.1.2. Edge Computing Device

The ECD has been implemented using a Raspberry Pi 4 4GB, but another device with less computational power could be used as long as it has a Bluetooth interface, connection to the wide area network—either through a WiFi or Ethernet interface—and the capability to run software of a certain complexity, as it communicates with real-time databases.

As introduced in Section 3.2, intelligent voice assistant policies have some limitations in terms of the proactivity they can have with the user, and they can condition the way in which applications, Google Actions in this case, are developed. Through the software developed for the ECD in combination with the smart bracelet of the elder, a more proactive dialogue can be established by the system that reminds the user about his/her progress.

As for the functions performed by this device, as introduced in the Section 3, they are divided into two threads that are in charge of adding the proactivity of the system: performing the monitoring and the assignment of the smart bracelet.

For the implementation of these threads, the Raspbian Buster operating system was used on the Raspberry Pi 4 and a Node JS application was developed. The PM2 process manager [57] was used to deploy the application. The application initially performs a setup process and then remains running in the background (in Daemon mode).

The first thread aims to search in 2 s intervals for the user’s smart bracelet, and if it finds it, it connects to it and dumps the data that the smart bracelet has collected to send it to the database used, Cloud Firestore in this case study. The changes produced in the Cloud Firestore collection are monitored by a Cloud Function that is in charge of processing the new data. After processing, the Cloud Function will establish whether it is necessary to transmit an encouragement message to the user. This message will be transmitted and processed by the Mailbox thread detailed at the end of this subsection.

In terms of the most important data obtained from the associated smart bracelet, the following should be noted:

• Smart bracelet’s MAC: This feature corresponds to the MAC of the smart bracelet and is necessary to check whether the detected smart bracelet is the one assigned to the user.
• Battery: This feature corresponds to data related to the smart bracelet’s battery, such as the remaining battery percentage, the last time the bracelet was charged, or whether it is charging.
• Pedometer information: This characteristic corresponds to the daily physical activity data recorded in the smart bracelet. This data corresponds to calories, distance, and steps travelled. The latter are the ones that are checked to see if the daily challenge has been met or not, and to generate a notification to the user if necessary.
• Time: Corresponds to the time of the moment when the smart bracelet data have been obtained.

The second thread, Mailbox thread, is in charge of handling the messages received by the platform, stored in Cloud Firestore, carrying out the necessary functions for the assignment of the smart bracelet, and the sending of notifications to the senior. In the smart bracelet assignment process, the Raspberry Pi 4 is connected to a new smart bracelet and the procedure described in Section 3.3.2 is carried out. As for tell or show progress messages, these messages are stored in Cloud Firestore, and when the daemon checks that a new message has been added, this thread performs the necessary function, sending the tell message to the Google Home via the CastV2 protocol for playback, or generating the progress chart and displaying it on the Lenovo’s screen, in case this type of message is received.

4.1.3. Bluetooth BLE Smart Bracelet

There are different smart bracelets for this purpose used in the literature by Tedesco et al. [58], Asimakopoulos et al. [59], and Kononova et al. [9] in their works. Some of these bracelets include the Lively watch, Apple Watch, Amazon Fitbit, and the Garmin Vivofit 2. The first bracelet has a higher price than others on the market, as it comes with smart pill dispensers. An added functionality for use by the elderly is the receipt of medication reminders. However, the main drawback of this smart bracelet is the lack of a detailed documentation to access the data it collects. As for the Apple Watch, it is one of the most expensive bracelets on the market, and there are no manuals or packages explaining how to connect and collect data from the device. If we add to this the tightness with which Apple usually protects its products, it is not a good choice for the case study. Something similar happens with the Garmin Vivofit 2, which is significantly cheaper than the Apple Watch, but still more expensive than some of its direct competitors, and there is no manual explaining how to connect to it and get its data. On the other hand, bracelets such as the Amazon Fitbit and the Xiaomi Mi Bands have the best prices on the market in terms of value for money, and there are also various libraries that allow you to connect and obtain data from these smart bracelets through a communication interface.

After evaluating these options, the Xiaomi Mi Band 2 and the smartwatch Amazfit Bip have been used as the smart bracelet devices for the elderly, due to their low cost, low power consumption, and long battery life (up to 30 days of battery life), and, above all, because of the possibility to access the data through a well-defined communication interface, for which several software libraries are available. In the case of the Xiaomi Mi Band, the second version has been used over the others, since it is simpler, cheaper, has a longer battery life, and it is easier to understand its operation, in addition to the fact that from the fourth model onwards a key is needed to carry out the connection process and obtain the data. These new models could be included in the system but would involve additional configuration steps for the user, and we have preferred to keep the process as simple as possible.

In this respect, the idea of developing an ad hoc smart bracelet for this purpose was considered, but a commercially available option that would suit the requirements of the case study was finally chosen.

4.2. Test Results

Returning to the case study, the first thing we did was to observe how the user got along by starting the conversation with the system and setting up his/her or her personal smart bracelet. Afterwards, when talking to EMERITI, it will list the different functions available. The results of these different functions of the system during the case study conversation are described below.

4.2.1. Daily Step Challenge

An example of the step challenge use case is shown in Figure 4. This test consists of modifying a user’s step challenge, and it is displayed in the Figure 5 using the Google Actions Console simulator for the sake of clarity, although the result in the proposed system would be the same (depending on the specific device that implements Google Assistant). In point 1 of the Figure 5, the EMERITI action is told that we want to set a step challenge, thus leading to the step challenge use case. In this case, the system checks that the user already has a challenge marked, so it starts the modification process by asking if the user wants to modify his challenge. In point 2, we confirm that we want to modify the step challenge, so EMERITI asks for the number of steps we want to set as the daily challenge. In point 3 of the image, we say that we want to mark 500 steps as a daily challenge, and EMERITI checks that it is a feasible amount and asks how often we want to be encouraged throughout the day, giving us three options: never, so that we are not encouraged, few, so that we are encouraged every 5 h, and many, so that we are encouraged every 2 h. Finally, in point 4 we choose that we want to be encouraged a few times, and EMERITI responds that the new step challenge has been saved correctly and now it will only alert us every 5 h.

4.2.2. User Proximity Detection

For the user proximity detection, three different RSSI signal filters have been applied—average, median, and Kalman—which were described in detail in Section 3.3.1 so that they can be compared with each other and with the case of not using any of them.

To study the effectiveness of the filters, RSSI values were taken with the antenna pointing towards the smart bracelet at different distances. The approximation of the distances was obtained from the signal intensity based on the path loss model (Equations (1) and (2)). The statistical distribution of the RSSI samples obtained at different distances can be seen in Figure 6, where the raw values obtained are shown on the left and their distance estimation with a box of the same color on the right; it can be seen that the quality of the measurement decreases as the distance increases with increasing error and deviation, so that each time the estimated distance is less accurate.

Since one of the requirements to be met by the system is that it should be as reactive as possible, and considering that the reactivity of this type of smart bracelet is not always optimal, narrow frame sizes were used in the filters average (n = 3), median (n = 3), and Kalman (n = 2). The results of the tests at fixed distances are shown in the Figure 6, where each color shows a distance. As also shown in the Figure 7. Application of the filters at different distances. at a greater distance we find a greater error.

To test the behavior of the system when the user wearing the smart bracelet is in motion, the experiment shown in the Figure 8 was performed, where the individual moves at a constant speed of 1 m/s, first approaching and then moving away from the ECD. This shows how the perceived distances vary as a function of time in each of the three filters and the raw values with respect to the ideal values.

From the graphical results obtained and the tests carried out, although the filters help regulate the noise of the actual RSSI, it can be concluded that they do not ensure the quick reactivity of the system in the vast majority of cases. This is clearly seen in Figure 8, when the user starts to move away, the filters may indicate that s/he is standing at the wrong position or still approaching, for example, around 5–6 s, when the direction is reversed, and the reaction of the filters is delayed. In this case, it is important to note that a constant speed is assumed and that correct readings are obtained every second, which does not usually correspond to a typical real case, in which the user would move faster and the delay would be greater. If the speed were to be increased, the delay in the filters would be even greater and therefore the error would be higher. As mentioned above, it is important that the system is reactive and notifications are made when the user is in close proximity. In this sense, unfortunately, the use of filters would make this task more difficult. Therefore, for the study it has been decided to use the raw values of the distance in order to generate a quicker and more direct response.

However, as can be seen from the data in Figure 5, the accuracy is high when the smart bracelet is less than 2 m away. This has been the distance considered to univocally detect the presence of the user.

4.2.3. Notices during the Day

When the user is close to the ECD, a check is made to see if there are any pending encouragement or congratulatory notices, and if so, the voice assistant broadcasts the corresponding notice.

In the first tests, every time the user approached, and his/her activity was checked, a notification was generated. Furthermore, the time of day was not considered, and the voice assistant provided the corresponding notices in the early hours of the morning. To solve these problems, we added a threshold so that the system would warn only once every two or five hours, depending on the user’s choice, and several conditions so that it would warn only during the day.

On the other hand, five types of notices were considered depending on the percentage of steps completed with respect to the challenge. Figure 9 shows the encouragement message sent to the user when s/he exceeds fifty percent of the daily steps. The congratulatory notification, which can be seen in Figure 10, takes place when the user completes the challenge and is only notified once.

Finally, to add even more proactivity to the system, a low-battery notification was added to warn the senior that his/her or her smart bracelet had less than twenty percent battery life. Thus, during the case study, the proactivity of the system and the encouraging effect of the notices were tested.

4.2.4. Physical Activity History

Another important case study of the research is to graphically display daily progress to the user. Figure 11 shows one of the tests performed for the use case of showing the user’s physical activity progress.

In this case, these charts have been used to debug the system and to see when the notices to the user have been made, in yellow, and when the challenge steps have been achieved, in red. The user would see a similar result, but without seeing the colored labels from when the notices were made. As for the charts presented by EMERITI, the one on the left corresponds to a cumulative graph in which the steps made by the user are accumulated as the day goes by, and the one on the right shows the number of steps made for each of the hours in a day. The latter allows the user to observe at what times of the day s/he is most active and thus be able to improve his/her habits and deliberate at what times s/he should walk more.

5. Conclusions

During the development of the research, it has been observed that we have been able to implement a system capable of interacting naturally and simply with elderly people to promote physical activity in this sector of the population. For the implementation, priority has been given to the use of low-cost devices that offer a simple use and a small learning curve to deal with the existing digital divide among the elderly.

In this regard, it is important to note that the use of activity smart bracelets and virtual assistants (devices with which the elderly interact) allows visually impaired users to avoid having to interact with them directly since communication is carried out through a conversation with the assistant.

Another important aspect of the research is that some proactivity is achieved on the part of the assistants. As mentioned in previous sections, one of the biggest drawbacks of virtual assistants is that they do not interact with the user unless the user asks them directly or invites them to participate in the conversation. However, in our system, the assistant can detect if the user is nearby and contact him/her informing him/her about his/her physical activity.

In the same way, and also thanks to this proactivity, the system can periodically remind users whether or not they are meeting their daily goals. This allows users not to have to be aware of it because, in this case, the assistant and our system can remind them.

However, one of the problems that we can find in this type of system, in which the resources and services of large multinationals such as Google or Amazon are used to analyze the interactions of users with the assistants, is the lack of privacy that can be generated by allowing their services to analyze the interactions of users with the assistants. To solve these potential drawbacks, an interesting future line would be to employ edge virtual assistants such as Rhassspy [60], which are currently on active development. These types of systems will be considered for future implementations of the proposed study.

On the other hand, another future line of research to be considered is the implementation of other effective gamification techniques to set different challenges to the users to achieve the daily goal progressively throughout the day. Similarly, a complete field study will be carried out in which several elderly people will test the system and observe the possible adaptation of the proposal to this sector of society. The study will be carried out with a significantly large population of users to conclude the effectiveness of the system proposed in this article in comparison with other existing systems and without any system at all. In the current study, only a small number of volunteers (three users with age between 57 and 82 years old and five developer users) have participated in the study with the main objective of testing the feasibility and adaptability of the system to their needs and requirements; therefore, no meaningful conclusions can be drawn.