Tele-Treatment Application Design for Disable Patients with Wireless Sensors

Abstract

This paper consists of the development of a system to help patients with different disabilities, affected by rare or chronic diseases or any kind of dependence through tele assistance, virtual interaction and intelligent monitoring. The main goal is to increase the quality of life of the minorities who cannot take full advantage of the healthcare system by providing an alternative way of monitoring them with the technology embedded in this paper. The result of the paper is not intended to be a single solution, but a modular system that allows the construction of an application that is able to measure the needs of a health administration and the patients. The paper also pursues an educational training to the facultative trainees in a new way to approach patient treatments. It can improve the quality of life of the patients by saving them time and other resources in moving to the Health center and the professionals can also save time as they can take advantage of the online treatments by using the proposed system.

1. Introduction

Telehealth care has revealed itself as a revolutionary high-quality approach to help to treat patients [1]. There are currently several implementations of systems that help to cope with patients that do not enjoy the same mobility as the majority of the population [2]. This seems to evolve and to grow in the last decade [3]. For example, Konstantas et al. [4] introduced a proposal of mobile monitoring establishing the fact that technology can offer promising results to remote patients as a support for conventional treatment. The theoretical study is to complete although the practical application of such system still shows several gaps in the communication channels of the system. Other authors, van Halteren et al., proposed a system that introduce the specifics of a module of telemonitoring by the use of sensors and other smart devices that collect signals from the patients [5]. This proposal, however, lacks of interactive, integrative and communication modules and it does not provide intelligent and automatic processing, such as image processing and computer vision alternatives. Besides, there is barely any use of social media.

As Smart devices and mobile apps are considered one of the first solid model applied to telehealth care, this paper complies with the following services:.

• Integrate into the audiovisual media system (mainly television, and smart phones because of its ease of use and its easy penetration in the domestic environment); although other elements or devices can be utilized as long as there is a physical channel of bidirectional communication between users, professionals and/or relatives.
• Provide mechanisms for assistance to the user by professionals, relatives or friends in different fields, such as the distribution of multimedia contents, videos, pictures, articles, etc.
• The user takes advantage of the TV or any other output device. The time the user spends on watching TV can be utilized for receiving alerts and gives the user the possibility of watching recommended videos that can supplement the therapies or treatments. Below there are some examples.
• Create alarms and warnings in the follow-up of the medication intake.
• Support the rehabilitation with the programming of multimedia contents to the user.
• Provide tele-accompaniment. The end user can be added a social network of patients with the same symptoms, so they can share experiences and tips to improve their results.
• Personalize programs with multimedia contents to the user. These consist of a composition of social and professional oriented materials to be utilized by the user’s knowledge and supervised by the professional.
• Develop monitoring systems mostly with the gyroscopes and wireless sensors connected to Smartphones and Smartwatches. This detects Situations and potential Conflicts that deserve special attention.
• Develop alarm systems and primary alert. The user has the option of notifying the need of assistance by using his watch or any other device.
• Construct a user notification system that sends reminders to take the right medicines, to review the materials the professional gathered and to be in contact with the supervisor.
• Develop of a modular architecture, so that the system is adaptable to customer needs.
• Define and to implement the access levels of Professionals in order to adapt the system to the current policies of the professionals and client institutions.

2. Related Work

Undoubtedly, the use of technology in Healthcare system is growing rapidly. Monitoring tools, software to support diagnosis, markers counters are used more and more by the professionals in hospitals and Health centers.

In particular, remote assistance has been studied in the last years as a support for traditional treatments [6]. A mobile healthcare system is described with a fair number of details. This solution allows patients to be cared for in their preferred location, typically at home. it provides patient education and support for preventive care; for example, to those trying to lose weight by using videophone or web based clinical consultations, replacing routine visits such as face to face annual reviews can be a good option. It also Improves the efficiency of other treatments more conventional. Proactive education and support, such as via web forums, may facilitate self-management and help with coping (for example, in people with chronic back pain) or prevent exacerbations of conditions such as asthma or any other problem triggered by stress. Use of monitoring techniques can enable earlier detection of disease exacerbations, thereby facilitating timely management and support. It allows greater opportunities for continuity of care with the same clinician and more frequent assessments. Furthermore, mobile healthcare is known to reduces costs for patients by obviating the need for time off school or work and due to the time travel. As complete as [6] is, still misses a lot of features that can be very helpful to the patients; for instance, the monitoring component only deals with basic bio signals and has no intelligent processing nodule. Moreover, the system does not have an option for a full integration with the TV which is one of the preferred environments for reduce mobility patients. The social schedule also is very poor. A complete social schedule with a hub of contacts with family and other patients, can make the whole process more enjoyable as the patient can share thoughts and accomplishments with other patients and friends.

Regarding automatic signal and image processing no major telehealth care systems have been fully develop beyond experimental trials. Arteta et al. [7] designed a new approach to improve experimental CPU’s that can speed up the pattern recognition processing applicable to image and computer vision in smart devices. The biggest problem this model faces is finding the Hardware/Bioware to implement such model [8].

The evolution of telehealth care that are described in Wac [3] is the basis. The requirement and possibilities are shown in this work. It aims for a more personalized and decentralized approach in helping patients. In particular, Wac [3] shows the trend of taking advantage of the technology to reduce the mobilization of patients reducing costs in terms of time, money and resources usage.

Smartphone apps have been played an important role in telehealth care too [9]. Several cases of use are shown such as information and time management, health records maintenance, communication and consulting patient monitoring, among others [10,11,12]. However, the approach shown by Dyer et al. [9] is very simple in terms of patient monitoring. Besides, the communication channels are roughly described. The system does not provide a logging server either, neither a communication server nor a centralized repository. In other solutions, image processing [13,14] and computer vision [14,15] have been tested for monitoring patients, but the design has not been implemented to be used for end users. In these cases, the main goals are not providing customized and personalized assistance to patients with disabilities or similar needs.

There are several attempts of recognizing automatic suspicious behaviors in patients by using image processing and computer vision [16]. This work has been very specialized and still needs improvements. Especially for movements that are supposed to be normal but follow after other ones that can mean a potential problem for the patient. Experimental trials with deep learning solutions [17] appear to be promising. The tradeoff is the computational costs and the amount of data required to create a proper learning model that classify correctly the rough movements of a patience.

Streaming multimedia content in healthcare systems over Internet/Intranet [18] has also been a key component in the last decade. A theoretical framework proposed by [19] to have distributor system that delivers the multimedia contents in the healthcare system to doctors and professionals involves a number of different features; however, social schedule and a reliable communication system are not fully functional. There is no monitoring module either, and still lack of many concepts to build a monitoring system as the one presented here. Furthermore, the use of social media apps was not been fully exploited in any of the related projects. The use of a social network integrated in the social agenda is not present in the telehealth care systems.

According to the World Health Organization, falls are the second most common cause of accidental or unintentional injury deaths worldwide. And a staggering 37.3 million falls are severe enough to require medical attention each year. A wide range of factors contributes to fall risks, such as environmental hazards, illness, neurological, mental conditions, muscle weakness, poor vision, and medication [10,20,21,22,23]. While there is no such thing as a $100\%$ effective fall detection or prevention solution, most experts agree that a combination of available sensor technology can greatly increase system reliability. And there is a growing acceptance by seniors as monitoring is a means to extend their independent living opportunities [21,23].

There is a system created and funded by European Agency: Ministry of industry, tourism and commerce, Spain. This project gets different users connected via smartphones and shares contents of TV programs that are currently on [24]. This is a way of sharing TV contents that are on and also of filtering the commercials based on the profile of the user. This system was not intended to be applied to telehealth care, although the technology and the idea can be used in the communication between the professional and the user to build a social agenda. Another application in telehealth care granted by the Ministry of industry, Energy and Tourism is introduced in Castellanos [25]. The system still requires a lot of improvements such as computer vision, multiple user interaction, and a complete social schedule. These three ideas can contribute with a major change in traditional tele treatment approaches. These three are in the top of the emerging and promising technologies and can improve dramatically the monitoring and the living quality of these minorities during the rehab process. Moreover, it is a small project, used as a demo in hospitals in different regions intended to monitor a small number of patients. This paper targets a bigger number of monitored patients by updating the resources of the communication servers. This is essential not only to increase the number of end users but also to keep technologies such as machine learning algorithms and computer vision running efficiently.

A novel algorithm for biomedical image segmentations is introduced together with an algorithm for biomedical image feature selection in Zhao and Belkasim [26]. Biomedical image segmentations and feature selection are foundations to detect and predict diseases. Discovering biomedical knowledge from medical text documents is a critical means for advancing medical practice and clinical outcomes in patients. A novel domain specific methodology is built to identify the word meanings in medical documents characterized by different domains [27]. This methodology uses a domain fusion algorithm to narrow domain concepts from different domain knowledge and avoids unknown domain problem; it can be effectively used for both biomedical information retrieval and text mining. An approach based on image compression and deposing scheme is shown in  Zhao and Belkasim [26], Zhao et al. [28]. Other authors [29,30] present another proposal done for digging the knowledge that hidden in a big amount of text data. This proposed project needs a further research on social network [31] and image retrieval in Internet [32] in such a way that they can contribute to it, along with image processing methods [33,34]. In addition, the biomedical data usually has uncertain information [35] that are indeed very helpful for this project [7,36,37,38,39,40,41,42].

3. Methodology

Several challenges have been arisen, key issues based upon economic, technological and social sides are briefly described.

System Development Cost: Apart from the required medical issues, enabling the population to benefit from the information and communication technologies generates additional costs. Both technologies are core infrastructure and a large amount of budget is required to invest in necessary computer equipment and developing tools. Hardware and software installation and maintenance together with qualified manpower and knowledge transfer demonstrate one-time and periodic investment.

System Implementation: Telemedicine affects traditional medical care processes and several social concerns may arise. Key concerns including legal, regulatory, security and human resources issues together with successful implementation of the telemedicine services are addressed. Building a telemedicine service is therefore not only about completing the system implementation but also about the impacts of system utilization.

Diagnostic Accuracy: Accuracy of the diagnosis is one of the key concerns especially when the physicians and the patients are at different locations. Face-to-face medical care usually brings conffidence to patients, especially the elderly. However, a short period of medical interrogation and vital sign measurement performed at a hospital may not reflect current symptoms due to the white coat syndrome. Physicians may obtain more accurate results by applying continuous health monitoring based upon electronic healthcare approach.

Telehealth care can be used to deliver medical services regardless of distance and time via information and communication technologies. Feasible opportunities of telehealth care include, among others, cost reduction, preventive medicine promotion, medical education, healthcare equality, service diversity, etc.

3.1. Architecture Principles

Architecture principles define the underlying general rules and guidelines for the use and deployment of all Information Technology (IT) resources and assets across the enterprise. They reflect a level of consensus among the various elements of the enterprise, and form the basis for making future IT decisions, see [43]. Each architecture principle should be clearly related back to the business objectives and key architecture drivers.

Principles are general rules and guidelines, intended to be enduring and seldom amended, that inform and support the way in which an organization sets about fulfilling its mission. In their turn, principles may be just one element in a structured set of ideas that collectively define and guide the organization, from values through to actions and results. Depending on the organization, principles may be established at any or all of three levels:

• Enterprise principles provide a basis for decision-making throughout an enterprise, and inform how the organization sets about fulfilling its mission. Such enterprise-level principles are commonly found in governmental and not-for-profit organizations, but are encountered in commercial organizations also, as a means of harmonizing decision-making across a distributed organization. In particular, they are a key element in a successful architecture governance strategy (see Architecture Governance).
• Information Technology (IT) principles provide guidance on the use and deployment of all IT resources and assets across the enterprise. They are developed in order to make the information environment as productive and cost-effective as possible.
• Architecture principles are a subset of IT principles that relate to architecture work. They reflect a level of consensus across the enterprise, and embody the spirit and thinking of the enterprise architecture. Architecture principles can be further divided into:

  –

  Principles that govern the architecture process, affecting the development, maintenance, and use of the enterprise architecture

  –

  Principles that govern the implementation of the architecture, establishing the first tenets and related guidance for designing and developing information systems

These sets of principles form a hierarchy, in that IT principles will be informed by, and elaborate on, the principles at the enterprise level; and architecture principles will likewise be informed by the principles at the two higher levels.

It is useful to have a standard way of defining principles. In addition to a definition statement, each principle should have associated rationale and implications statements, both to promote understanding and acceptance of the principles themselves, and to support the use of the principles in explaining and justifying why specific decisions are made [43].

In the context of the paper, see Figure 1, three major profiles are defined:

• Final user: This is the main agent of the product. These are those minorities affected by chronic diseases, people with disabilities and special needs, or with any type of dependency.
• Client: It is any entity, association, Pharmaceutical laboratory, Hospital or Health center that Implants the system to provide assistance to end users.
• Professional. Any person providing the service (volunteers, social workers, health workers, etc.).

3.2. Cases of Study

Following are the different scenarios the system considers:

• Scenario 1: The professional receives an alert on their device with a motion alert that a rough falling movement has occurred. It activates the communication between the professional and the patient and check the status of the user.
• Scenario 2: The professional schedules the alerts that reminds the user to take their medication: The user visualizes and listens to this alerts on their TV. The alerts can include the visualization of recommended videos explaining more about the benefits of the medicines and more recommendations.
• Scenario 3: The user need to contact with other users. The professional can contact him and provide networking tools with other users in the same situation and can share experiences through the use of multimedia content, if that is what they decide. This is supervised by the professional.
• Scenario 4: The device installed at the user’s home has relevant information from them, such as date, body temperature, and next appointment with the professional. They can check the device or that can be always present in the corner of the TV.
• Scenario 5: The user’s family can work together with the professional to recommend audiovisual contents in the system for the user.
• Scenario 6: The professional contact the final user (patient) to check on his daily exercises. He supervises online the movements of the user during the rehabilitation process.

3.3. System Components

The system is a combination of different assembled parts (some of them, can be disabled depending on the needs of the user):

• Recommendation module. A customized menu of options of recommended action to their users based on their preferences.
• Monitoring module: User is monitored directly (with a device in the smartphone) and indirectly (with cameras and external devices integrated in TV). There are movement detection and monitoring of the user’s movement that detects rough motion, such as falls. (Computer vision module).
• Tele Rehab: The rehabilitation to recover the mobility with the help of supervisor/professional
• Tele dosage: The Assistance can be focused on the medicine intake and how to control the dosage by the professional through the system.
• Social scheduling: A customized program of contents for the user by the system agents (family, volunteers, social agents, professionals and so on) The Architecture of the system is intended to be client /server. The professionals will use a webapp and the users AppAndroid. Both will connect to the backend part with a specific Application Programming interface. (API REST)

3.4. Technical Components (Used Technology)

A brief technical description of the system is found below, see Figure 2.

• Backend + API REST: The system backend will have an Application Programming Interface available, that centralizes and coordinates all systems actions from the server side. Such API is able to provides an interface for the communication. For the frontend initially, the project use Python, Django and Tastypie.
• Webapp: The web portal for the professional management is implemented by using HTML5, CSS3, JavaScript, jQuery and possibly Twitter Bootstrap. The communication with this part is managed by the output objects from the API.
• AppAndroid: AppAndroid for the user is installed in the device connected to the TV (set top-box) and/or smartphone, they are linked to the API with communication objects (possibly JSON) same way as the Webapps does.
• Database Server (Backend): The database is a relational one, initially MySQL, although the structure might change to a BI model depending on the number of signals to be stored for any user. This is located in a dedicated stand-alone server and accepts Secure-Shell connections (ssh command line utility). In the future, we might consider the option to migrate to a SQL Server (stand-alone) server depending on the growth of data and the needs for storing it.
• Communication Server: The communication server will keep track of the communication flow and logs the activity between the different components. Elastix technology is a good candidate as it allows VoiP, PBX emails and twits and it does not rely on external sources to process and modulate the calls. Elastix also uses SIP as standard interactive multimedia calls such as video, voice o messaging.

The Client is sending a Session Initiation Protocol (SIP) package to the User Agent Client (UAC) that will initiate the User Server Agent (UAS).

The Proxy server SIP Proxys will reroute the SIP to the UAS and returns the SIP to the UAC. Any request can go through different proxy servers to the UAS for allocating and redirecting server to locate the user and to notify to the rest of the components. This process is the ideal way of communication, although we might consider different alternatives. The system also provides a logging server that logs the physical address. The physical and logical address is associated in a routing table located in the logging server. Internet calls will use SIP, Android client ImsDroid (Open source software), Client Html5 Sip and Server WebRTC2SIP. For signal processing in the communication server we use an experimental simulation of a massive parallel bioinspired system, which will allow to work on new approach for mobile computing. Initially will try P-system [8] architecture in conventional hardware.

The methods can be applicable in an integrated platform of visual and communication devices. The hardware components for the system agents are:

• Dedicated server for training the Computer vision mode: Desktop Server 2 GPU units for training a Convolutional Neural Network (CNN).
• Standby Server part: Remote server for logging communication messages and to synchronize the system
• Professional module: Video camera, Smart device
• User module: Video camera, Boxes, TV or any other domestic smart device, smartphone

The target system requires an internet connection and as many user modules as patients who will to use the system, see Figure 3.

4. Details of Interaction

Users are engaged in a passive (1) and active way (2).

• While watching TV and being monitored by the professional on the camera inserted on the TV device. The professional contacts the patient to check on progress of the rehabilitation process.
• The user actively contacts either the professional or the other users who are part of the social schedule module of the application integrated on the TV box.

The monitoring will be done by using computer vision techniques. A Convolutional Neural Network (CNN) is a deep learning algorithm that can recognize and classify features in images for computer vision. It is a multi-layer neural network designed to analyze visual inputs and perform tasks such as image classification, segmentation and object detection. The software used is Keras which is used as a friendly user interface for using tensor flow library. This contains numerous algorithms for building a convolutional network that are designed for computer visions. The idea is training the CNN with images of a user failing. Initially 15,000 pictures of possible falls are part of the training process. The camera feeds pictures to the computer vision’s algorithm.

4.1. Monitoring with Computer Vision Details

The multi model training system to dynamically train the captured images. People suffered from different diseases/illness may have different ways to fall and stay; different training models have different advantages. Our multi-model training system will use several popular machine learning methods, such as support vector machine, k-means clusters and deep neural network to train the captured images, then combine the results to get the final decision. This process is dynamical, it focuses on the people and the environment. Deep neural network has been widely applied. Although deep neural network has good performance, the training process is still black box. Support Vector Machines (SVM) and K-means clustering are traditional methods whose mathematics background have been well studied. Through comparing the results from SVM and K-means clustering with the ones from deep neural network [44,45], we may get more insights of the mathematics principles hidden in the deep neural network, which can contribute to the improvement of deep neural network as well as the machine learning. In our case, there are only one scenario only to consider, making the computer vision approach fairly fairly simple. Training is done by capturing a series of images from the users that simulate a fall and stay in the floor for different users and taking serially generated images of falling from internet too.

The elective system is using a convolutional network with the following stages:

4.1.1. Training Phase

There will be both simulations and images scanned by different users falling and reuse of other images and videos of falling individuals. During that phase we will train the model by initially using 15,000 images of a person falling. During the training the system will take into account the position of the person falling and the speed of movement, configuring the system for alerting when the speed of the motion drastically changes and the sequential positions generated by the individual fit one of the predefined falling sequences positions defined in the filter of the convolutional layer of the CNN.

The CNN architecture, see Figure 4 will follow a standard that can be adjusted to optimize fitness values.

The input layer will take 3 depth images 280 × 280 × 3. The image is classified as falling/not falling (2 classes, or labels). This classification is done done manually for 15,000 pictures (3-dimensional vectors).

4.1.2. Testing Phase

For testing initially, we considered to use a cross validation approach with a data set split parameter $\nu =0.2$. (corresponding to a split into 5 datasets of 3000 pictures each and use 12,000 pictures for training and 3000 for testing in each round, 5 rounds total). This process returned an acceptable rate of hits, detecting successful falls with accuracy (true positives) of 72.2%. False negatives are only 8% of the distribution.

5. Evaluation and Validation of the Developed System

The performance of proposed fall detection system was tested in a real home environment. An external camera is used for recording the real video sequence with the image size of 280 × 280. The recorded video sequence is processed by using OpenCv and Keras for the CNN training.

For evaluating this fall detection system, each volunteer (student) is asked to simulate different falling and non-fall activities in different directions. Fall is detected when the lying posture is classified as falling by the CNN, see

Table 1. Experimental results show the proposed CNN classifier achieves a high fall detection accuracy.

Activity	Number	Falls	Nonfalls	Success Rate
Falls	98	74	24	75%
Walk (nonfall)	92	1	91	75%
Sit (nonfall)	86	30	56
Bend (nonfall)	132	45	87

. The proposed CNN based fall detection systems could accurately detect falls (75%) with a low false alarm rate (25%). The misclassifications are attributed to the reason that the bend posture in some directions is viewed similar to the lie posture and we can adopt multiple cameras to capture different posture views for ameliorating it.

The accuracy of the CNN is bellow 75%, some preprocessing effort could be made to optimize CNN structure and improve the results.Yu et al. [46] obtains a better performance but the images need to be preprocess in order to generate a silhouette. From a recorded video stream, background subtraction is applied to extract the human body silhouette. Extracted silhouettes corresponding to daily activities are applied to construct a convolutional neural network, which is applied for classification of different classes of human postures and detection of a fall event.

Rates (72.2%–75%) have been the maximum accuracy rate reached considering the standard CNN topology and a variable data set split factor (0.15–0.28). Results suggest that with these numbers of images the accuracy might not be able to improve considerably. Thus, more training images would be needed to ensure a more reliable system. Nevertheless, considering the number of training observations, and the outcome of using the cross-validated test, this module clearly adds value to the entire system.

In reality, only part of falling human body is included in some images, this technique detect the falling events provided a 80% of body is shown in images. A future research includes information from multi-modality sensors together with deep learning techniques for developing a more robust fall detection system in case the whole human body is not present in the images.

6. Conclusions

This paper has introduced a system designed for helping in remote medical treatments by offering tele assistance and tele support through different modules of the system. In particular, wireless sensors and capturing devices are integrated by collecting biosignals and images from the users at the same time. The biosignals are able to help the professional in being warned about the biometrics of patients and the images are able to catch a possible fall of the patient right in the moment. Apart from that, systems like this offer a new possibility for remotely treat patients with technological solutions that integrate wireless sensors, Desktop Servers, smartphones, Machine learning algorithms, and traditional databases. Wireless devices together with Machine Learning algorithms will play an essential role in the next generation of telehealth care applications. Future work includes exploitation information from multi-modality sensors (such as audio and video sensors) together with deep learning techniques for developing a more robust fall detection system.