New Advances and Challenges of Fall Detection Systems: A Survey

Abstract

Falling, as one of the main harm threats to the elderly, has drawn researchers’ attentions and has always been one of the most valuable research topics in the daily health-care for the elderly in last two decades. Before 2014, several researchers reviewed the development of fall detection, presented issues and challenges, and navigated the direction for the study in the future. With smart sensors and Internet of Things (IoT) developing rapidly, this field has made great progress. However, there is a lack of a review and discussion on novel sensors, technologies and algorithms introduced and employed from 2014, as well as the emerging challenges and new issues. To bridge this gap, we present an overview of fall detection research and discuss the core research questions on this topic. A total of 6830 related documents were collected and analyzed based on the key words. Among these documents, the twenty most influential and highly cited articles are selected and discussed profoundly from three perspectives: sensors, algorithms and performance. The findings would assist researchers in understanding current developments and barriers in the systems of fall detection. Although researchers achieve fruitful work and progress, this research domain still confronts challenges on theories and practice. In the near future, the new solutions based on advanced IoT will sustainably urge the development to prevent falling injuries.

1. Introduction

Falling is one of the major public health problems in the world. An estimated 646,000 fatal falls occur over the world each year, making falling the second leading cause of unintentional injury death [1]. It is a dangerous vital action and not easy to prevent for the elderly, since older people gradually cannot control their action smoothly as they grow older. The rapidly ageing population enhances this threat to public health. A latest report from World Health Organization (WHO) indicates that adults that are older than 65 years of age suffer the greatest number of fatal falls, and death rates caused by falls are highest among adults over the age of 60 years [1].

Fall detection is an important and significant research direction in the study of falls. The time that the elderly remains lying on the floor after falling is one of the key factors that determines the severity of a fall [2]. Timely detection of falls enables immediate assistance by caregiver and minimizes the negative consequences of falls [3]. A robust fall detection system monitors the fall and provides alerts, which could alleviate the burden of caregivers and resource-strained health care systems.

Although fall detection is promising, it faces many challenges. A generally accepted definition of fall is as an event which results in a person coming to rest inadvertently on the ground, floor or other lower level [1]. However, it is difficult to distinguish a fall from other actions based on this definition, since there are many similar actions. For example, the most confounding activity to falling is lying [4]. Another issue is that it is not easy to obtain the real data in the daily life, since it is a kind of random, accidental and dangerous action. Therefore, it is a daunting task for researchers to build datasets with real fall data to study and analyze. Furthermore, due to different kinds of sensors used in fall detection, there is a lack of a universal solution to this problem. To obtain a global view, a comprehensive survey should be conducted. Some researchers have done such meaningful work of literature reviews before 2014 [2,5,6,7,8,9]. In recent years, the related research increases dramatically with the advent of new sensors and technologies. However, all the reviews were done several years ago; there are few surveys to review the development of fall detection research during recent years.

The rapid development of smart sensors and the Internet of Things has brought new vitality to fall detection research and produced a series of research results. To review developments in this field after 2014, firstly, we comprehensively searched with related keywords such as “fall detection” using Google Scholar, and exacted the twenty most cited paper from 6380 related papers. Then, based on these most influential papers, we reviewed the tendency into three main aspects: revolutions of sensors, changes of fall detection algorithms and performance of fall detection systems. Finally, we summarized issues and challenges emerging in this field and provided some future direction in this field.

The rest of this paper is organized as follows. Related work is provided in Section 2. Section 3 analyzes the tendency of development of fall detection after 2014. Following the analysis, the challenges, including four points, are outlined in Section 4. Finally, the paper ends with a discussion of the conclusion and future work.

2. The Related Work on Prior Surveys

In this section, we discuss prior surveys on the topics of fall detection briefly. Fall detection cannot be considered as a brand new hot field, since it has been explored in the last two decades and has evolved rapidly. Several researchers have reviewed and assessed the prior research work of fall detection, and presented the ideas, theories, and the significant literature published from various perspectives before 2014.

The first review paper on fall detection was proposed by Noury et al. [5] in 2008. Two years later, Perry et al. [8] reported existing fall detection methods that have been found effective by others, as well as documented the findings of their experiments, the combination of which would assist in the progression towards a safe, unobtrusive monitoring system for independent seniors. In 2013, Mubashir et al. [6] gave a survey on different systems for fall detection and their underlying algorithms, and divided fall detection approaches into three main categories: wearable device based, ambience device based and vision based. These approaches were summarized and compared with each other and a conclusion was derived with some discussions on possible future work. In the same year, Igual et al. [2] reviewed the fall detection systems in an extensive literature review of fall detection systems, including comparisons among various kinds of studies. It aimed to serve as a reference for both clinicians and biomedical engineers planning or conducting field investigations. Some challenges, issues and trends in fall detection were identified after the review. In 2014, Delahoz et al. [9] thoroughly reviewed and qualitatively compared fall detection systems and fall prediction systems, in terms of design issues and other parameters. In 2014, Habib et al. [7] presented a survey focusing on smartphone-based solutions for fall detection and prevention.

The research work aiming at fall detection has increased dramatically in recent years. There are thousands of papers related to fall detection published after 2014. However, there is no updated survey paper to review the great changes and the tendency of this field in this period. According to a search using Google Scholar, we carefully selected twenty influential papers that have been cited at a high level in this field since 2014, all of which have been cited above 30 times. We hope that this work could reflect the development and the trend for fall detection via analyzing these representative research works.

3. Bird’s Eye View of Fall Detection Research from 2014

Since fall detection is an interdisciplinary research problem, data were collected from Google Scholar in this review, which is one of the most useful and popular tools in research community, covering almost all research domains. To overview this research domain, we comprehensively searched Google Scholar with the keyword search terms: “fall detection”, “fall prediction”, “health care”, and “elderly care”. Data collection was conducted based on Google Scholar for the period 2014–2017. Finally, a total of 6830 papers were collected and used for extraction. We ordered them by the cited counts and chose the top twenty most cited paper as the influential and representative work to profile a technical roadmap of fall detection system in the past years.

Table 1. Twenty most cited papers on fall detection published from 2014.

Reference	Sensors	Algorithms	Accuracy	Counts of Citations	Time
Stone and Skubic [10]	Kinect	Decision Tree (DT)	N/A	195	2015
Mastorakis and Makris [11]	Kinect	Threshold and Shape Features	100%	191	2014
Wang et al. [12]	Wi-Fi	Random Forest Algorithm (RFA)	94%	122	2017
Gasparrini et al. [13]	Kinect	Ad-Hoc Segmentation	N/A	83	2014
Ma et al. [4]	Kinect	Shape Features and Artificial Neural Networks (ANN)	86.83%	65	2014
Wang et al. [14]	Accelerometer	Thresholds	97.5%	59	2014
Kwolek and Kepski [15]	Accelerometer and Kinect	Support Vector Machine (SVM)	98.33%	57	2014
Bian et al. [16]	Kinect	Decision tree (DT)	97.60%	55	2015
Su et al. [17]	Doppler Radar	Discrete Wavelet Transform (DWT)	89.37%	49	2015
Aguiar et al. [18]	Accelerometer	Decision Tree (DT)	97.55%	46	2014
Kau et al. [19]	Accelerometer	Discrete Wavelet Transform (DWT)	92%	44	2015
Castillo et al. [20]	Accelerometers and Video cameras	Decision tree (DT)	79.57%	43	2014
Kwolek and Kepski [21]	Kinect and Accelerometer	k-Nearest Neighbor (kNN)	100%	40	2014
Pierleoni et al. [22]	Accelerometer	Support Vector Machine (SVM)	95.19%	40	2015
Li et al. [23]	Kinect	Nonnegative Matrix Factorization (NMF)	98%	38	2014
Chua et al. [24]	Video cameras	Shape Features	90.50%	35	2015
Amin et al. [25]	Doppler Radar	Discrete Wavelet Transform (DWT) and Support Vector Machine (SVM)	N/A	34	2016
Lee et al. [26]	Accelerometer	Thresholds	N/A	33	2015
Kwolek and Kepski [27]	Kinect and Accelerometer	k-Nearest Neighbor (kNN) and Support Vector Machine (SVM)	95.77%	31	2015
Liu and Lockhart [28]	Accelerometer	Thresholds	97.83%	31	2015

shows the results ordered by counts of citation from high to low (note that citation counts dynamically change, and the data used in our paper were collected on 26 December 2017).

The top three most cited studies have been cited over 100 times, especially the third one [12], which has only been published less than one year but has been cited 122 times. It is amazing even through the entire computer science domain. This phenomenon not only reflects how important this work [12] is, but also indicates how fast and popular the fall detection research develops. Therefore, our survey is important and necessary for an overview of the ideas, theories, and significant literature in this field. In the remaining sections, we will present a detailed analysis on the research status and development trends of this research field from three aspects: sensors, algorithms and performances.

3.1. Evolution of Sensors

One of the significant progresses in fall detection system is the use of advanced sensors. In recent years, sensor technology has developed remarkably, which can perceive different kinds of information from the environment more precisely and conveniently than ever before. Researchers have always attempted to adopt new devices to help them detect a fall. According to types of sensor, Mubashir et al. [6] divided fall detection algorithms into three main categories: wearable device based, ambience device based and vision based. Igual et al. [2] roughly categorized them into two types: context-aware systems and wearable devices.

After analyzing the situation of sensors used in recent years, we have divided them into three main categories: vision-based, accelerometer-based, and Radio Frequency (RF) sensor-based. The first category refers to the visual sensors including the normal camera and the depth camera, such as Kinect. The Kinect sensor features an “RGB camera, depth sensor and multi-array microphone running proprietary software”, which provides full-body 3D motion capture, facial recognition and voice recognition. In contrast to RGB camera, it can get 3D image and video according to infrared projector and a special microchip. It can distinguish the foreground from the background directly, while a normal camera cannot. In our work, camera refers to the normal RGB camera or webcam. The second category includes specialized accelerometer devices and accelerometers built-in the smart phone. The last one covers Wi-Fi and Radar. We collected types of sensors used in fall detection system from most cited related papers before 2014 (the most cited paper’s data obtained from [2]) and compared data from the most cited related papers after 2014. As shown in Figure 1, we compare the trends of used sensors before 2014 and after 2014. Since some studies have used both Kinect and accelerometer, we plot the figure and regard it as a standalone sub-category in Figure 1b to present it more precisely. According to Figure 1a,b, we conclude four main changes from the period before 2014 to the period after 2014.

First, the numbers of fall detection systems using camera declined a lot after 2014, which was second place before 2014. Recently, sensors associated with vision-based technology have been developing rapidly. Previous vision-based algorithms always adopted the camera to detect a fall. The advantages of this technology are low cost and easy installation of the camera in home environment. Due to the camera’s natural weakness of merely capturing 2D image, it is difficult to distinguish the foreground from the background, and it is sensitive to light variation. Therefore, fall detection based on camera lacks high robustness. Due to camera’s inherent weakness, that is, privacy preserving, it has been used less in the indoor environment recently. Chua and coworkers’ work [24] is the only one of twenty most cited the papers published after 2014 that employed the camera to detect a fall.

Second, statistical results indicate that Kinect replaced the camera and became the most popular type of sensor used in fall detection systems after 2014. It (including Kinect, as well as Kinect and the accelerometer) accounts for almost half of the total systems. The emergence of the depth camera, such as Kinect, overcomes some weakness of traditional cameras. It can provide full-body 3D motion captured by an infrared projector, a camera and a special microchip. Since Kinect is less affected by lighting conditions, Mastorakis et al. [11] used this feature to propose a real-time fall detection system. Gasparrini et al. [13] put Kinect on-ceiling and let it face downwards to detect fall based on analyzing the raw depth data, which provided an automatic, privacy-preserving fall detection method for the indoor environment. Similar to using a ceiling-mounted 3D depth camera, Kepski et al. [21] analyzed the potential fall action by depth images. Kwolek et al. [27] supposed that the available algorithms based on Kinect for fall detection are neither robust nor exhibit high sensitivity and specificity. They proposed an architecture to combine the accelerometer and Kinect. The accelerometer was used to indicate a potential fall, while Kinect was used to authenticate the eventual fall alert. Based on a similar idea, another work studies fall detection based on combining an accelerometer and Kinect [15]. It achieved good results according to different technical solutions.

Third, the accelerometer firmly holds the second position after 2014. It also has an explicit change: accelerometer usually referred to a kind of specific device before 2014, while it mostly referred to the built-in component in the smart phone after 2014. Based on a smart phone and the third generation (3G) networks, Kau et al. [19] proposed the fall accident detection and corresponding wide area rescue system. Aguiar et al. [18] proposed a fall detection system based on smart phone. When detecting a fall event, the user location is tracked and notifications Short Message Service (SMS) and email are sent to a set of contacts. Shen et al. [29] proposed a fall prediction system on smartphones. It could record users’ gait datasets, and predict users’ fall risk based on gait statuses. Although accelerators can only get motion information of part of the body, they are still often used in fall detection systems. It reflects that the researchers put ease of use in an important position. Furthermore, since the accelerometer is widely built in smartphone now, it can achieve fall detection and danger information alarm all in one device.

The last important evolution with regard to the type of sensor used after 2014 is the adoption of Wi-Fi or radar as sensors to detect the fall. It does not use a new sensor but investigates fall detection from a brand new angle. The main idea of this kind of fall detection method is using wireless techniques to identify environment change and building the relationship between the wireless signal and human activities [12]. It not only allows users to perform daily activities naturally and continuously without wearing any devices on the body [30], but also protects users’ privacy. This provides a new road leading to future.

3.2. Changes of Fall Detection Algorithms

Fall detection algorithm is another deterministic factor in fall detection systems. As sensors advance, fall detection algorithms change. We reviewed fall detection algorithms adopted in most cited work since 2014, and found the tendency by comparing with the algorithms most cited work before 2014. Figure 2 shows the comparison of changes of between the period after 2014 and before 2014. We compared this tendency from a coarse view to a fine-grained perspective. The main categories of fall detection algorithm are presented firstly. It should be noted that the conventional algorithms that we defined include threshold-based, rule-based and shape-based algorithm.

With regard to fall detection algorithms used before 2014, we used the data on most cited paper on fall detection in Igual and coworkers’ previous survey paper [2]. As mentioned previously, the accelerometer was a mainstream sensor used by fall detection system in early days. Since it can only get velocity and acceleration of one part of human body wearing devices, the thresholds-based method became the preferred method. For camera-based methods, the fall detection algorithms always adopted machine learning. In Figure 2a, these two kinds of algorithms were almost equally adopted in fall detection systems. The fall detection method based on machine learning is not a general solution, thus diverse methods were adopted, as shown in Figure 2c.

For fall detection algorithms used after 2014, the fall detection algorithms based on machine learning become a new mainstream, as shown in Figure 2b. Since the increased new types of advanced sensors can perceive more detail of human activities, the threshold-based algorithm is inadequate to achieve the goal. From the aspect of the specific type of algorithm, Support Vector Mechine (SVM) and Decision Tree (DT) are still the first two most used algorithms, same as the situation before 2014. Based on Kinect, Stone et al. [10] used DT to compute a confidence that a fall preceded on a ground event. Using an accelerometer built-in smartphone, Aguiar et al. [18] chose DT for retrieving features and threshold information to detect a fall as well. Using accelerometer, Pierleoni et al. [22] employed SVM to select acceleration thresholds. These algorithms not only trade off performance for sensitivity and specificity, but also are appropriate and effective for small datasets.

Furthermore, with the introduction of RF sensors, such as Wi-Fi and Doppler radar, digital signal processing methods have been employed in fall detection systems, such as Wavelet transform (WT). According to a ceiling mounted Doppler range control radar, Su et al. [17] used WT to detect human falls. Amin et al. [25] used the signal processing algorithms and techniques involved in elderly fall detection using radar as well. In addition, Wang et al. [30] presented a real-time, contactless, low-cost accurate indoor fall detection system, named as RT-fall, using the phase and amplitude of the fine-grained Channel State Information (CSI) in 2017. Just after a few months, this work has been quoted for more than 20 times. Although is not one of the top twenty most cited papers after 2014, due to being published recently, it has a significant impact on others. These works based on RF sensor indicate a new way differing from the past, showing a kind of future direction of health care in-home.

Finally, there is an interesting phenomenon, that is, deep learning as one of the popular machine learning algorithms, has not been adopted in the twenty most cited papers. We assumed that the one of main reasons is the size of the datasets of fall detection are small, which limits deep learning’s performance.

3.3. Performances of Fall Detection Systems

To evaluate the performances of fall detection systems, the accuracy of fall detection algorithm is a significant indicator. Figure 3 shows the accuracies of most cited papers since 2014. Some algorithms are tested on several different datasets (e.g., [17,18,22]). We compute the accuracy of this algorithm in this system by mean of all accuracy values working on all datasets. In addition, some papers did not indicate the accuracy of fall detection system (e.g., [10,13,25]), or just provided sensitivity and specificity (e.g., [26]).

The accuracy of fall detections from selected most cited works are relatively high. As observed in Figure 3, it is found that most fall detection algorithm accuracies are above 90%. Accuracy of the work by [11,31] even reach 100%, which means that all falls were accurately detected in their testing scenarios. The lowest one is in [4], which reaches 79.57%. Note that most of these experiments are evaluated on a self-captured or simulated dataset.

However, we did not find a successful fall detection application based on any kind of sensors in the market. We assumed that there are two reasons for difficulties of applying fall detection to the real world.

First, most existing fall detection datasets are simulated datasets. Only a few work added limited real fall actions to dataset (e.g., [10], which included nine naturally occurring resident falls, from four residents). Fall is an occasional dangerous human action. Real fall is a kind of unpredictable action, so it is hard to build a dataset associating with real data. Researchers attempt to solve this problem. For example, some studies divided fall into several sub-categories: forward, backward and sideward falls, as well as mimicking them by stunt men or volunteers, respectively. Nonetheless, simulated data are subtlety different from real data. Another issue is that datasets include limited human actions, such as sitting, walking and lying. Researchers can only classify the fall from the other few kinds of action. However, real life is not like that: there are hundreds or thousands of different kinds of actions. Therefore, these fall detection algorithms with high accuracy in lab performs are unsatisfactory in practice.

Second, the current features of fall cannot profoundly describe the fall action. Fall detection can be considered as a classification problem, which distinguishes the fall from other actions. The core point is to define the fall based on different features. For example, the threshold-based method used the feature to detect a fall: the speed of a fall differs from other actions. Nevertheless, this method is not robust for classifying the actions being similar speeds with the fall, such as jumping. This problem will be stated in details in the next section.

4. Challenges and Potentials

Although researchers have made fruitful efforts on fall detection and the technology is promising, challenges and issues still exist. We classify them into four main points, which are from the perspectives of the definition of the fall, application, privacy and the future direction.

4.1. How to Define the Fall Based on Universal Features

Before solving a problem, the predominant and antecedent step is to define the problem. The definition of the fall determines in which way the researcher detects the fall, and then impacts which sensors are deployed and used, which algorithms are employed, and how the system is built. This problem is difficult to address due to the lack of knowledge on the complexed dynamics model of the fall. Some of researchers have attempted to define human’s fall action based on thresholds, shapes or rules. Naturally, they proposed these conventional methods: threshold-based, shape-based or rule-based algorithms. To distinguish a fall from other actions and process complex information, researchers use methods based on machine learning and extract some features of the actions which performed well on classifying the fall and other actions. However, it merely reflects one aspect of the fall rather than the essence of the fall, which will make the system sensitive to situations. Once the experiment condition is changed, the algorithm and system cannot be reused any more. Therefore, it is necessary to build the model and analyze fall in a comprehensive way to reflect the essence of fall. In our work, we regarded the fall as a kind of imbalance action from the view of biomechanics. In this way, the fall does not require to be categorized into specific actions, but the mathematical model is built to define the fall. Based on this idea, the tentative study has already worked in progress [32,33].

4.2. How to Apply the Theories and System to Reality

Most of the fall detection algorithms perform well in the lab or on simulated datasets. However, few cues show that they could be successfully used in real life. To apply the theories and system to reality, the issues that would be introduced by sensors and algorithms must be considered. For example, Kinect has been adopted by several fall detection systems, due to its ability of easily obtaining 3D information by tracing the human. However, the physical limit of Kinect is obvious: the effective detection distance ranges from 0.4 m to 3 m. If it is used in real life, one Kinect cannot cover an entire room. The deployment of sensors in real rooms should be considered and investigated. Besides, reliable datasets determine the performance of machine learning algorithms such as DT, SVM, and ANN. Most existing datasets are set up by simulated fall data, which may make the system less convincing.

4.3. How to Protect Privacy

From the perspective of users, privacy is another challenge for fall detection, especially when using vision-based algorithms, since no one wants to put a camera in his/her room. This is another reason many algorithms cannot be used in reality. Kinect provides an alternative way. It only collects depth image and skeleton data, in which blurred information has been used to describe human action. Furthermore, the methods to detect fall based on Wi-Fi technology are promising and open a door in the future. It is ubiquitous and device free in contrast to methods based on wearable sensors. However, currently, it is hard to detect falls from several persons simultaneously, due to signal interference.

4.4. How to Move on from Detection to Prediction

Fall detection is the initial step to protect the elderly but not the final one. It can help reduce the interval from falling to rescuing, but it cannot prevent the human from being injured. Regarding the study of fall, the Holy Grail is fall prevention and protection. To achieve this goal, fall prediction must also be investigated, which underlies the study of fall protection. We have attempted and achieved good performance of predicting a fall (91.7% in 333 ms) before an elderly person is injured (hitting the floor) using one existing health care dataset [32].

5. Conclusions

The threat of fall concerns not only the elderly but also researchers working in related areas. As one of the hottest topics in health-care, it has made great progress in several aspects in recent years. We surveyed 6380 papers discussed twenty representative and highly cited works in a profound way, and summarized the developments of fall detection from three main angles:

• Revolution of sensors: Depth camera, especially the Kinect, has been widely used; RF sensors, such as Wi-Fi, appear on the stage of fall detection.
• Changes of algorithms: Machine learning becomes the mainstream in fall detection, but feature extraction does not have an obvious innovation.
• Performance of system: Accuracy remains at the high level; however, it is still far from becoming a practical application.

Four main challenges on theories and practice were discussed in a fine-grained way:

• How to define the fall based on universal features
• How to apply this study to practice
• How to protect privacy
• How to from detection to prediction

Although there are many issues and challenges, as smart sensors and IoT advance and new technologies are introduced, we envision that this field will have a promising future.