Pattern Recognition of Human Postures Using the Data Density Functional Method

Abstract

In this paper, we propose a new approach to recognize the motional patterns of human postures by introducing the data density functional method. Under the framework of the proposed method, sensed time signals will be mapped into specific physical spaces. The most probable cluster number within the specific physical space can be determined according to the principle of energy stability. Then, each corresponding cluster boundary can be measured by searching for the local lowest energy level. Finally, the configuration of the clusters in the space will characterize the most probable states of the motional patterns. The direction of state migration and the corresponding transition region between these states then constitute a significant motional feature in the specific space. Differing from conventional methods, only a single tri-axial gravitational sensor was employed for data acquirement in our hardware scheme. By combining the motional feature and the sensor architecture as prior information, experimental results verified that the most probable states of the motional patterns can be successfully classified into four common human postures of daily life. Furthermore, error motions and noise only offer insignificant influences. Eventually, the proposed approach was applied on a simulation of turning-over situations, and the results show its potential on the issue of elderly and infant turning-over monitoring.

1. Introduction

State-of-the-art techniques on time-dependent biomedical signal analysis significantly exhibit their successful progress both on robust sensor architecture and elegant statistical learning-based approaches [1,2]. Prospective experimental schemes and research in recent years has focused on the interpretations between physiological and functional expressions in terms of biomedical signals from heart rate variations, brainwave monitoring, respiration rates, and so forth [3,4,5,6]. Additionally, several methods of rehabilitation based on statistical learnings have also fulfilled fruitful performances for the recovery of human activities damaged by injury or known diseases, such as heart failure [7], stroke [8,9,10,11], Parkinson’s disease [12] and osteoporosis [13]. Among these applications, the hardware schemes of data acquirement commonly utilized wireless sensors [14,15,16,17], personal portable devices [9,16,18], robotic assistance [19,20,21,22], or other commercial equipment [1,23,24]. The acquired results were then fed into relevant image- or video-based techniques [10,11,12,17,21,25,26] for further human posture recognitions.

Sensor-based approaches often offer the merits of portable convenience and contactless framework [15]. Under the hardware schemes, human activities and motional patterns were detected and recorded by employing inertial sensors [1,2,15,16,17] and/or image sensors [1,23,24,27]. The acquired analog signals were then commonly analyzed using machine learning methods [15,17,22,23,27,28,29,30,31,32]. Thus, the data acquirements of human mobility and activities can have less interventions in the duration of data collection. Informative feature extractions mainly rely on the support vector machine [27,33], K-means clustering algorithm [33,34], or linear discriminant analysis [35], whereas the hidden Markov model was commonly used for human activity recognition [22,31,33,34]. To further increase the accuracy of posture recognition in both industry and academia, the image and inertial sensor fusion is a popular technique, performed by commercial equipment, the Microsoft Kinect [1,10,11,23]. Based on the proposed experimental schemes, the approaches could be categorized as the skeleton-joint-based approach [1,17,24,28,29,30,31,32] and the silhouette-based approach [23,36]. Further combining the technique of virtual reality, these approaches can provide an alternative [10,11,12] for guiding correct motions in rehabilitation and can simultaneously motivate the psychological emotions of users. Although the accuracy of activity recognition can be significantly increased by adding the number of sensors set up on the skeleton points, high-budget apparatus and complex hardware structures might reduce the acceptance from users.

Among relevant methods, contemporary physics-based techniques were also ambitiously developed to find effective solutions on the pattern recognition of biomedical signals. For instance, the Hilbert-Huang transform based on the method of physical energy spectrum analysis [37] can decompose analog signals into several independent modes in frequency domains. Then, the relationship between signal powers and informative features can be further delineated in specific modal spaces, so that the significant features can be extracted according to the importance of the corresponding signal powers. On the other hand, the method of multiscale entropy analysis based on classical thermal statistics [38,39,40] well clarifies the importance of systematic complexity in a developing biological system. The biological system, having higher systematic complexity estimated using multiscale entropy, will exhibit higher initiative to accommodate itself to the surrounding environment or external pressures.

Since the aforementioned methods were established on theoretical fundamentals in both physics and information theory, abundant informative features hidden in a biological system of interest would be sufficiently extracted. Under a similar framework, the data density functional method (DDFM) was erected on the foundation of combining many-particle physics and machine learning methods [41]. To improve the feasibility technically, this article proposes an alternative avenue by balancing the complexity of sensor architecture and the approximation of feature extractions. By utilizing the structural orientation from a single tri-axial gravitational sensor (G-sensor) as prior information and introducing the concept of kernel mapping simultaneously, the time signals can be mapped into higher dimensional physical spaces under the framework of DDFM. The data points are treated as physical particles to measure the cluster significance and similarity in specific physical spaces. Then, the cluster centroids can be estimated using any sophisticated machine learning method so that the most probable cluster number and the corresponding cluster boundaries can be delineated by measuring the local lowest energy levels. Eventually, the human motional patterns can be recognized by the directions of state migrations in the physical space.

2. Materials and Methods

2.1. Theoretical Framework

Differing from the conventional methods of machine learning, we proposed a new method for the high-scale data analysis based on the density functional theory (DFT). The mathematical framework of the DFT, founded on quantum mechanics [42,43], provides an elegant approach for high-scale data systems. In an N-particle system within a three-dimensional (3D) physical space, for instance, all data information could be fully elucidated using a 3D particle probability density function (PDF) [44,45,46] rather than processing 3N-dimensional correlative estimations. In other words, the framework of DFT offers participants an approach to reduce the computational complexity by extracting the data PDF in a system of interest. Therefore, although in an N-dimensional system wherein each dimension includes M data lengths, the conventional methods have to deal with a problem with N × M data sizes; the computational complexity can be reduced by only utilizing N data PDFs under the DFT framework. In scientific applications, the DFT has been utilized in the fields of quantum chemistry, solid state theory, material science, bioscience, molecular dynamics, and so forth [44,45,47,48,49,50]. Based on the theoretical investigations from the DFT, the DDFM further fused the functionality of estimations from machine learnings so that the N-particle system can be assigned statistical meanings, such as significance, centrality, similarity, and so on [15,41]. Once the data PDF n(r) in a system of interest with arbitrary high-dimensions can be sufficiently defined, the data features in the system could be extracted by respectively estimating the corresponding kinetic and potential energy density functionals, t[n] and u[n], under the DDFM framework.

The corresponding theoretical forms in each term of the DDFM in a D-dimensional energy space can be generally expressed as [45,51,52]:

$$t\left[n\right]={\int }_{\mathit{k}\le {\mathit{k}}_F}^{}\frac{{\left|\mathit{k}\right|}^2}{2}d\mathsf{\Omega }\left(\mathit{k}\right)/{\int }_{\mathit{k}\le {\mathit{k}}_F}^{}d\mathsf{\Omega }\left(\mathit{k}\right),$$

(1)

and

$$u\left[n\right]=\frac{1}{2}\int \frac{n\left({\mathit{r}}^{\prime }\right)}{\left|\mathit{r}-{\mathit{r}}^{\prime }\right|}{d}^D{\mathit{r}}^{\prime },$$

(2)

where k is a D-dimensional wave number enclosed by a hyper-volume element dΩ and k_F is the corresponding hyper-Fermi vector. The parameters r and r’ are the observation point and the source point in the feature coordinates, respectively. For approaching the motional patterns of interest, the theoretical form of DDFM in a two-dimensional (2D) physical space can be simplified as [41]:

$$t\left[n\right]=\frac{{\int }_0^{k_F}dk·k·\left(\frac{k^2}{2}\right)}{{\int }_0^{k_F}dk·k}=2{\pi }^2·n\left(\mathit{r}\right),$$

(3)

and

$$u\left[n\right]\cong \frac{1}{2}\sum _{i=1}^N\frac{n\left({\mathit{r}}_i^{\prime }\right)}{\mathit{r}-{\mathit{r}}_i^{\prime }{}_{\mathit{r}\ne {\mathit{r}}_i^{\prime }}},$$

(4)

where the relation between the wave number and the PDF is governed by the D-dimensional space-PDF relation [41]:

$$k_F\left[n\right]=2\pi {\left[D·n\right]}^{1/D}.$$

(5)

The dimensional factor $D=2$ in our case. Full mathematical forms of Equations (3) and (4) can be referred to the algorithm flowchart. For easy programming implementation, the integrational parameter appearing in u[n] of Equation (4) had been merged in Equation (8), as follows. The factor N is the data length and i is the location index of $i$th data point.

It is obvious that the kinetic energy density functional (KEDF) t[n] shown in Equation (3) is directly proportional to the data PDF, while the potential energy density functional (PEDF) u[n] shown in Equation (4) has an inverse correlation with the pair-data distance. In the views of statistical learning, the theoretical forms of the KEDF and the PEDF imply the measurements of data weighting (significance) and the data similarity, respectively [41]. Therefore, the global morphology of the Hamiltonian density functional (HDF) exhibits the corresponding centrality of each data cluster, and that of the Lagrangian density functional (LDF) delineates the corresponding cluster boundaries [15,41]. Theoretically, the mathematical forms of the HDF and the LDF can be respectively expressed as:

$$\mathcal{H}\left[n\right]={\gamma }^{2}t\left[n\right]+\gamma u\left[n\right],$$

(6)

and

$$\mathcal{L}\left[n\right]={\gamma }^{2}t\left[n\right]-\gamma u\left[n\right].$$

(7)

The adaptive scaling factor γ in Equations (6) and (7) was used to resolve the issue of dimensional mismatch occurring in mapping the datasets from original spaces to physical spaces. The form is expressed as [15,41]:

$$\gamma =\frac{1}{2}\frac{\langle u\left[n\right]\rangle }{\langle t\left[n\right]\rangle }.$$

(8)

Consequently, the adaptive scaling factor is simply the ratio of global expected values between the u[n] and t[n]. It should be emphasized that the adaptive scaling factor will be estimated automatically due to the fact that it is an intrinsic characteristic of the system.

2.2. Experimental Framework

Figure 1a sequentially illustrates the flow of data collection and the method of data analysis in the study. In the experimental framework, a single tri-axial G-sensor with a sampling rate of 50 Hz was first adhered to a subject’s chest as shown in Figure 1b and used to detect every change in human posture. Three common postures in daily life are sequentially shown in Figure 1c–e. The characteristic length and weight of the employed G-sensor is about 4 cm and 7 g, respectively. Thus, the employed sensor is suitable for the applications of wearable sensing and personal health care as mentioned. Then, a wireless communication protocol using the technique of radio frequency data buffer (RFDB) associated with Bluetooth 4.0, Xenon [53,54], was adopted for the long-term experiments. Eventually, all well-collected data would be automatically uploaded to a personal cloud database or a portable device for further customized posture analysis.

The procedure of algorithmic execution is illustrated in Figure 2. The analog datasets acquired by the G-sensor in each sensing axis will be mapped into their corresponding 2D Cartesian space in a matter of axis-to-axis. Then, each set of the 2D mapped data points will be sequentially fed into the algorithm to estimate the cluster number $i$ as shown in Step (1). As illustrated in the algorithmic flow, the initial cluster number is set to be one and the initial number will be continuously updated until reaching the stop conditions (see Step (7)). While the cluster number is given, the data PDF in the 2D space will be estimated using any sophisticated machine learning estimator in Step (2). In this case, the Gaussian mixture model (GMM) [55,56,57] was adopted for the following algorithmic estimations. Then, the corresponding KEDF and PEDF can be estimated using Equations (3) and (4), respectively, in Step (3). When i$\ge 2$, as shown in the algorithmic flow, these density functionals become linear combinations of measurements contributed from i clusters. After each estimation of density functionals, the adaptive scaling factor listed in Equation (8) will then be calculated to avoid the issues of scaling deformation occurring in the data mapping procedure (see Step (4)). Therefore, in Step (5), the HDF and the LDF respectively listed in Equations (6) and (7) can be estimated accordingly.

It should be emphasized that the proposed DDFM algorithm provides an avenue to estimate the most probable cluster number in a studying system [41]. Based on the concept of energy stability in physics, the most probable cluster number can be estimated by measuring the turning point in the trend curves of the global Hamiltonian functional (HF). The theoretical form of the global Hamiltonian functional is shown in Step (6). Meanwhile, the DDFM algorithm also provides the stop conditions to automatically terminate the estimation process. As listed in Step (7) of Figure 2, two stop conditions were used in the DDFM algorithm. Generally, the whole estimation process will be terminated when the global Hamiltonian functional reaches a stable value. Additionally, the ill-conditioned PDF estimation was also employed for the termination of estimation to avoid the over-estimation of cluster number. At this stage, the most probable cluster number can be defined and the most probable value is $i-1$, as shown in Step (8). Eventually, the corresponding cluster boundaries can be delineated by finding either the zero points of the LDF derivative or the lowest level of LDF that only encloses one center, as described in Step (9).

3. Results

Five healthy subjects, three females and two males, were recruited in this studied case to simulate the expected human motional patterns. The average age of these subjects is 22.8 years old with a standard deviation of 1.92 years old, and the average height is 161.8 cm with a standard deviation of 7.43 cm. Since the collected signals from the subjects were highly similar, one dataset of signals from a subject was randomly employed in the following study. Figure 3 depicts the original time-dependent signals collected by the tri-axial G-sensor in each corresponding sensing axis. Each numbered principal axis (PA) is correspondent to each sensing axis. In the proposed experimental scheme, two different periodic times, 20 and 40 s, were adopted and the motions were repeated eight times within three minutes. The different periodic times between these studying cases were deliberately arranged in the experiments in order to check the immunity of DDFM from the different experimental circumstances. The signals shown from Figure 3a–c sequentially describe the repeated motional patterns from lying face-up to turning left (TL) in each PA and that from (d)–(f) shows the patterns from lying to turning right (TR), respectively. By considering the hardware architecture shown in Figure 1, the direction of PA 1 was the same as the rotating axis of the subject body so that the signal variation in PA 1 was much weaker than that in both PA 2 and PA 3 in each studied case. The waveforms of PA 3 shown in Figure 3c,f are similar and they also have similar signal levels in the lying states. On the contrary, the directions of waveforms of PA 2 shown in Figure 3b,e are opposite, even though they also have similar signal levels in the lying states. These sequential variations of motional patterns in PA 2 and PA 3 caused by the architecture of the G-sensor constitute the feasible features of the motional pattern recognition in the study and may be used to compensate for the lack of sensor number. Thus, these features were used as the prior information in the DDFM algorithm.

To introduce the DDFM to the time-dependent signal analysis, the datasets were first mapped into 2D pseudo-physical spaces as shown in Figure 4 and Figure 5. Figure 4 and Figure 5 respectively exhibit the classified results of motional patterns of Lying-to-TL and Lying-to-TR states. By comparing these classified results, it is obvious that the difference in the periodic times employed in the experiments would not substantially affect the accuracy of the DDFM algorithm. Meanwhile, the overshooting signals depicted in Figure 3 also only offered insignificant noise levels. As expected, the scattered plots of Figure 4a,b exhibit the ignorable motional patterns in PA 1, whereas that of Figure 4c reveals significant physical couplings between body motions and the rotating directions of the G-sensor. Additionally, according to Steps (6)–(9) in the proposed algorithmic flow, the most probable cluster number and the corresponding cluster boundaries could be well defined by estimating the systematic global Hamiltonian functional and the energy characteristics of LDF morphologies, respectively. The data PDFs in the 2D pseudo-physical spaces were statistically estimated using the GMM method associated with the expectation-maximization (EM) algorithm, and the corresponding trend curves of the global Hamiltonian functional were also delineated in Figure 4d and Figure 5d. The average global Hamiltonian functional was then used to track the most probable cluster number, and the characteristic factor was set to be ε = 0.02HF[n]_i−1 in each case. As indicated by the red arrows in the plots, the most probable cluster numbers in these studied cases all pointed to cluster number two, as expected. It is also noted that the fluctuations that occurred behind the most probable numbers were caused by non-ideal effects [41]. For instance, the GMM-EM algorithm cannot exactly catch the main locations of clusters in the studied cases, as shown in Figure 4f. Additionally, the chaotic changes of human posture also cause non-ideal covariance matrices of data PDFs so that the irregular data distributions increase the estimation errors of the EM algorithm.

Eventually, the most probable cluster numbers were employed as input information for constructing the morphologies of the LDF in the proposed DDFM algorithm, and then the corresponding LDF morphologies are respectively shown from (e)–(g) in Figure 4 and Figure 5. In order to speed up the estimation process, the skill of downsampling was used in the procedure of LDF estimations and the adopted ratio was 20%. The corresponding cluster boundaries were successfully delineated by visualizing the corresponding LDF morphologies in each studied case. Each of them can be directly defined, as implied in Step (9) of the algorithmic flow, by finding the lowest energy levels that only enclose one cluster center, as indicated by the black arrows. It should be emphasized that since the LDF was a linear combination of the PDFs with different weighting factors, the LDF morphology is equivalent to a probabilistic map. The data points enclosed by their corresponding boundaries were classified as the same clusters and were assigned into the specific states of motional patterns. The other data points that were not classified into these clusters were assigned into the transition state. It should be emphasized again that these specific states of motional patterns were defined by the prior information associated with the G-sensor architecture. In other words, the different directions of state migrations in these cases, respectively show in Figure 4c and Figure 5c, exhibit important inter-state features for systematic motional pattern recognition. By considering the G-sensor architecture and the directions of state migrations, the directions of PA 2 are always along the tangential direction of the state migration, while that of PA 3 is along the normal direction. The phenomenon was evidenced by the mapped plots shown in Figure 4c and Figure 5c, wherein the illustrations of human turning were also inserted. Since this information only can be recognized by simultaneously checking the directions of state migrations and the G-sensor architecture, it was assigned as the inter-state feature for resolving the issue of motional pattern recognition. All classified results are illustrated in (a)–(c) of Figure 4 and Figure 5, and the results were then fed back into the time signals as exhibited in Figure 3. Therefore, the most probable states of the motional pattern in the time signals are successful classified using the proposed DDFM algorithm.

To verify the proposed DDFM algorithm, a simulation of turning-over monitoring was employed to validate the feasibility. Figure 6 shows an experimental mapping plot, which was a simulation of turning-over situations. These results shown in Figure 6 were randomly picked from a relevant dataset of the subjects. Under the experimental scheme, a subject lay on a bed and faced up. Then, the subject turned left twice (now face-down) with a periodic time of 10 s. By combining the extracted features from the Lying-to-TL and -TR cases, the experimental results can be automatically divided into four states, as expected. Each cluster centroid was marked by a black dot in Figure 6. The migrations of the mapped data points and the illustration of human turning again evidences the feasibility of the inter-state features. Especially, the states of face-up and face-down can be well recognized using the proposed DDFM algorithm due to their inter-state features. The inclination with a value of 10.6 degrees might be caused due to the unbalanced postures during the subject motions. In a nutshell, the proposed algorithm and the experimental scheme offer a simple avenue to resolving the demanding issue of elderly and infant turning-over monitoring.

4. Discussion

The proposed DDFM algorithm provides an avenue for resolving the problem of motional pattern recognition. The successful classified results of motional patterns of Lying-to-TL and Lying-to-TR cases in specific physical spaces have been respectively shown in Figure 4 and Figure 5, and that in time domains has been shown in Figure 3. An algorithmic flow is provided as well. By mapping the time signals into a 2D physical space, the most probable cluster number can be estimated using the trend curves of the global Hamiltonian functional and the most probable cluster boundaries can also be determined by finding the lowest energy levels in the LDF morphologies. Thus, the DDFM is a self-consistent method for pattern recognition.

In the proposed experimental scheme, the rotating directions of the sensor architecture were used as prior information for the algorithmic estimations, so that the most probable states of motional patterns can be definitely determined by combining the features extracted from PA 2-PA 3 scattered plots, i.e., the proposed inter-state features. By simultaneously considering the direction of state migration in the PA 2-PA 3 plot of Figure 4c and the sensor architecture, the progressively increasing curve exhibits an increase in both of the electric voltages in PA 2 and PA 3. Then, this motional pattern was classified to the Lying-to-TL case. A similar deduction can be applied to the Lying-to-TR case in Figure 5c. Therefore, the motional pattern recognitions can be correctly and easily recognized using the proposed DDFM algorithm. The classified results of the patterns can be remapped into the time-dependent space, and the classified time signals of the cases are respectively shown in Figure 3. It is worth discussing the limitation of the proposed algorithm. As aforementioned, the non-ideal effects, as exhibited in Figure 4d and Figure 5d, will occur in the estimation procedure of the global Hamiltonian functionals. In these cases, the employment of the estimator and the chaotic level of the data distribution might significantly limit the accuracy of cluster number estimation and that of the cluster boundary definition as well.

5. Conclusions

In summary, the proposed DDFM algorithm and sensor architecture offer a nonparametric experimental scheme for the time-dependent data classification and motional pattern recognition. The most probable cluster number and the corresponding boundaries can be well defined, so that the most probable states of the motional patterns can be easily extracted. The successful experimental results and the simulation of turning-over monitoring reveal the feasibility of the proposed method. In order to make a contribution to clinical investigation, the proposed method would further fuse the key techniques from other objective methods for resolving the issues of pattern recognition and automatic segmentation in the future. In addition to solving the proposed segmentation and recognition in time-dependent problems, the proposed scheme can be further implemented to the major topics in clinical research, such as the patterns of cough in subjects in the intensive care unit (ICU), gait analysis, the patterns of vital physiological signals, work-related postures in industrial settings, and so forth.