Submit to Sensors Review for Sensors Propose a Special Issue

Journal Menu

Journal Browser

► Journal Browser

Sensors for Human Activity Recognition II

Special Issue Editors
Special Issue Information
Benefits of Publishing in a Special Issue
Related Special Issue
Published Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Wearables".

Deadline for manuscript submissions: closed (29 February 2024) | Viewed by 22826

Share This Special Issue

Special Issue Editors

Dr. Hui Liu

E-Mail Website
Guest Editor

Cognitive Systems Laboratory, Faculty of Mathematic/Informatics, University of Bremen, 28359 Bremen, Germany
Interests: biosignal processing; feature selection and feature space reduction; human activity recognition; real-time recognition systems; knee bandage; machine learning
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Hugo Gamboa

E-Mail Website
Guest Editor

Laboratory for Instrumentation, Biomedical Engineering and Radiation Physics (LIBPhys-UNL), Faculty of Sciences and Technology, NOVA University of Lisbon, 2820-001 Caparica, Portugal
Interests: instrumentation; signal processing; machine learning; human activity recognition (HAR)
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Tanja Schultz

E-Mail Website
Guest Editor

Computer Science, Faculty of Mathematic/Informatics, University of Bremen, 28359 Bremen, Germany
Interests: biosignal processing; human-centered man–machine interfaces; user states and traits modeling; machine learning; interfaces based on muscle and brain activities; automatic speech recognition; silent speech interfaces; brain interfaces
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Human activity recognition (HAR) has been playing an increasingly important role in the digital age.

High-quality sensory observations applicable to recognizing users' activities, whether through external or internal (wearable) sensing technology, are inseparable from sensors' sophisticated design and appropriate application.

Having been studied and verified adequately, traditional sensors suitable for human activity recognition—such as external sensors for smart homes, optical sensors such as cameras for capturing video signals, and bioelectrical and biomechanical sensors for wearable applications—continue to be researched in depth for more effective and efficient usage, among which specific areas of life facilitated by sensor-based HAR have been continuously increasing.

Meanwhile, innovative sensor research for HAR is also very active in the academic community, including brand new types of sensors appropriate for HAR, new designs and applications of the abovementioned traditional sensors, and the introduction of non-traditional HAR-related sensor types into HAR tasks, among others.

This Special Issue aims to provide researchers in related fields a platform to demonstrate their unique insights and late-breaking achievements, encouraging authors to submit their state-of-the-art research and contributions on sensors for HAR.

The main topics for this Issue include but are not limited to:

Sensor design and development;
Embedded signal processing;
Biosignal instrumentation;
Mobile sensing and mobile-phone-based signal processing;
Wearable sensors and body sensor networks;
Printable sensors;
Implants;
Behavior recognition;
Applications to healthcare, sports, edutainment, and others;
Sensor-based machine learning;
HAR in XR.

Dr. Hui Liu
Prof. Dr. Hugo Gamboa
Prof. Dr. Tanja Schultz
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Benefits of Publishing in a Special Issue

Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Related Special Issue

Sensors for Human Activity Recognition: 3rd Edition in Sensors (1 article)

Published Papers (11 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Editorial

Jump to: Research, Review

5 pages, 173 KiB

Open AccessEditorial

Human Activity Recognition, Monitoring, and Analysis Facilitated by Novel and Widespread Applications of Sensors

by Hui Liu, Hugo Gamboa and Tanja Schultz

Sensors 2024, 24(16), 5250; https://doi.org/10.3390/s24165250 - 14 Aug 2024

Cited by 1 | Viewed by 843

Abstract

The Special Issue Sensors for Human Activity Recognition has received a total of 30 submissions so far, and from these, this new edition will publish 10 academic articles [...] Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

Research

Jump to: Editorial, Review

26 pages, 7249 KiB

Open AccessArticle

Biosensor-Driven IoT Wearables for Accurate Body Motion Tracking and Localization

by Nouf Abdullah Almujally, Danyal Khan, Naif Al Mudawi, Mohammed Alonazi, Abdulwahab Alazeb, Asaad Algarni, Ahmad Jalal and Hui Liu

Sensors 2024, 24(10), 3032; https://doi.org/10.3390/s24103032 - 10 May 2024

Cited by 16 | Viewed by 1499

Abstract

The domain of human locomotion identification through smartphone sensors is witnessing rapid expansion within the realm of research. This domain boasts significant potential across various sectors, including healthcare, sports, security systems, home automation, and real-time location tracking. Despite the considerable volume of existing research, the greater portion of it has primarily concentrated on locomotion activities. Comparatively less emphasis has been placed on the recognition of human localization patterns. In the current study, we introduce a system by facilitating the recognition of both human physical and location-based patterns. This system utilizes the capabilities of smartphone sensors to achieve its objectives. Our goal is to develop a system that can accurately identify different human physical and localization activities, such as walking, running, jumping, indoor, and outdoor activities. To achieve this, we perform preprocessing on the raw sensor data using a Butterworth filter for inertial sensors and a Median Filter for Global Positioning System (GPS) and then applying Hamming windowing techniques to segment the filtered data. We then extract features from the raw inertial and GPS sensors and select relevant features using the variance threshold feature selection method. The extrasensory dataset exhibits an imbalanced number of samples for certain activities. To address this issue, the permutation-based data augmentation technique is employed. The augmented features are optimized using the Yeo–Johnson power transformation algorithm before being sent to a multi-layer perceptron for classification. We evaluate our system using the K-fold cross-validation technique. The datasets used in this study are the Extrasensory and Sussex Huawei Locomotion (SHL), which contain both physical and localization activities. Our experiments demonstrate that our system achieves high accuracy with 96% and 94% over Extrasensory and SHL in physical activities and 94% and 91% over Extrasensory and SHL in the location-based activities, outperforming previous state-of-the-art methods in recognizing both types of activities. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

12 pages, 61978 KiB

Open AccessArticle

Adopting Graph Neural Networks to Analyze Human–Object Interactions for Inferring Activities of Daily Living

by Peng Su and Dejiu Chen

Sensors 2024, 24(8), 2567; https://doi.org/10.3390/s24082567 - 17 Apr 2024

Cited by 2 | Viewed by 1054

Abstract

Human Activity Recognition (HAR) refers to a field that aims to identify human activities by adopting multiple techniques. In this field, different applications, such as smart homes and assistive robots, are introduced to support individuals in their Activities of Daily Living (ADL) by analyzing data collected from various sensors. Apart from wearable sensors, the adoption of camera frames to analyze and classify ADL has emerged as a promising trend for achieving the identification and classification of ADL. To accomplish this, the existing approaches typically rely on object classification with pose estimation using the image frames collected from cameras. Given the existence of inherent correlations between human–object interactions and ADL, further efforts are often needed to leverage these correlations for more effective and well justified decisions. To this end, this work proposes a framework where Graph Neural Networks (GNN) are adopted to explicitly analyze human–object interactions for more effectively recognizing daily activities. By automatically encoding the correlations among various interactions detected through some collected relational data, the framework infers the existence of different activities alongside their corresponding environmental objects. As a case study, we use the Toyota Smart Home dataset to evaluate the proposed framework. Compared with conventional feed-forward neural networks, the results demonstrate significantly superior performance in identifying ADL, allowing for the classification of different daily activities with an accuracy of 0.88. Furthermore, the incorporation of encoded information from relational data enhances object-inference performance compared to the GNN without joint prediction, increasing accuracy from 0.71 to 0.77. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

24 pages, 5574 KiB

Open AccessArticle

MeshID: Few-Shot Finger Gesture Based User Identification Using Orthogonal Signal Interference

by Weiling Zheng, Yu Zhang, Landu Jiang, Dian Zhang and Tao Gu

Sensors 2024, 24(6), 1978; https://doi.org/10.3390/s24061978 - 20 Mar 2024

Viewed by 943

Abstract

Radio frequency (RF) technology has been applied to enable advanced behavioral sensing in human-computer interaction. Due to its device-free sensing capability and wide availability on Internet of Things devices. Enabling finger gesture-based identification with high accuracy can be challenging due to low RF signal resolution and user heterogeneity. In this paper, we propose MeshID, a novel RF-based user identification scheme that enables identification through finger gestures with high accuracy. MeshID significantly improves the sensing sensitivity on RF signal interference, and hence is able to extract subtle individual biometrics through velocity distribution profiling (VDP) features from less-distinct finger motions such as drawing digits in the air. We design an efficient few-shot model retraining framework based on first component reverse module, achieving high model robustness and performance in a complex environment. We conduct comprehensive real-world experiments and the results show that MeshID achieves a user identification accuracy of

95.17 %

on average in three indoor environments. The results indicate that MeshID outperforms the state-of-the-art in identification performance with less cost. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

22 pages, 592 KiB

Open AccessArticle

Empowering Participatory Research in Urban Health: Wearable Biometric and Environmental Sensors for Activity Recognition

by Rok Novak, Johanna Amalia Robinson, Tjaša Kanduč, Dimosthenis Sarigiannis, Sašo Džeroski and David Kocman

Sensors 2023, 23(24), 9890; https://doi.org/10.3390/s23249890 - 18 Dec 2023

Cited by 1 | Viewed by 1633

Abstract

Participatory exposure research, which tracks behaviour and assesses exposure to stressors like air pollution, traditionally relies on time-activity diaries. This study introduces a novel approach, employing machine learning (ML) to empower laypersons in human activity recognition (HAR), aiming to reduce dependence on manual recording by leveraging data from wearable sensors. Recognising complex activities such as smoking and cooking presents unique challenges due to specific environmental conditions. In this research, we combined wearable environment/ambient and wrist-worn activity/biometric sensors for complex activity recognition in an urban stressor exposure study, measuring parameters like particulate matter concentrations, temperature, and humidity. Two groups, Group H (88 individuals) and Group M (18 individuals), wore the devices and manually logged their activities hourly and minutely, respectively. Prioritising accessibility and inclusivity, we selected three classification algorithms: k-nearest neighbours (IBk), decision trees (J48), and random forests (RF), based on: (1) proven efficacy in existing literature, (2) understandability and transparency for laypersons, (3) availability on user-friendly platforms like WEKA, and (4) efficiency on basic devices such as office laptops or smartphones. Accuracy improved with finer temporal resolution and detailed activity categories. However, when compared to other published human activity recognition research, our accuracy rates, particularly for less complex activities, were not as competitive. Misclassifications were higher for vague activities (resting, playing), while well-defined activities (smoking, cooking, running) had few errors. Including environmental sensor data increased accuracy for all activities, especially playing, smoking, and running. Future work should consider exploring other explainable algorithms available on diverse tools and platforms. Our findings underscore ML’s potential in exposure studies, emphasising its adaptability and significance for laypersons while also highlighting areas for improvement. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

26 pages, 5411 KiB

Open AccessArticle

Decoding Mental Effort in a Quasi-Realistic Scenario: A Feasibility Study on Multimodal Data Fusion and Classification

by Sabrina Gado, Katharina Lingelbach, Maria Wirzberger and Mathias Vukelić

Sensors 2023, 23(14), 6546; https://doi.org/10.3390/s23146546 - 20 Jul 2023

Cited by 2 | Viewed by 2068

Abstract

Humans’ performance varies due to the mental resources that are available to successfully pursue a task. To monitor users’ current cognitive resources in naturalistic scenarios, it is essential to not only measure demands induced by the task itself but also consider situational and environmental influences. We conducted a multimodal study with 18 participants (nine female, M = 25.9 with SD = 3.8 years). In this study, we recorded respiratory, ocular, cardiac, and brain activity using functional near-infrared spectroscopy (fNIRS) while participants performed an adapted version of the warship commander task with concurrent emotional speech distraction. We tested the feasibility of decoding the experienced mental effort with a multimodal machine learning architecture. The architecture comprised feature engineering, model optimisation, and model selection to combine multimodal measurements in a cross-subject classification. Our approach reduces possible overfitting and reliably distinguishes two different levels of mental effort. These findings contribute to the prediction of different states of mental effort and pave the way toward generalised state monitoring across individuals in realistic applications. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Graphical abstract

12 pages, 469 KiB

Open AccessArticle

Exploring Regularization Methods for Domain Generalization in Accelerometer-Based Human Activity Recognition

by Nuno Bento, Joana Rebelo, André V. Carreiro, François Ravache and Marília Barandas

Sensors 2023, 23(14), 6511; https://doi.org/10.3390/s23146511 - 19 Jul 2023

Cited by 1 | Viewed by 1823

Abstract

The study of Domain Generalization (DG) has gained considerable momentum in the Machine Learning (ML) field. Human Activity Recognition (HAR) inherently encompasses diverse domains (e.g., users, devices, or datasets), rendering it an ideal testbed for exploring Domain Generalization. Building upon recent work, this paper investigates the application of regularization methods to bridge the generalization gap between traditional models based on handcrafted features and deep neural networks. We apply various regularizers, including sparse training, Mixup, Distributionally Robust Optimization (DRO), and Sharpness-Aware Minimization (SAM), to deep learning models and assess their performance in Out-of-Distribution (OOD) settings across multiple domains using homogenized public datasets. Our results show that Mixup and SAM are the best-performing regularizers. However, they are unable to match the performance of models based on handcrafted features. This suggests that while regularization techniques can improve OOD robustness to some extent, handcrafted features remain superior for domain generalization in HAR tasks. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

19 pages, 6496 KiB

Open AccessArticle

Classification and Analysis of Human Body Movement Characteristics Associated with Acrophobia Induced by Virtual Reality Scenes of Heights

by Xiankai Cheng, Benkun Bao, Weidong Cui, Shuai Liu, Jun Zhong, Liming Cai and Hongbo Yang

Sensors 2023, 23(12), 5482; https://doi.org/10.3390/s23125482 - 10 Jun 2023

Cited by 1 | Viewed by 1880

Abstract

Acrophobia (fear of heights), a prevalent psychological disorder, elicits profound fear and evokes a range of adverse physiological responses in individuals when exposed to heights, which will lead to a very dangerous state for people in actual heights. In this paper, we explore the behavioral influences in terms of movements in people confronted with virtual reality scenes of extreme heights and develop an acrophobia classification model based on human movement characteristics. To this end, we used wireless miniaturized inertial navigation sensors (WMINS) network to obtain the information of limb movements in the virtual environment. Based on these data, we constructed a series of data feature processing processes, proposed a system model for the classification of acrophobia and non-acrophobia based on human motion feature analysis, and realized the classification recognition of acrophobia and non-acrophobia through the designed integrated learning model. The final accuracy of acrophobia dichotomous classification based on limb motion information reached 94.64%, which has higher accuracy and efficiency compared with other existing research models. Overall, our study demonstrates a strong correlation between people’s mental state during fear of heights and their limb movements at that time. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

16 pages, 8330 KiB

Open AccessArticle

Counting Activities Using Weakly Labeled Raw Acceleration Data: A Variable-Length Sequence Approach with Deep Learning to Maintain Event Duration Flexibility

by Georgios Sopidis, Michael Haslgrübler and Alois Ferscha

Sensors 2023, 23(11), 5057; https://doi.org/10.3390/s23115057 - 25 May 2023

Cited by 3 | Viewed by 2179

Abstract

This paper presents a novel approach for counting hand-performed activities using deep learning and inertial measurement units (IMUs). The particular challenge in this task is finding the correct window size for capturing activities with different durations. Traditionally, fixed window sizes have been used, which occasionally result in incorrectly represented activities. To address this limitation, we propose segmenting the time series data into variable-length sequences using ragged tensors to store and process the data. Additionally, our approach utilizes weakly labeled data to simplify the annotation process and reduce the time to prepare annotated data for machine learning algorithms. Thus, the model receives only partial information about the performed activity. Therefore, we propose an LSTM-based architecture, which takes into account both the ragged tensors and the weak labels. To the best of our knowledge, no prior studies attempted counting utilizing variable-size IMU acceleration data with relatively low computational requirements using the number of completed repetitions of hand-performed activities as a label. Hence, we present the data segmentation method we employed and the model architecture that we implemented to show the effectiveness of our approach. Our results are evaluated using the Skoda public dataset for Human activity recognition (HAR) and demonstrate a repetition error of

\pm 1

even in the most challenging cases. The findings of this study have applications and can be beneficial for various fields, including healthcare, sports and fitness, human–computer interaction, robotics, and the manufacturing industry. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

21 pages, 16467 KiB

Open AccessArticle

SENS+: A Co-Existing Fabrication System for a Smart DFA Environment Based on Energy Fusion Information

by Teng-Wen Chang, Hsin-Yi Huang, Cheng-Chun Hong, Sambit Datta and Walaiporn Nakapan

Sensors 2023, 23(6), 2890; https://doi.org/10.3390/s23062890 - 7 Mar 2023

Cited by 1 | Viewed by 3409

Abstract

In factories, energy conservation is a crucial issue. The co-fabrication space is a modern-day equivalent of a new factory type, and it makes use of Internet of Things (IoT) devices, such as sensors, software, and online connectivity, to keep track of various building features, analyze data, and produce reports on usage patterns and trends that can be used to improve building operations and the environment. The co-fabrication user requires dynamic and flexible space, which is different from the conventional user’s usage. Because the user composition in a co-fabrication space is dynamic and unstable, we cannot use the conventional approach to assess their usage and rentals. Prototyping necessitates a specifically designed energy-saving strategy. The research uses a “seeing–moving–seeing” design thinking framework, which enables designers to more easily convey their ideas to others through direct observation of the outcomes of their intuitive designs and the representation of their works through design media. The three components of human behavior, physical manufacture, and digital interaction are primarily the focus of this work. The computing system that connects the physical machine is created through communication between the designer and the digital interface, giving the designer control over the physical machine. It is an interactive fabrication process formed by behavior. The Sensible Energy System+ is an interactive fabrication process of virtual and real coexistence created by combining the already-existing technology, the prototype fabrication machine, and SENS. This process analyzes each step of the fabrication process and energy, fits it into the computing system mode to control the prototype fabrication machine, and reduces the problem between virtual and physical fabrication and energy consumption. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Figure 1

Review

Jump to: Editorial, Research

15 pages, 718 KiB

Open AccessReview

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

by Yifan Bian, Dennis Küster, Hui Liu and Eva G. Krumhuber

Sensors 2024, 24(1), 126; https://doi.org/10.3390/s24010126 - 26 Dec 2023

Cited by 4 | Viewed by 3824

Abstract

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models. Full article

(This article belongs to the Special Issue Sensors for Human Activity Recognition II)

► Show Figures

Journal Menu

Journal Browser

Sensors for Human Activity Recognition II

Share This Special Issue

Special Issue Editors

Special Issue Information

Benefits of Publishing in a Special Issue

Related Special Issue

Published Papers (11 papers)

Editorial

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI