Smarty Pants: Exploring Textile Pressure Sensors in Trousers for Posture and Behaviour Classification ^†

Abstract

In this paper, we introduce a new modality for capturing body postures and social behaviour. Vice versa, we propose a new application area for on-body textile sensors. We have developed “smart trousers” with embedded textile pressure sensors that allow for classification of a large variety of postural movements as well as interactional states. Random Forest models are used to investigate those. Here, we give an overview of the research conducted and discuss potential use cases of the presented design.

1. Introduction

Using technologies in a ubiquitous way which has been integrated into our day-to-day life has long been established. Smart environments [1] or interior designs [2,3] have contributed to imagining a fully connected, Internet of Things inspired life. A central part of such sensing networks is to take measurements of the human body itself. Often, this is achieved by collecting data through objects we come in touch with or encounter, transforming them into sensing surfaces—for example, seat covers [2] or even ceiling lights [4]. A more dynamic and flexible approach to human-centred sensing is wearable computing, using mobile phones, smart watches, or other accessories to track bodily actions continuously.

The range of applications in which these technologies can be used is wide. They are often presented in a context for novel interface designs in Human–Computer Interaction, but are also used in health and medical care for patient monitoring or rehabilitation purposes, as well as in social sciences, to explore interactional dynamics of individuals and groups [5,6]. There are, however, problems with such sensing systems. Many of them are obtrusive and augment our natural environment. Furthermore, many devices record more data than necessary, leading to problems of privacy and obtaining consent of subjects. One modality that has proven to overcome many of these concerns is textile sensing. Utilising this material with which we are so familiar and by which we are surrounded in many situations has gained increasing interest in the past decade. Embedded in pillows [7], tablecloths [8], or garments [9,10], electronic textiles (e-textiles) are able to replace rigid components and capture bodily movement in a novel, holistic way on which other technologies have had to compromise.

In our work, we aim to establish smart clothing as a novel method to capture social interaction. We focus on measuring non-verbal behaviours like postural movement and touch interactions through textile pressure sensors in custom-made trousers. Here, we present an overview of this research and report on the design engineering, verification, and findings of our “smart trousers”.

2. Designing Sensing Trousers

The way we move in social interaction (and in general) is, in part, determined by the affordances of our clothes. Certain materials and pattern cuts allow or restrict bodily actions. This has to be accounted for when designing textile sensors, too. Shape, number, position, and type of sensors should follow the purpose of the determined use case. Additionally, the integration of these sensors in a garment also requires specific manufacturing considerations and design principles.

The starting points of addressing these aspects were ethnographic studies of seated, spontaneous conversations between people. The findings of these observations informed hypotheses on postures linked to social behaviours—for example, identifying typical movements of speakers compared to listeners.

The shell of the trousers is made of a conventional fabric, a viscose–cotton mix jersey knit for a legging-like design and good wearing comfort. The embedded sensing system uses conductive fabrics as well as knitted single jerseys with different properties. We used a resistive fabric from Eeonyx (EeonTex LTT-SLPA Conductive Stretchable Fabric) for the middle layer of the pressure-sensing area, and highly conductive nylon stripes to build the connections (see top right of Figure 1). The fabrics were arranged to form a matrix of 10 × 10 sensors, one for each leg of the trousers. This creates 100 data points per leg, mapped around the thighs and buttocks, as shown on the left side of Figure 1. Each crossing point of the rows and columns of the matrix could be treated as a separate pressure sensor.

The sensors are connected to a battery-powered microcontroller that collected their data locally with a micro SD-card. The circuit board is hidden in the hem of the trousers to be the least intrusive to the wearer. Pressure recordings were collected at 4 Hz, following the principle of [11]. For further pre-processing and final analyses, the sensor data was normalised.

For the integration of this sensing system, trousers were tailored to conceal all electronic components and wiring in an unobtrusive way. Panels on the inside leg were added for wiring and connections to the microcontroller, different clothing sizes were accounted for, and fabric layering was accumulated for optimised wearing comfort. Details are shown in Figure 1 (left). A more detailed description of the design process can be found in [12].

Figure 1. Left: Inside and outside view of “smart trousers” with the layer of textile pressure sensors embedded around thighs and buttocks on upper legs. Right: Inside of the matrix, showing the distribution of rows and columns as well as the layering of the pressure sensors.

Figure 1. Left: Inside and outside view of “smart trousers” with the layer of textile pressure sensors embedded around thighs and buttocks on upper legs. Right: Inside of the matrix, showing the distribution of rows and columns as well as the layering of the pressure sensors.

3. Methodology

The performance of our “smart trousers” as methods to capture postures of conversation was tested in two user studies. First, we evaluated how well the trousers can distinguish a large variety of static sitting postures. Single users were instructed to perform a range of 19 postures combining upper and lower legs, as well as hand touch on legs and positions of the upper body in relation. A visualisation of two leg crossing postures is shown as an example in Figure 2.

Secondly, we looked at seated multiparty conversations, focusing on behavioural cues that do not necessarily correlate with marked postural movement, but that determine conversational states. In particular, we evaluated the following against each other: Talking, listening, laughing, nodding, and backchanneling. This was examined in three-way conversations recorded and processed in similar ways to those of the first user study. An overview and comparison of both studies is shown in Table 1.

Figure 2. Simple visualisation of pressure distribution for two leg crossing sitting postures. Each circle represents a data point of the textile sensor matrix in the trousers.

Figure 2. Simple visualisation of pressure distribution for two leg crossing sitting postures. Each circle represents a data point of the textile sensor matrix in the trousers.

An overview and comparison of these two studies is shown in Figure 1.

Table 1. Overview of the settings used in the two studies in which the smart trousers were tested.

	Mode	Participants	Set Up	Software	Classification	Classes
Study 1: Postures	single user	6	controlled	Weka, Elan	Random Forest	19
Study 2: Behaviours	multi-party	20	spontaneous	Weka, Elan	Random Forest	4 + 1

Table 1. Overview of the settings used in the two studies in which the smart trousers were tested.

	Mode	Participants	Set Up	Software	Classification	Classes
Study 1: Postures	single user	6	controlled	Weka, Elan	Random Forest	19
Study 2: Behaviours	multi-party	20	spontaneous	Weka, Elan	Random Forest	4 + 1

4. Results

All studies were video recorded and annotated using Elan [13] The sensor data was time aligned with annotations of the video recordings, marking the gross events we examine here, i.e., postures for the first, and conversational cues for the second study.

In both studies, we used machine learning techniques to detect the determined postures and behaviours. Random Forest classification with a 10-fold cross-validation tested individual and community models. Average accuracies and F-measures were overall notably better for individual participants than for groups. Moreover, posture classification showed significantly better results than behaviour classification, with up to 99.75% accuracy for individual models and 64.26% for community models. In summary, this aligns with our preliminary expectations of this new sensing system. Despite leaving room for improvement both in analysis methods as well as in data processing, we were able to show that simple textile sensors in trousers are a valid method to capture a large range of human behaviour.

5. Discussion

In summary, the results of both studies show that there is a large variation across participants, leading to poorer results in community models. Nevertheless, there are clearly reoccurring patterns of movement and pressure distribution within the same classes of posture or behaviour. The conclusion to draw from this is that we move similarly to ourselves, but there can be a significant difference in how different people perform different movements. This finding can be used for future applications that focus around individuals—for example, for patients in rehabilitation or for athletes’ customised training programmes. With a larger data set, the performance of community models could be improved as well, leaving room for further use cases assessing social dynamics of groups through smart clothing, such as detecting behavioural actions of new interactants.

In this research, we have explored the potential of one modality—pressure sensing. However, there are wearable sensor designs offering a more hybrid approach, being able to additionally capture proximity or other touch interactions. Further iterations on the design of the sensor matrix used here regarding materials, embedding in trousers, or optimised pattern cutting are also proposed in [14] and can contribute towards a more fine-grained and even more connected sensing network to gather bodily data. In addition, if trousers can tell whether someone is speaking or not without recording video or audio, other pieces of clothing may be able to capture more fine-grained and complex behaviours, and can be validated as a method to understand non-verbal behaviours in an even more detailed manner.