Embracing First-Person Perspectives in Soma-Based Design

Abstract

A set of prominent designers embarked on a research journey to explore aesthetics in movement-based design. Here we unpack one of the design sensitivities unique to our practice: a strong first person perspective—where the movements, somatics and aesthetic sensibilities of the designer, design researcher and user are at the forefront. We present an annotated portfolio of design exemplars and a brief introduction to some of the design methods and theory we use, together substantiating and explaining the first-person perspective. At the same time, we show how this felt dimension, despite its subjective nature, is what provides rigor and structure to our design research. Our aim is to assist researchers in soma-based design and designers wanting to consider the multiple facets when designing for the aesthetics of movement. The applications span a large field of designs, including slow introspective, contemplative interactions, arts, dance, health applications, games, work applications and many others.

1. Introduction

We are currently experiencing a rise in movement-based design, both commercially and in academic explorations. This trend follows the development of new interaction technologies—sensing, actuating, wireless connectivity—together forming a rich space of design possibilities. As we will argue here, these movement-based designs can draw upon the history of movement practices that predate the academic field of Human-Computer Interaction (HCI) [1].

Some predict that the domain of body-, movement- and biosensor-based [2] interactions could be as big as or even bigger than desktop and mobile. They will be reaching beyond application areas in HCI where physical coordination and learning-by-doing are naturally important (such as movement rehabilitation, dance movement therapy, movement education including dance and sports training, self-cultivation, yoga, emotion regulation for management of stress and anxiety related health problems, and similar) into areas that do not yet see aesthetics of movement as a fundamental cornerstone in their design processes (such as communication within multi-stakeholder design processes, interaction with everyday Internet of Things applications, quantified self and health apps, elderly homecare).

As interactive technologies proliferate in everyday life, they start to shape our cultural expressions and, furthermore, they start to shape us. Technologies transform our experience through amplifying or reducing reality, and by translating our behavior through inviting and inhibiting actions [3]. We see it already in technologies such as the smartphone, which has altered everything from how we meet, socialize, or play, to even changing our bodily movements (e.g., [4]). We inscribe our understanding of our bodily selves into them and they in turn shape us, what we do and how we do them, both in terms of practices, but also in a literal, corporeal sense. These technologies will (depending on how we design them) encourage certain movements, certain aesthetic experiences, certain practices and understandings of our bodies—while not encouraging others. They will influence our availability for certain qualities of interaction and not others. They will alter how we move, our postures, our muscles and nervous system reactions and orientations. This will continue to affect how we do our work tasks as well as our leisure activities, social communication, play, and artistic expressions, and therefore ultimately, how we live. This is why we need to cultivate our understanding of what it means to be a sensing, feeling and moving body, living in the world, shaping and being shaped by our lifeworld.

It is worth noting how many commercial design attempts fall short in delivering on the promise of movement-based interaction. Movement interfaces, such as Wiimote or Kinect, created a lot of excitement, but it has been notoriously difficult to design games and other applications for these platforms [5,6,7]. Likewise, we have seen an increased interest in virtual and augmented reality [8,9], but again, only few applications last past the initial excitement. We might argue that this is fine—many games and mobile apps are short-lived—but there seems to be a missed opportunity in sustaining longer-term experiences, and even more importantly, there is an opportunity to expand beyond what these first technologies have provided so far.

The recommendations in this paper flow from a workshop held in conjunction with the ACM SIGCHI-conference in 2016, involving some of the most prominent researchers in the field, coming together to work out the most promising and rigorous strategies for engaging in soma-based design. By soma-based design we mean a large space of movement-based designs and design practices specifically engaging with aesthetics, which will be further explicated over the course of this paper. We noted that we had many joint experiences and insights, as well as concerns. We compared our approaches to design and found many common traits: certain methods, certain practices, certain sensitizing exercises, certain orienting conceptualizations—together forming a systematic way of approaching design, providing us with rigor in our research. In this paper, we aim to make two contributions: we will lay out the research area, and second we will share what we mean by rigor in our work through explaining and unpacking one of the most prominent traits of our practice: a first-person perspective on movement-based design and why it is crucial in this area.

As this work has its home in Human-Computer Interaction (HCI) we use the term ‘user’ to denote the person who will, in the end, use the system created. This term has been much debated in the community, as it communicates a passive stance, that is someone who is given a design and not an active co-constructor of the interaction. In reality, whenever a system is realized in the world—being used—it continues to be ‘designed’ by the interpretations and choices of the end-user, filling it with meaning and interactions, often filling it with content [10,11]. In addition, most systems these days are in a perpetual beta-state, where they are being redesigned while being used in response to end-user behaviors. The line between being a designer and being a user is blurred. This perpetual beta-state also means that a system is never entirely finished—it is a work in progress, continuously shaping and being shaped by user activities. In the context of the work presented here, the term actor, participant or co-constructor would have been more appropriate. The end-user will actively co-construct their experience, the meaning of the interactions, together shaping practices and engagements over time. But as actor is not a well-established term in HCI, we have decided to continue to use user.

Given this richer understanding of what we mean by user, in short, a first-person perspective places the user’s lived experience at the core of the design process—the lived experience of moving and being moved become the main unifying activity during the design process. In particular, by using the concept soma (drawing upon Hannah [12] and Shusterman [13]), we emphasize the living, purposive, sentient, perceptive body or bodily subjectivity of the first-person perspective. We do not shun from engaging with our own subjective feelings and passions. If one wants to design for the lived experience of moving and being moved, designers have to be engaged in their own somatics by doing and experiencing while designing [14]. As designers, we must be able to distinguish between all the fine nuances of different movements, emotional experiences, tactile engagements, or mirrorings of our bodily processes in interactive design. We also need to feel, touch, interact with, and experience the affordances of the “materials” (digital or other materials) we use to compose the dynamic gestalt of the interaction [15].

We unpack and substantiate the first-person perspective through three accounts. First, we provide an annotated portfolio [16] through first-person accounts of design exemplars. The design exemplars—or ultimate particulars in the words of Stolterman [11]—serve as definitions made through design [17,18] in a Research through Design (RtD) practice [6,18,19,20,21,22,23,24,25,26,27]. The design exemplars serve two purposes here: first, they show how our first-person perspective is able to generate viable, functioning, qualitatively different design work feeding off our movements and bodily engagements; second, through providing first-person accounts of use we provide a glimpse of the kind of somatic engagement they enable.

In our second account of the first-person perspective, we share some of the soma design methods employed by the authors. At the core of each method lies a double first-person engagement: on the one hand, empathically shaping experiences for future users through our own, first person, soma-based engagements, and at the same time changing our soma-based aesthetic appreciation skills as designers. This double engagement requires authenticity and personal engagement.

In our third account, we provide some of the theoretical underpinnings and contributions from our work, building on concepts from pragmatism and somaesthetic theories.

Taken together, these three accounts substantiate what we mean by rigor in soma-based design. With a first-person perspective, there is no short-cut, no simplified wireframes, no language-based distancing between designers and what is being designed. The only path to design is through thoroughly living your own designs, experiencing them on a profound level, empathically engaging with the somatic experiences of your future users.

2. Background

Let us start by providing a brief background on the first-person perspective; providing theoretical underpinnings, as well a review of how it has already been used in HCI and interaction design research and practice. Second, it is helpful to ground the discussion that took place with terminology that the workshop participants drew upon. Here, we provide background on concepts that surround the soma-based design space: aesthetics, somatics, and somaesthetics.

2.1. First-Person Perspective: Theoretical Underpinnings

A number of research projects on soma-based interaction have used Merleau-Ponty’s phenomenology of the body [28] as a theoretical backdrop (e.g., [29,30]). Merleau-Ponty makes a distinction between the first-person perspective on the body, the lived body, the body through which we live our lives and experience the world, and the third-person perspective of seeing the body as an object in the world. In German there are two terms for the body, Leib and Körper, corresponding to the first- and the third-person perspective, respectively, while in English we use body for both. In addition to the first person and the third person perspectives on the body, there is also the interpersonal second person perspective of empathically experiencing the body of the other through our own bodies. In [31], Merleau-Ponty goes deeper into how our bodies are in this way interconnected through body language and gaze.

Merleau-Ponty’s phenomenology of the body differs from Dewey’s aesthetics [32] in that he was not primarily concerned with experiences that require a certain framing, such as aesthetic experiences, but with the body in “unframed” everyday living. In everyday living, the meaning of a certain movement, like turning your head, differs with context. The same head movement could be part of watching out for cars while crossing a street, or part of a headshake to signal disagreement. From a phenomenological perspective it thus becomes meaningless to ask for the meaning/experience of a specific body movement decontextualized from the totality of the life of that person. As with all design, it is important to be aware of the multiplicity of possible bodily user experiences that can arise from a product in use, depending on the users, their tasks and use situations. While first- and second-person perspectives are crucial for successful soma-based design, such practices do not eliminate the need to explore in depth the different contexts-of-use for the product and be open to the fact that it might be used in ways we as designers can never imagine.

2.2. The First-Person Perspective in HCI Studies and Design

Different perspectives on the body, third-person (I-It), second-person (I-You) or first-person (I-Me), inspire different approaches to design. Most user-centered design projects are done from a third-person perspective; observing, interviewing, and testing on users, but not stepping into their shoes (even if, e.g., Beuthel and Wilde attempt to literally feel someone else’s disability or bodily condition to inform the design process [33]). In contrast, the first-person approach that is exemplified by the cases in this paper uses the designer’s lived body as a resource in the design process. Doing this requires a certain practiced sensitivity to the kinesthetic “feel” dimension of interaction design [29,34], and allows for a close loop between design and user experience.

The first-person perspective does, however, create a blindness about the ways in which our lived bodies are different, and how these differences color our user experiences. One can imagine having a different body, e.g., being short or tall, skinny or obese, but one cannot experience being another body. The closest we get is through a second-person perspective on the body, where the designer uses his or her body as an instrument for feeling the bodily aspects of the user experience of the other in co-design practices.

The first-person engagement with design is not new to HCI. In ethnography, the first-person, subjective perspective, has been used to introduce the observer’s/researcher’s subjective voice in so-called autoethnographic accounts of experiences. In HCI, autoethnography has been a way to get at bodily experiences. One example is in Höök’s study of her own horseback riding experiences and how those could be translated into design [35].

In design work, the first-person perspective has been introduced in works on autobiographical design [35,36,37]. As Neustaedter and Sengers frame it, such a design process “rest(s) on the user’s genuine need for the system”, which allows for “a much tighter coupling between user input and implementation” [38].

Beyond using their own needs as input, the experts who participated in the writing of this paper have all used design methods that not only start from their own needs, but strongly engage with their own somatics, tying their work to the felt dimensions of movement.

2.3. Aesthetics—Building on the Pragmatist Movement

Aesthetics is a way to examine connections between sensation, feeling, emotion and subjective understanding and values. A shared starting point for many of the authors of this paper lies in the writings on experience and aesthetics of pragmatist philosopher John Dewey [32] (often used to define UX in HCI-literature, e.g., [39,40,41,42]). Dewey saw an aesthetic experience as related to, but also set apart from, the general flow of experiences in our everyday life, with a clear beginning and end. Using the analogy of a mountain peak that emerges from the land around it, aesthetic experiences represent an intensification of life experiences. Significantly for Dewey, an aesthetic experience was something that we can refer to afterwards in definite terms: “An experience has a unity that gives it its name, that meal, that storm, that rupture of a friendship. The existence of this unity is constituted by a single quality that pervades the entire experience in spite of the variation of its constituent parts” [32]. As any experience is always unique, even the mundane, everyday ones, Dewey set aesthetic experience apart with his emphasis on “that” meal, storm, or rupture of a friendship. This emphasis on aesthetic experience as something outside the ordinary is problematic with regard to some of the somaesthetic design aims outlined below, as we may also want to talk about the aesthetic experiences of the everyday, of the repeated engagement, or experiences that are perhaps not so memorable that we can talk about them as “that” experience. Note also how the beauty of an aesthetic experience is not equal to only the positive emotions or safe experiences. Scary or disturbing experiences can be considered aesthetic as well.

A second insight we bring from Dewey’s account of aesthetic experience is how we can achieve them—both as artists/designers and also as viewers/users. A little-known historical fact regarding Dewey’s theoretical development is that he studied for 16 years with the somatic pioneer F.M. Alexander, and this practice had an enormous influence on his pragmatic philosophical writings about aesthetic experience [43]. Dewey saw aesthetic experience not as something that happens to us, but as something we engage in, actively, subjectively. When we craft or engage with our aesthetic experience, we should, according to Dewey, attempt to “empty” the material (out of which we build the experience) of all its potential. We should make use of every element that is used to create it, letting the different parts be explored and exploited to their fullest.

2.4. Somatics and Somaesthetics: Bringing in a Stronger Emphasis on the Soma

Later philosopher and pragmatist Richard Shusterman further developed Dewey’s pragmatist stance towards aesthetic experiences by introducing a theoretical framework he named somaesthetics [44]. By putting together the word soma, with aesthetics, to refer to our sensory appreciations, Shusterman draws our attention to the importance of our bodily movements, sensations, and somatic training, as part of our ways of being and thinking—but more importantly here, the aesthetics of those experiences. We draw upon Hannah’s definition of somatics and soma here: “Somatics is the field which studies the ‘soma’: namely the body as perceived from within by first-person perception. When a human being is observed from the outside—i.e., from a third-person viewpoint—the phenomena of a human ‘body’ is perceived. But, when the same human being is observed from the first-person viewpoint of his own proprioceptive senses, a categorically different phenomenon is perceived: the human soma” [12].

In developing aesthetics of somatic experience, Shusterman’s aim was to take Dewey’s pragmatic stance towards aesthetic experience even closer to the realm of life and practice through bringing the living, sentient body into the equation. While Dewey and others had mentioned the body and “the lower cognitive skills” that enable us to experience beauty, their interpretation of aesthetics was not fundamentally grounded in the movements of our bodies and the sensibilities developed through somatic training—in contrast to the work by Shusterman: “My concept of soma as a living, purposive, sentient, perceptive body or bodily subjectivity provides an altogether different direction. We can, as I have tried to demonstrate, improve our perceptual faculties through better use of the soma” [13].

In the second sentence, he points to the possibility of improving through “better use of the soma”. While training your aesthetic sensibilities has been a long-standing aim for anyone interested in the arts, Shusterman emphasizes that the route to improved aesthetic sensibilities goes through active engagement through the soma.

Shusterman’s aim is grounded in the original purpose of being a philosopher: to improve oneself. He does not take aesthetic experience as a given, an experience to be studied as an object out there, but instead wants to engage us all in actively, creatively changing and improving our experiences. By educating ourselves somatically, we are engaging in “the highest art of all—that of living better lives” [13].

Shusterman’s somaesthetics has been informed through a range of bodywork traditions and methodologies, from yoga and tai chi to more contemporary somatic movement education methods such as those developed by F.M. Alexander (Alexander Technique) [45] and Moshe Feldenkrais (Feldenkrais Method) [46]—both of which focus on the use of touch and movement as information for learning more effective coordination of body movement. These bodywork practices emphasize skills that we have previously learned, often at a sensory-motor level, that have become habitual and often below conscious awareness (as in walking or sitting)—which is in general a good thing, so we can operate in daily life without having to consciously think about and plan every movement. But we also adopt bad habits that can cause pain and loss of agency, making us unnecessarily restricted in our movements, thus leading us to miss out on the richness and aesthetics of our own somatics.

Engaging in different bodywork practices to rediscover or reacquire skills can only happen through first-person experience and reflection. Novel ways of engaging with our soma cannot be ‘put into us’—we cannot be moved into shape. The concept of self-agency in somatics practice is key. Self-agency is the result of the reflective practice of self-observation coupled with intention. It is only when we move that we can experience “being moved” emotionally, physically, and analytically, through our own somas [47]. A limited repertoire of movement becomes a limited repertory of experiences [46].

But what sets apart aesthetic movements of the soma from just any movement or any experience? In his writings, Shusterman repeatedly comes back to the need to disrupt habits, to focus and become more aware so as to be able to discern the different parts of a movement or emotion, feeling every small change in your body. For example, training your breathing to achieve better focus, or correcting your eating habits from what is “habitual” to shape new, more reflective, thoughtful actions. Returning to Dewey, Shusterman discusses the difference between “routine, un-intelligent habit” and “intelligent or artistic habit” that is fused with thought and feeling. To Shusterman, the art of somatic reflection and conscious control is thus itself a “refined intelligent habit emerging from and coordinating a background of countless other habits that constitute the developing bundle of complex, unstable, opposing attitudes, habits, impulses we call the self” [44].

These discoveries, or learnings, may lead to better appreciation, richer more meaningful bodily ways of being in the world, and more beauty. Shusterman talks about an “awakening” from mindless, joyless behaviors (referring to Thibaud’s idea of an awakening). This training is important for artists and designers to refine their designerly skills and consequently their designs, as well as for users to be able to experience somaesthetics in interaction and appreciate richer bodily ways of being in the world.

That is, when we speak of the ‘user experience’ of some design, it will be an active somatic stance towards engagement—a living, purposive, sentient, perceptive body or bodily subjectivity engaging in meaning-making processes. The user experience is not determined in a first brief encounter with some particular design, but is a co-constructed aesthetic appreciation that arises over time. It requires an active engagement (both by designer and user), resulting in an improvement of perception, bodily movement, emotion, intentionality and meaning-making. It aims to change those involved, increase their skills.

3. Methods

The insights presented in this paper derive from two phases of investigation. First, we conducted a workshop at the ACM SIGCHI annual conference [23] to explore the first-person perspective, attracting some of the leading experts in the field of soma-based design. Second, during the writing of this paper, we continued the analysis of theoretical concerns, engaged in ongoing dialog about our practice and research methods, discussed design exemplars not presented at the workshop, and conducted an extensive literature review. Overall, our research method can be characterized as a Research through Design (RtD) analysis. Let us describe these activities in some more detail before we turn to the data the two phases generated.

3.1. Move to Be Moved Workshop

The workshop was divided into two main parts. The first part involved sensitizing and foregrounding somatic and experiential sensibility through a series of first person experiences. For example, to hone the participants’ abilities to be aware of and be empathetic towards oneself and others, we performed a Collaborative Walking exercise devised by one of the authors [25] (drawn from the Body Weather performance training methodology [48]). All workshop participants stood in a line, shoulder to shoulder, very close to one another, with a red thread joining the whole line, from the first participant to the last, through their mouths, see Figure 1. We were asked to close our eyes and then walk together, very slowly, without losing contact with our neighbours. After the exercise, we were asked to articulate our experience, sharing any insights or discoveries on walking. This transported us to the space of felt experiences—reminding us not only intellectually of why we were there, but somatically. It also provided a shared somatic experience of performing a simple movement exercise, but with heightened awareness. The exercise itself was selected for its close similarity to the everyday act of walking, but reframed in an unusual configuration for the purpose of defamiliarizing the habitual—a somatic principle we elaborate upon in Section 4.2.1.

To fully engage with the issues of designing for somatic experiences, several of the participants had brought soma-based designs so that they could be tested, felt and discussed. We all engaged in hands-on first-person experiences with each design. After testing a system by sensing, feeling, listening to and participating in the interaction, and probing our own somatic responses, we were ready to discuss key aspects of the design, the experience, or the design process with its designer. One of the key aspects that we explored was to notice how the exploratory exercises and interactions affected our physical state, and to bring that state into our dialogues and discussions. This deepened our sense of what it means to engage end-users in aesthetic, movement-based interactions—and made us more honest.

3.2. Research through Design Analysis

The second half of the workshop (and the writing of the paper post-workshop) was spent in discussion, aimed at articulating aspects of our research, such as key aesthetic and material considerations, design methods and theoretical perspectives, as well as views on the politics of the body and its role in interaction design.

One reason HCI struggles with the first-person perspective lies in how we ‘prove’ the validity of our knowledge contributions in the academic HCI and interaction design fields. Traditionally, HCI, given its heritage from cognitive psychology, has put emphasis on the final user study as the ultimate—objective—proof that your design does what it claims. Replicable user studies are seen as a sign of quality [49]. But in addition to this view, the RtD community has successfully argued that the process of designing can serve a similar role to that of the “final user study”. In the design process, we follow different design paths, some ending in failure, some in success, occasionally obvious through involving end-users in testing early ideas, but often the failure or success can already be determined when sketching. The decisions rely on design judgments [50], on our ability to imagine future interactions, on our feel for aesthetics, and on our empathy with a future imagined user [26]. Sketching and form-giving become the analytical thinking process. The design process becomes the evidence of why the final result is valid [27,51,52].

As the material we design in is best framed as a socio-digital material [53,54]—consisting of the various digital and other materials we use to create our design, and at the same time speaking of a use practice and behavior unfolding over time—a design cannot be understood solely by inspecting it as a static object. It comes to life as we start making it part of our practice.

In RtD, specific design exemplars serve as ‘definitions’, ‘facts’ or as a physical hypothesis [55], organized in design research programs (in design studios) [10] or created for paradigms [56], used to extract knowledge in the forms of annotated portfolios [16,57], strong concepts [58], methods, and experiential qualities [37]. None of this work is or can ever be replicable, as design is a creative practice [59]. But it can be internally consistent and fulfil criteria such as being contestable, defensible and substantive [60].

Together, we decided to analyze the design exemplars brought to the workshop (as we had all jointly experienced them), the design methods that shape our practices, and provide an account of the theoretical concepts that has started to shape and be shaped by our design practice—definitions, strong concepts, experiential qualities, and so on.

What RtD literature does not always acknowledge is the strong role of the designer/user—the subjective attitude, aesthetic sensitivity, politics, taste, values and bodily experiences—governing design choices and use. In the analysis of our design exemplars, design practices and theory contributions, started during the workshop but later continued in the work behind this paper, the role of the designer/user and their felt experience of the interaction were repeatedly emphasized. This can perhaps be contrasted with the current artificial intelligence trend turning towards data analytics as the solution to our design problems. Data analytics can be used as a way to understand user behavior (as in A/B-testing), or even as a way to create designs automatically [61]. Though we are not opposed to making use of data analytics, we argue strongly in this paper for a continuation and even a deepening of designer-driven experiential practices and shifting of the perspective to instead seeing data as a material in the design—with and through designers and users.

4. Data: Design Exemplars, Design Practice and Theoretical Underpinnings

Let us now turn to the data generated by the two activities (the workshop and the RtD analysis). We divide our data into: an autobiographical account of five design exemplars, jointly experienced at the workshop; first-person design methods and material encounters soma-based interaction practitioners typically engage with; and some of the theoretical underpinnings of our work (that is, not the generic concepts from philosophy introduced in the background above, but the concepts specifically arising when creating or engaging with soma-based designs).

4.1. Annotated Portfolio of Design Exemplars Experienced at the Workshop

Let us start with an autobiographical account of the five designs that were brought to the workshop, illustrating how a first-person perspective is necessary in designing, experiencing and communicating what they are.

4.1.1. BrightHearts

The first example is BrightHearts [62], an interactive artwork and biofeedback application for mobile phones and tablets, developed as a tool for reducing pain and anxiety experienced by children undergoing painful medical procedures. Decreases in heart rate are used to modulate the diameter and color of a collection of overlapping concentric circles, see Figure 2. The more relaxed you are, the less bright and warm colors appear at the interface. The application invites users to explore ways to decrease their average heart rate through a combination of muscle relaxation, slow breathing and relaxing memories—decreases are rewarded with gentle chimes, and new layers of colorful circular imagery. The project continues Khut’s on-going exploration of the somaesthetic potential of biofeedback as a way to facilitate deeply embodied experiences, self-efficacy, and reflections on the ways we experience and apply our attention to voluntarily influence our stress and relaxation responses. First-person experience is central to design of these displays—the movement and responsiveness of the various layers and sounds is literally coded onto and around the artist/designer’s own physiology, and adjusted to achieve a balance between responsiveness to changes in heart rate, and movement (animation) qualities appropriate for the application’s relaxation-training aims. These mappings are then tested with other users. Works such as this are designed to respond in an aesthetically compelling and rewarding way to voluntary adjustments to nervous system reflexes—the responsiveness of the sounds and visuals to the dynamics user’s nervous state—the sense of how these displays ‘fit’ in relation to these movements—can only be fully appreciated by the person interacting with the display, in much the same way as with clothing—where ‘fit’ is best judged by the person wearing the clothing item.

Here is a first-person account by Höök (to appear in [63]):

I tried the BrightHearts app at a workshop that I was running. I was quite stressed about the whole situation as I was trying to keep track of the progress of the workshop, orchestrating activities, making sure everyone got to try out all the demos participants had brought, trying to facilitate discussions, pushing us towards the interesting topics we wanted to debate. Once I put on the system, I was also stressed about the fact that everyone could see how my relaxation was progressing through watching the interface (portrayed on an iPad placed on the floor between us). I am usually able to calm myself down through deep breathing and turning my focus inwards, but here it took me a while to calm down the animations and make the color go from warmer to colder colors. As my mind was scurrying in all directions to do with the workshop and the situation at hand, and the chitchat and discussion by people sitting around me distracted me. But the animations of my heart were also mesmerizing. I remember wishing that I could go and sit in a corner on my own to explore my connection to the system.

4.1.2. The Soma Mat

The Soma Mat uses directed heat stimuli to subtly guide your attention to different body parts [64,65,66], see Figure 3. The mat can be used as a support for exercises aimed at increased body awareness such as guided meditation, for body scanning, in Feldenkrais sessions, or simply for relaxation. Integrated into the mat are a number of heat pads that can provide spatially directed heat pulses, and a pair of speakers. An application on a mobile phone is used in conjunction with the mat, with a number of pre-recorded audio sessions aimed at increasing body awareness. The instructions engage participants in very slow movements, while simultaneously directing attention to the different parts of our bodies and how they interact.

The carefully tuned thermal stimuli serve to guide and sustain attention to different body parts. While engaging in these somatic exercises, you sometimes feel quite vulnerable. Pains, aches and bodily memories surface during the exercise. The materials of the mat are chosen to provide enclosure and softness to help you feel taken care of and feel empathy with yourself. Workshop participants tried out the mat and instructions.

Here is a first-person account of the Soma Mat, used together with a second system, named Breathing Light (it subtly follows your breathing by dimming a light [64]). This account is by Jonsson using the system in a different setting—in the open office space of the research center he was working in:

The Soma Mat and Breathing Light demo was situated in a corner of an open office space. The oval mat on the floor was partly secluded by something looking like a lamp shade with a curtain of cotton strings hanging down towards the mat. As I laid down on the mat, I was first disturbed by the people passing by, feeling slightly embarrassed by the act of laying down on the floor at work in the middle of the day. As I positioned myself on the mat, the sting curtain from the Breathing Light somewhat helped to block out the surrounding activities.

My mind was still occupied with the various activities I had been engaged with during the day, and I felt a bit reluctant and stressed, questioning whether I really had time for a session at this time. I tried to shake off these feelings and took a deep breath, trying to relax. Immediately I noticed how the soft light above my head changed with my breathing, dimming out slowly as I exhaled and intensifying again when I inhaled.

I picked up the phone and put on the headphones, opened the app and started the session. As I could hear the familiar voice starting to talk in my headphones, I leaned back, closed my eyes, and took some time to reflect on how I actually felt. I could feel the stress lingering in my body, and the throbbing pain in my heel, that I never really seem to entirely get rid of, the tension over my neck and shoulder, and spikes of pain in parts of my back. The recorded voice continued by asking questions to help guide my attention to different parts and experiences of my body—the different limbs, the skin, the breathing, the head, my back. As I directed my attention to different parts of my body, the mat subtly heated up underneath those body parts. The warmth came on slowly and dissipated slowly, creating a soothing sensation.

After a while, the voice was replaced by a calm rhythmic soundscape, similar to the sound of waves on a beach, while the slow pulses of heat continued to move between different parts of the body, moving from heels to shoulders and from left to right. Through my closed eyelids, I could still notice how the intensity of the light above my head changed in close correspondence with my breathing. The warmth from the mat had spread throughout my body, and I noticed the different thermal sensations of heat coming from the mat, and the coolness from the surrounding air.

At this stage, I felt entirely secluded from the surrounding space, enclosed in a safe and calm space, and my attention was subtly guided by heat and light towards different body parts and functions. I felt immersed, in the space as well as in the experience emanating from the mixed sensations of the external stimuli felt through my body, and the sensation of feeling my body through the external stimuli. My initial distinct sensations of pain had gradually been replaced by richer and more nuanced sensations, which let the sensations of pain to fade into the background, overshadowed by the richer bodily sensations of my breathing, different sensations of heat and light, and the sensations of different parts of my body resting on the floor.

When the voice finally asked me to open my eyes and slowly get up, I woke up to a very different state of mind compared to before the session. Initially I felt a bit dizzy and foggy-headed, but entirely calm and present in my body, with the sensations of tension and pain much less prominent. Gradually, I felt increasingly revived and energetic, and more present in the now, a sensation that lingered for hours after the session.

4.1.3. The Tail and Ears

Our third example from the workshop is the Tail and Ears project [67], see Figure 4 and Figure 5. These are two design projects involving mechanical body extensions for use in theatre settings. The tail responds to the swaying of your hips, using an inertial measurement unity, with accelerometer and gyro, on the hip. The ears are controlled by bend sensors on a glove.

The way the creators describe the background to these two projects is “The lived body is our experienced body, the body through which we live our lives, which is different from seeing the body as an object in the world. Designing for the body with the first-person perspective of Merleau-Ponty makes us aware of how technology is incorporated into the experienced body, and how it thus changes us” [67]. The idea is that the tail and the ears extend the actor’s body, creating a richer array of expressions. In [68], Svanæs reported his own first-person experience of wearing the tail that he had created: “One of the most interesting experiences for me was when I took off the tail after wearing it for an hour, and had a surprising physical feeling of being tailless”.

Workshop participants engaged with the tail and ears in social, playful, open-ended ways. For example, one participant controlled the ears that were placed on another participant’s head.

Here is a first-person account of testing the Tail and Ears at the workshop by Höök, taken from [63]:

I got to try the Ears and Tail at a workshop. At first, I felt mildly embarrassed walking around with a tail wiggling evocatively behind my back, arousing smiles and embarrassed laughter from my workshop friends. Once I overcame my discomfort, though, I found the experience strangely expressive. To make the tail wiggle, I had to move my hips in different patterns, for example: a circular motion in the sagittal plane, or a smooth sashay from side to side, or a tilt as if I was putting the tail between my legs. As I did these movements, I could feel the tail as a clear substance, a counterweight, not too heavy nor too light, changing in response to my movements.

The circular movements of my hips encouraged movements that I associate with a feminine, sexy, flirtatious way of walking. As the flirtation was directed backwards, behind my back so to speak, I repeatedly turned my head, cocking it, nodding, smiling, to check out the effect on others behind me. My whole body posture and movement schema was altered. The feeling I had was that my tail happily wriggled in a sashay, almost slightly aggressive, “look at me”, cat-like manner.

Putting the tail between my legs put bodily reality and substance into the expression of cowardice and shame. I could feel my face crumble into a sad embarrassed expression—again looking behind me to see the effect on others, but now looking under my brows, with a facial expression saying “excuse me”. The tipping of my hips required a conscious movement that could not be done while walking, or at least not in the few minutes I wore the tail.

The flapping ears affected me less powerfully—and in some ways serves to show how soma engagement is not easy to design for. Rather than an experience where my movements immediately spurred the ears to move, feeding back into my experience of myself, I felt more removed as my hand made gestures having some effect on the ears which I could not sense or see myself. Perhaps training in front of a mirror to synchronize these movements with my facial expressions or feeding off the reaction of others watching my ears might have created a more embodied experience: I am guessing that this is what the actors had to do in order to make their performance expressive and seamless.

In the workshop, one participant figured out that she could wear the glove while another participant wore the ears. This became a playful interaction between them where she tried to match his facial expression by flapping the ears in the right tempo and direction. As it turned out, we all felt we knew how the ears should be turning to express certain emotions, probably from our experiences of cats, dogs, horses and other domesticated animals, as well as from various media, theatre, movies and so on.

4.1.4. GangKlang

Workshop participants also tried out this mobile app that makes use of a person’s walking rhythm to generate a soundscape [69] as they stride along. GangKlang was inspired by work songs, and how they integrate the singing voice within a repetitive body movement. The sung words underscore and structure the rhythm of the activity. Fusing mind and body, they absorb the worker’s attention to the present moment [70]. GangKlang aims to re-emphasize the bodily dimension within the walker’s experience, supporting a first-person perspective by providing functionality for feeling into and engaging with the movement. The app detects movement phrases through which the user brings forth the walking process in interaction with contextual conditions and translates peculiarities of each phrase into characteristics of a soundscape in real time. This makes the user’s movements and occurring changes in the movement audible. The user’s attention is directed towards the present walking movement and how its characteristics are influenced by the spatial environment, the others, and the sonic feedback itself. It provides an experiential way to explore how our walking movements are an expression of what ‘moves us’.

During the design process of GangKlang and other instantiations of sonic interaction for walking, it became apparent that sound and interaction designers need to experience from a first-person perspective the interplay of body movement, sound and mobile context.

Acknowledging the designer’s experiential awareness as an integral part of designing interactive sound, the mobile App was extended to a rapid prototyping system [71]. This prototyping system enables designers to (a) specify the sonic response to a predefined set of walking movements in an editor interface, (b) use a mobile App to experience how their design for sonified walking unfolds in interplay with body movement in mobile situations, and (c) analyze process data that is collected by the mobile App to reflect on their sonified walking experience afterwards. In this way, the prototyping system provides a short design-experimentation-feedback loop to explore and understand possibilities and challenges in the design of interactive sound.

Here is a first-person account of GangKlang by Hajinejad:

Putting on the headphones and positioning the smartphone in my trouser pocket, I felt like preparing for something important. I started walking and was highly concentrated on listening, similar to the beginning of an unknown concert. The first steps passed by in silence and increased the suspense, and then, evolved a regular rhythm of marimba sounds. I placed my attention on the sensation of moving my legs, how I placed my foot on the ground and listened to the response in the sounds. Focusing on both at once, my body movement and listening, was quite demanding. In order to return to a walking pattern as in everyday life, I had to shift my attention to the outside and the route.

I listened carefully as the composition became more complex, and tried not to lose my walking rhythm. Later, as sustaining flowing sounds joined the percussive ones, my steps felt buoyant and energized. I started pondering the relationship between my walking movements and the sounds, and my thoughts drifted away. It was only with the silence phase that nudged me out of my thoughts and drew my attention to the walking process again.

As these different feedback modi repeated for some cycles, the sounds seemed not as interactive as in the beginning, and more like meditative background music. The feedback was not exciting anymore, and I stopped listening consciously. However, the calming sounds had a positive effect on my thoughts, and the silence phases kept bringing me back from my thoughts. Altogether, the sonic feedback made me walk with more awareness and yet without judgement. I felt like having walked with a companion.

4.1.5. Embodied Encounters Studio

The last example system engaged during the workshop was the Embodied Encounters Studio [72,73]. Hummels has used this system to visit well over 80 inspirators around the world, envisioning and exploring together hands-on what potential futures can be. The studio consists of an evocative toolbox with three components: (1) the Embodied Ideation Toolkit: a set of tangible magnetic non-descriptive objects, (2) Hexn, a modular tangible notation system existing of hexagonal transparent discs with icons, and (3) Inspiration Cards, small RFID-enhanced laminated cards showing pictures that relate to content in the database, e.g., slides and videos that are shown on a stationary iPad, see Figure 6.

The Embodied Encounters Studio was available during the interactivity track at the ACM SIGCHI annual conference [23]. For the workshop, only the Embodied Ideation Toolkit was used. The idea behind the design is that playful encounters using these objects allow two or more participants to brainstorm together, engaging in participatory sense-making, while at the same time, the encounter is being documented through video, audio and sensor data. The studio supports nonverbal communication and social coordination in action, instead of a more dialogue-oriented discussion in which we exchange messages originating in one ‘mind’ that are received by another. Hummels states: “Embodied Encounters Studio aims at offering a more informal, playful, intimate and engaging setting, where participants can take a first-person perspective and explore concepts hands-on [74]. Our previous research has shown that bodily engagement seems to drive participants towards concrete ideas and away from the abstract, and to stimulate interaction based on personal experiences. Moreover, bodily encounters seem to lower the threshold to merge the perspectives from people with different backgrounds” [75].

Here is a first-person account of testing the Embodied Ideation Toolkit at the workshop by Höök from [63]:

I was immediately drawn to the magnetic objects. Hummels had included old radio antennas that could be extended, serving as “pointing” devices or as a way to pick up the magnetic objects. I extended the antenna and started picking up objects with it. Some objects were too heavy to stay on the antenna and would fall off. Other participants playfully chipped in with their antennas or other objects to help out, or started dragging around objects, attached to one another through the magnetic force. The effect on all of us was immediate—we instantly became playful, engaging one another through joining forces or stealing objects from one another. We touched the objects, explored their qualities: the space they occupied, their velocity, their affordance for movement, whether they could be combined into larger forms without breaking apart, and their capacity to sustain play between us. We touched, fiddled, built, destroyed, explored, and engaged both with ourselves and with one another. It opened a path to social play.

4.1.6. What Unites these Diverse Design Exemplars?

First, note how there is a diversity of aims in these applications: supporting learning skills in emotion-regulation and relaxation; understanding your own walking habits; spurring playful discussion with one another; whole-body movements; or adding to our bodily expressiveness. This shows how a first-person soma design stance can generate many different applications in diverse domains. In fact, turning to the literature on soma design, we see a field that is growing, engaging with one application domain after another (see, e.g., [33,42,53,62,64,76,77,78,79,80,81,82,83,84,85,86,87,88]). Some even claim that a slow, soma-based, first-person engagement will be relevant to all forms of interactive design work [63], but perhaps in particular designs with the plethora of upcoming interactive materials, such as shape-changing interfaces, Internet of Things, sensor networks, and all forms of movement-based interactions.

But what became more interesting to workshop participants was not so much the uniqueness of each application, but the similarities between them. When engaging through a first-person perspective, it was clear that they shared a certain designerly, aesthetic sensitivity clearly coming through in the design. We ‘recognized’ this similarity, but this was not apparent to us through studying or comparing their surface form or shape, or the functionality or business model they provided, as might be the case with a website design or mobile app. The familiar feel of these systems revealed itself only through engaging yourself, through a first-person experience. A range of insights arose through this first-person experience.

Apart from the obvious focus on movements of different kinds in all of them, the systems share a playful approach in which habitual movements from our everyday life are disrupted. Laying down on the floor and slowly engaging with very small movements, walking differently, brainstorming with others through manipulating fun objects, or flapping ears or sweeping your tail are all non-habitual movements. While all are within our capacity as human beings—within our human morphology, as Sheets-Johnstone would phrase it [47]—we might never have moved in this manner ever before, or perhaps not moved in this particular manner since we were children. Reconnecting with our moving selves through non-habitual movements connecting us to novel interactions, and thereby novel experiences, was inherently interesting, making us playful and engaged.

Each design also offered a delightful and engaging experience, creating novel meaning-making processes. They engaged us somatically, making us connect emotion, thinking, movement, and expression into one subjectivity. Each interaction experience transported us into a different space, engaging our senses, awakening our corporeal sensuous perception, and encouraging us to be guided by our subjective experiences, alone or sometimes together, in order to open to ideas beyond rational mindsets, to evade well-trodden tracks, concepts and relationships.

This does not mean that any combination of movement, emotion, thinking and expression will ‘work’. This perhaps most clearly comes into focus when the interaction fails—as with the flapping Ears by Svanaes. As the interaction separated input from output, the meaning-making flow did not properly connect your first-person experience of moving your hand with the flapping of the Ears. Instead, others could see the Ears flapping, and you could second-guess what others would see; engaging, in effect, the wearer in second- and third-person interpretations of what was going on. Closing the loop—the affective loop [89], the feedback loop—is of key importance. In effect, this is the same argument as provided by Norman and Tognazzini’s critique of mobile phone gestures, where gestures are not properly connected to feedback at the interface [90]. But our insight here goes beyond providing relevant feedback to make sure learning of a function can be made easy, creating the correct affordances. Instead, the feedback coupling (when it worked) engaged, on a profound level, connections between movements, emotional and intellectual experiences. Beyond functional affordances, aesthetically enhanced actions and sensations are developed in these applications—as ways to expand the user’s repertoire of movements and orientations, to differentiate and practice new or forgotten patterns for action and response. As discussed above, we see aesthetics as a path to ‘examine’ and improve on connections between sensation, feeling, emotion and subjective understanding and values [91]. Each interaction with the systems introduced above transformed our somatic selves.

4.2. First-Person Methods and Material Encounters

Let us now turn to the second set of data: our methods and material encounters. These were compiled during the discussion at the workshop and in subsequent collaboration during the construction of this paper.

Attending to your first-person experience is, on the one hand, easy, as your body is always there for you, and as Thecla Schiphorst phrased it during the workshop: “the body is generous with tips”. But at the same time, it requires attention and focus. To discern all the small details of an experience, you might need to slow down, disrupt or engage with some specific practice in order to ‘see’ yourself, your movement, emotion and the linkages between all the different parts of your first-person presence. It is sometimes a slow learning process, but at the same time a rewarding process, as it increases your presence, understanding and aesthetic sensibilities overall. In Section 4.2.1, we describe some example tactics to achieve a richer first-person perspective.

Second, interaction design is always the study of the socio-digital material: the coming together of people with designed interactive artefacts, and the behaviors and experiences generated in the combined material of people, practices, tools and artefacts. It is not enough to attend to your own soma, you also need to touch, feel and interact with the digital/technical materials that you aim to shape into the envisioned application. Interaction design always thrives on the affordances of the technologies and materials used to shape the artefacts. In Section 4.2.2, we turn to some of the tactics used to engage with technology to attain deep material connoisseurship.

4.2.1. Attending to Your Own Soma

The design material in the exemplars above in a sense consists of two parts: on the one hand the digitally mediated materials, such as programmable heat pads or bend sensors, and on the other hand our bodily selves—our somas. Both can be, and are in fact, shaped in and through the interaction. Our movements change dynamically in response to kinetically dynamic possibilities in our environment. Those possibilities, of course, include any tools and artifacts we create, cultural practices, and so on. There is infinite variation in the world; in our felt experience, but also in the variations of the social and cultural landscape we are shaped by and actively take part in shaping, reinforced and reinstated through all our everyday acts.

There is, luckily, a path out of the dilemma of the infinite space of possibilities in which design may shape the world, and that is to engage with your own body, your own movement, your own felt emotional experience and sense of aesthetics, and let the design process feed off of your own felt understanding and experience. This is, on the one hand, easy, as your body is always there for you. You just need to return to it, listen to it, and discern all the different small nuances of movements and the meaning making they create. On the other hand, such processes are demanding, and they might require guidance. This is particularly the case, as these methods will also change your body and perception, increasing your aesthetic sensibility. Here, it is important to make clear that you do not need to become an expert in yoga or meditation or some other bodily practice to do this. Let us provide a brief introduction to some of the methods and tactics used by the experts in this field.

Change and Interest. Directing attention to your own soma relies on the interplay between two different cognitive processes that Shusterman has developed further from William James: change and interest [92]. By change, he refers to the importance of subdividing the bodily experiences into more specific body areas or functions and then engaging in activities that shift focus from one area to another and back, in order to provide a more nuanced and rich perception of fine-grained movements. The notion of interest, on the other hand, deals with finding means to achieve a sustained attention towards the part of the body currently under examination. Or as Shusterman phrases it: “To reach precise bodily introspection the key is to direct our focused attention first to one part then another, a clearer sense of relations of parts to whole can be obtained. This transition of focus, provides sense of change, it also renews our interest in each new body part” [44].

Lee, Lim and Shusterman have together applied these six tactics to design [36]. To provide for heightened attention and interest to the body, they suggest:

“1. Questions: Asking questions about different aspects and relations of what we perceive.

2. Division into parts: Subdividing the body and directing our attention to each part, one by one.

3. Contrasts of feeling: Discriminating the different feelings in one part of the body from those in another.

4. Associative interests: Making the noticing of what we are trying more precisely to feel a key to something we care about.

5. Avoiding distracting interests: Warding off competing interests to what we are trying to attend to and feel.

6. Pre-perception: Preparing our attention to notice what we are trying to discriminate in what we feel” [36].

This method was used in the design process behind the Soma Mat and the Breathing Light described above.

Disrupting the Habitual—Estrangement. Another method frequently mentioned, as well as acted out in the Collaborative Walking exercise above, is to slow down or disrupt a habitual movement to be able to discern small changes, to feel how different parts of your body are connected, to note how the movements relate to your emotional experiences, to enjoy sensations or feel pain, to be engaged [25]. Defamiliarizing habitual movement patterns is a core principle in many somatic practices [45,46,93], based on the rationale that only by bringing to the surface what has become automatic and tacit, can we change deeply ingrained habitual patterns of movement. Often this is achieved by providing constraints to the habitual pathways in a movement pattern. In terms of movement-based interaction design, this awareness and playful approach to re-patterning movement possibilities enables us to explore new kinds of meaningful movements in a structured way by drawing on techniques that have already been developed and refined in somatic and dance practices [25].

This method was used in the design of, for example, GangKlang.

Somatic Connoisseurship. Schiphorst has argued for the value of applying somatic connoisseurship in the design and evaluation of aesthetic soma-based designs [1]. The practice of somatic connoisseurship highlights the significance of somatic facilitation as a role within technological design processes. The role of somatic connoisseurship characterizes expertise that is developed, expressed, and passed on through the constantly refining process of soma-based aesthetic practice. This practice develops expertise that can access and train experiential acuity, including observation, discernment, synthesis, empathy, and focus. The somatic body-based traditions require techniques that use attention, observation, and discrimination, applied to the material of experience for the purpose of self-cultivation. These techniques are developed through training that is tested and validated through the efficacy of practice. A somatic connoisseur is someone who is skillful in an embodied movement-based practice that guides, mentors and shares their insights with co-designers and participants, guiding collaborators in what to attend to, how to move and feel. Somatic connoisseurs may be our co-designers, or they might not know how to deal with the digital materials engaging researchers in interaction design. The practice of somatic connoisseurship may inspire transfer of qualities from their body-based practice to our technologically situated practices. The experts in the workshop reported bringing in somatic connoisseurs such as choreographers, chefs, or Feldenkrais practitioners.

Laban-Movement-Analysis. Another path to discerning movement as a language that bridges aesthetic potential with functional instrumentalization comes from the work of Rudolph Laban. He created Laban Movement Analysis (LMA) and its notational systems, Motifs and Labanotation [94]. Laban Movement Analysis has a rich history of practice that is based upon first-person experiential training of the moving body in relation to itself and its environment: the Body (what is moving), Effort (the quality of how the body moves), Space (where the body is moving in relationship to itself, others and the larger geographical movement that exist in the world), and Shape (which describes the interaction or relationship between the experiential moving body, others and the artefacts and the larger world in which it exists). LMA also includes somatic exploration of the body’s developmental patterns, personal movement signatures, proxemics and cultural movement patterns that influence our own and larger group patterns of movement style. LMA is valuable for how it explicitly bridges the first-person experiential with connection to the external environment, including how we relate to others. While LMA is considered a movement language born from Western knowledge histories, Rudolf Laban was influenced by historical teachers of self-cultivation such as Gurdjieff, Delsarte and Sufism. In HCI, it has been used as a means to connect experience and movement (e.g., [95,96,97]).

In LMA, the dimensions that address the inner experience of a movement touch on the underlying qualities of a first-person experience. The Effort and Shape dimensions are particularly interesting in this regard. To really understand the experiential meaning of these dimensions, you need to be trained in LMA.

4.2.2. Attending to Design Materials and Shaping Concepts

Attending to our own somas is half the story. The other half is attending, with a first-person perspective, to the design materials we use to create the dynamic gestalt of our designs. While debating where design ideas come from is too large a topic to deal with here, let us describe a few of the design tactics the experts in the workshop had found useful during different phases of the design process.

Autobiographical Design. Almost all the workshop attendees had, at one point or another, engaged in autobiographical design enquiry to shape their designs. That is, using yourself, your own experience and desires, in developing the design. This requires taking on all the roles in the design: we were designers, researchers and users of our own design concepts as they evolve. In those design processes, we have to repeatedly try different digital and physical materials, simulating interactions with manually controlled sensory stimuli, and testing them in situ to find the ones that would make sense. The interactions have to be simulated and acted out in order for us to really feel their impact on our bodily experiences. Simply imagining what they would be like was not enough to qualify the experience [65]. An interesting framework that helps to structure this process has been proposed by Trotto and Hummels: the Designing in Skills framework [72,73].

Embodied Sketching is an umbrella term to describe ideation design activities in the domain of embodied interaction that leverage the participants’ in-the-moment embodied experience to explore and design for particularly interesting physical and social activities [5]. Examples of embodied sketching include: bodystorming, where designers envision physical and social activities supported by technology and physical artefacts; and physical movement sketching—using movement to develop a design idea, to experience that movement, to transfer that movement to be experienced by others through imitation, and then to reflect on the movement through discussion. It includes physical movement as a frame of reference for future design, through inviting designers themselves to explore and trust their own bodies’ movement capacities as a tool of reference in that work [98]. Another example of embodied sketching is choreography of interaction—an approach where the product comes into existence through creating activities and movements, i.e., in the choreography of interaction. The choreography of interaction is the total system of user, product and other elements involved in interaction. It embodies a trinity of physical involvement, dynamic quality and expressed meaning [24]. There are also sensitizing activities, where designers hone their sensitivity towards a particular bodily aspect or aesthetic they want to discuss, brainstorm, and ultimately design for, by engaging with a bodily experience that supports this aspect or aesthetic in an interesting way (perhaps in a non-conventional one, like with techniques that disrupt our ways of acting or thinking) [5]. Lastly, there are co-creation ideation activities, where users play around with (test and appropriate) a technology prototype to support particular experiences [99]. All these ideation activities heavily involve the moving body and are meant to happen early in the design process, before the construction of fully-functional prototypes.

Data and Program Code. Data can be used to shape our designs by means of recent advances in artificial intelligence and machine learning. Machine learning designates a set of methods able to learn from examples (data) in order to take decisions (e.g., recognizing a movement, recognizing who is performing and how) instead of explicitly defining rules that would lead to these decisions. Applying a first-person perspective to machine learning allows for shifting the focus from technical computations to designer intentions. Indeed, as designers we can provide the system with examples of behavior or idiosyncratic goals and the system could learn from these examples. For example, in the design of music interaction, a performer-designer can demonstrate to the system personal movements and gestures, and the machine will learn this performer-specific vocabulary [100,101]. This is a powerful tool that can be used by non-technical designers to explore and create designs. As such, non-specialists used it recently in participatory design workshops in order to transfer high-level body-centered sonic interaction design ideas into working prototypes [102].

Yet another path is the data-enabled design framework by Bogers and colleagues [103]. By using data specific to each individual, we can imagine designs adapted to each, where a first-person perspective is at the heart of the interaction, fostering idiosyncratic goals and somaesthetic experience.

4.3. Conceptualizations Arising through Design Research

The various design exemplars discussed above serve as definitions/facts through which we, as expert design practitioners, can ‘feel’ and share our knowledge. We picked these particular ones as they were brought to our workshop where we could all interact with them, but there is a whole range of systems built along the same lines (e.g., [9,12,20,37,43,50,53,58,64,65,66,69,74,82,89,104]). While this is early days, we do not yet know which ones will be the prototypical or canonical exemplars that will serve as stylistic or orienting concepts to our soma-based design practice.

But we already note how some design elements and experiential qualities in our soma-based designs are re-used (borrowed, stolen, re-interpreted, transformed) in novel design work. By naming and articulating those and fine-tuning our articulations, we start forming our own research program, our own theoretical concepts. This is no small endeavor. As pointed out by Shusterman [13], linguistic tags can be used as a resource to improve perceptual nuances:

“Linguistic tags or descriptions, for example, can make a very vague feeling less difficult to discriminate by tying that feeling to words, which are much more easily differentiated. James argues, for instance, that the different names of wines help us discriminate their subtly different flavours far more clearly and precisely than we could without the use of different names. […] The rich and value-laden associations of words can, moreover, transform our feelings, even our bodily ones”.

Note how choice of language will not be an innocent choice here, but may shape what we are able to see and feel. Similarly, as design researchers, our choice of articulations to frame our knowledge and design insights will shape what we ‘see’. It is important to continue to bring more design exemplars to shape this space. But, in addition to what Shusterman’s explanation of what linguistic tags can do, our articulations need to also address the socio-digital materials—both our bodily experiences, but also how those are guided by and relate to specific design choices.

The articulations we have seen in our field so far, apart from the design exemplars, are reported in the academic literature as strong concepts (such as biofeedback loops, affective loops, somaesthetic appreciation, or somaesthetic touch), experiential qualities (such as slow, inwards-turning meditative experiences, uncomfortable interactions [105], scary experiences [106] or playful somaesthetics), and design methods. These strong concepts, experiential qualities and methods should not be seen as given, rigidly formed rules or patterns for interaction. They are conceptual lenses that help us ‘see’ and ‘feel’ potential design opportunities.

5. Discussion: First-Person Perspectives and Rigor?

Through our engagement with one another’s design exemplars, design practice and in our analysis, we note how successful soma-based designs all seem to share a structured, careful, thoughtful first-person engagement. While the experts may employ different strategies to become more somatically and aesthetically aware, there is a shared understanding that you need to employ structured tactics to access your own movements, somatics and aesthetic sensibilities. We slow down our movements; disrupt the habitual to help us grasp and articulate what is there; we direct our attention to specific areas (change) and put our sustained attention to it (interest); we playfully engage in movement.

A first-person perspective in design puts emphasis on the individual designers’ (or researchers’) subjective aesthetic experiences. But this also means taking seriously your own sensations and experiences as a prerequisite for communication and collaboration, for example within a design team. Given the “tacit” nature of bodily experiences, in that these kinds of sensations may be hard to articulate or verbalize, it becomes especially important to create common grounds for sharing such experiences. By engaging, as a group, in bodily and somatic experiences, these shared experiences can work as a common ground from which intersubjectively constructed meanings [107], or kinaesthetic empathy [108], can be created. In this process, we bring in the (digital and other) design materials early on, touching, feeling, interacting with them, thereby letting them ‘speak’ back to us [109], letting our design concepts thrive off their affordances.

This is not to say that we ignore what can be learnt through engaging with users outside the design team. Rigor will also come from testing and exhibiting our designs, as well as letting others experience and create meaning with them. But, as mentioned above, it is hard to get well-articulated feedback from end-users, unless they are trained in some aesthetic bodywork practice on the specifics of some designs. Often, our designs are open to interpretation, and the meaning and practice develops in dialogue with use, and we need to document and analyze those to provide depth in understanding. This requires methods for end-user involvement that share the kinds of long-term, deep engagements employed in participatory design (as it was initially practiced in the Scandinavian countries [110]).

While we emphasize that there is rigor in our practice, this is not in opposition to seeing design as a creative practice, engaging with our tacit aesthetic sensibilities, fundamentally a studio-based practice that will render different results depending on which studio and which designers are engaged.

6. Conclusions

Through the lens of the first-person perspective, we have uncovered some of the issues in addressing aesthetics in soma-based designs. The first-person perspective runs like a thread through the analysis of our design exemplars, the description of our design methods as well as our material encounter strategies. Theoretical concepts such as lived body, aesthetic experience, somatics, and somaesthetics provide some of the theoretical underpinnings for our first-person perspective. As such, they are shaping our design practice, but they are also being filled with new insights and experiences through our design work—blending and amalgamating into a new research program on soma-based design. Our aim is to demystify soma-based design practices, to show that, when it is done as part of a design research practice, there is rigor to both how the design is done and to the articulations of gained knowledge.

We believe this work is timely, as we face a new wave of technologies, entering into every walk of human life (in the era of Internet of Things, ubicomp, and big data—shifting from dialogue-oriented interaction to faceless interfaces and implicit interaction). There is a window of opportunity where we make designs harmonize with our basic human condition: our first-person, felt experiences.

But much work remains to be done, which is why we end with a call for action, inviting other researchers to engage with soma-based interactions. Some of our questions are ancient philosophical inquiries. We need to keep asking: what is (felt) experience? What is aesthetics? What is the ‘living body’? As expressed by one of the workshop participants: “there are so many things in doing this work that have confused me”. This is a call for action, for more research and design work, for a rigorous systematic treatment of aesthetics in soma-based design.