Stress Detection Using Experience Sampling: A Systematic Mapping Study

Abstract

Stress has been designated the “Health Epidemic of the 21st Century” by the World Health Organization and negatively affects the quality of individuals’ lives by detracting most body systems. In today’s world, different methods are used to track and measure various types of stress. Among these techniques, experience sampling is a unique method for studying everyday stress, which can affect employees’ performance and even their health by threatening them emotionally and physically. The main advantage of experience sampling is that evaluating instantaneous experiences causes less memory bias than traditional retroactive measures. Further, it allows the exploration of temporal relationships in subjective experiences. The objective of this paper is to structure, analyze, and characterize the state of the art of available literature in the field of surveillance of work stress via the experience sampling method. We used the formal research methodology of systematic mapping to conduct a breadth-first review. We found 358 papers between 2010 and 2021 that are classified with respect to focus, research type, and contribution type. The resulting research landscape summarizes the opportunities and challenges of utilizing the experience sampling method on stress detection for practitioners and academics.

1. Introduction

The chaos and rush of today make it impossible to live a stress-free life. People’s interactions with their colleagues and the environment in their daily lives create stress in varying degrees. Thus, stress is a factor and internal experience that causes changes in one’s life routine in response to leaving one’s comfort zone. Stress is an inner experience felt as a result of changes in a life routine in response to leaving one’s comfort zone. We can all feel stressed at work [1], with family [2], in traffic [3], and even with friends [4]. For example, at work, we may have to do a lot of work in a limited amount of time. Disagreements arising from differences of opinion between family members or friends can affect people negatively. As if all this were not enough, we may feel even more pressured in the face of external factors, such as the COVID-19 epidemic [5], wars [6], and economic crisis [7]. The human body still has primitive responses to stress as these reactions prepare the body for battle as if there is a real threat. The body adapts to the situation in various ways in order to survive. However, when we feel stressed all the time, this can lead to health problems as the body acts as if there is a constant threat [8]. For example, when we see a lion in real life, we feel a threat and our heart rate increases. However, the heart is a muscle, and just as our muscles swell when we do heavy sports, the heart can contract after a while when it works at a high tempo all the time. This situation is extremely dangerous for human health. Stress has physiological effects on people as well as psychological effects [9]. Effects such as depression [10], anxiety [11], and behavioral disorders [12] can be seen in individuals who are exposed to long-term stress.

According to the World Health Organization [13], stress is predicted to be one of the most important causes of dysfunction by 2020. It can threaten a person emotionally and physically, affecting their work performance and even health status [14]. Additionally, it is important to have appropriate emotional resilience and job satisfaction to increase work performance. According to Colligan and Higgins [15], stress in the workplace is detrimental to the well-being of employees and can lead to increased absenteeism, organizational dysfunction, and reduced productivity. Many studies have shown that attention interventions reliably reduce both general psychological stress [16,17,18] and occupational distress [19]. Furthermore, other evidence suggests that mindfulness is associated with coping success during stressful events [20].

Stress has a significant effect on every aspect of an individual’s life. The experience sampling approach emerges as one of the most effective ways to examine these effects. The experience sampling method (ESM), also known as ecological momentary assessment [21], is a structured daily technique that evaluates the current context and psychological events such as mood in daily life. In the experience sampling studies, participants are usually asked to answer various questions through a smartphone-based application. It is necessary to make instant self-evaluations from the participants who are notified by the mobile platform at various moments during the day. Thus, this method provides the opportunity to capture instant experiences in real-time. This method is used to collect data on different types of problems, such as addictive behavior [22,23,24], pain [25,26,27], evaluation of psychiatric problems [28,29], fatigue [30,31], and more recently, medical evaluations [32,33,34,35,36]. In particular, studies conducted in the working environment have been carried out to examine the mood of employees and investigate the positive and negative events that affect employees in the workplace.

This article presents the results of published studies on the practice of experience sampling that aim to shed light on trends and emerging practices in stress research. This study differs from previously published secondary papers in the literature [15,16] by presenting a thematic analysis instead of narrative summaries that focus on a qualitative review. The contributions of this study are three-fold:

• In this study, research articles on the experience sampling method and stress issues published between 2010 and 2021 were retrieved and evaluated;
• The issue of stress and experience sampling has not been systematically investigated until now, and to the best of our knowledge, this is the first review study using a systematic mapping approach for stress with experience sampling method;
• Selected primary studies have been evaluated from a wide variety of perspectives for identifying potential gaps in current research to identify areas for further investigation.

The remainder of the article is organized as follows. In the following section, the definition of the experience sampling method and its application principles is presented. In Section 3, the aims of the mapping study, research questions, and scope are defined. Following that, Section 4 describes how the systematic mapping methodology has been applied. The results of the mapping and the answers to the research questions are presented in Section 5. Section 6 compiles the results obtained from the research questions. The mapping of demographics, limitations, and future directions are presented in Section 6 and Section 7, respectively. Section 8 discusses the findings, and finally, Section 9 presents our conclusions.

2. Experience Sampling Approach

Experience sampling is one of the methods used to analyze emotional reactions [18] and habits. It aims to create a rich overview of the phenomena by examining the subject participating in the study, and the answers to a couple of questions are sought in the following process. One of the most critical methodological decisions in the study is to decide on question planning. The literature defines three different approaches for experience sampling methodology (ESM): random, time-based, and event-based [37]. As the name suggests, random sampling aims to collect data from the participants at non-periodic intervals. Random sampling is often preferred in scenarios where the indicators of the research topic cannot be determined. However, if the goal and focus of the research topic are specific, choosing other methods increases success. Time-based reporting requires participants to respond at specific times each day. In other words, it requires reports at the same time each day. This design is perfect for routine activities (such as how much food a participant eats) or easy-to-remember things. These exercises usually take about 1–2 weeks, as they do not require much time or effort. However, some researchers are concerned that such studies focus on a specific context. Consider a scenario in which test participants are viewed only at 9 o’clock while at work. Subject A appears to be more relaxed, while subject B displays higher stress levels. Thus, the participants may encounter a problem in a certain period of time, which can be considered the primary drawback of this approach. Event-based experience sampling studies require participants to answer questions about a specific event (e.g., smoking). These are extensively applied for social interactions, anger management, and stress detection. This design is the most reliable option for occasional special events. Since events are unpredictable, there is no need for specific schedule planning. With these methods, a mobile device is used during experience sampling. When the participants answer various objective questions through a program on a mobile device, they simultaneously create a ground truth with their smartphone instruments. Participants report their thoughts, feelings, behaviors, and/or surroundings at that moment or immediately after. The motivation of this process is to collect as much data as possible. More commonly used methods [38] to collect data include survey, interview, observation, document/record review, focus group, checklist, oral history, ethnography, case study, and experiment.

ESM has many benefits, but its implementation varies. The ecological validity of ESM is high because assessments are made in the natural course of real life [39]. While the metrics used in ESM design vary depending on the research question and the researcher’s preference, these metrics have various advantages and pitfalls in practice. One of the important factors is the correct equipment selection because the result is directly related to this choice. Certain decisions must be made between cost, effectiveness, practicality, and time spent. The most important limitation of the pen-and-paper method is the emergence of misleading data due to participants’ past or future tense responses. Technology-based designs, especially smartphone applications, increase data accuracy while providing more participants [40,41,42].

3. Goals, Questions, and Metrics

We used the Goal–Question–Metric (GQM) paradigm [43] to pose meaningful research questions in the evaluation of these systematic mapping goals, which are as follows:

• G1: To classify the articles of experience sampling applications regarding their application domain;
• G2: To understand the various perspectives of experience sampling (e.g., type of data, type of survey, coverage) that are investigated by the researcher;
• G3: To reveal the technologies and tools used for experience sampling;
• G4: To investigate how stress is triggered during the experience sampling and how the collected data is analyzed;
• G5: To analyze demographic and bibliometric data by identifying researchers and their affiliated organizations in this field;
• G6: To discover recent trends and future research directions in this area.

G1, G2, G3, and G4 are defined to reveal the practical use of the experience sampling method and understand the parameters that directly affect the participant–system interaction. These goals guide our initial research questions.

• RQ 1.1: What kind of data collection method was used in ESM?
• RQ 1.2: What type of data has been collected?
• RQ 1.3: How many participants were studied?
• RQ 1.4: How many questions were asked of the participants?
• RQ 1.5: What type of experience sampling method was used?
• RQ 1.6: What type of analysis method has been used?
• RQ 1.7: What type of stress was studied?
• RQ 1.8: What were the methods used to trigger stress?
• RQ 1.9: What was the average time spent on an experience sampling study?

To answer the questions listed, we examine the articles in detail, collect relevant metrics, create classifications that respond to the data and findings reported in the articles, and obtain frequencies. We do not provide a subjective view to answer any of these questions. Therefore, all measurements are objective. G1, G5, and G6 goals are about understanding the demographics and bibliometrics of articles and authors. These goals raise our second set of research questions listed below.

• RQ 2.1: Who are the authors with the most articles on experience sampling topics?
• RQ 2.2: Which countries produced the most articles?
• RQ 2.3: What is the academia/industry ratio of the author affiliations?
• RQ 2.4: Which venues have the highest number of articles?
• RQ 2.5: What is the annual publication trend?
• RQ 2.6: What are the most influential articles in terms of citation count?
• RQ 2.7: What is the number of citations by venue type?

The trends and limitations reported in the articles are extracted and presented to the readers. Question group 3 listed below serves this purpose.

• RQ 3.1: What limitations are reported in the papers?
• RQ 3.2: What lessons learned are reported?
• RQ 3.3: What future research directions are suggested?

The answers to the questions are based on the opinions and research results of the original authors who conducted the primary studies. After setting the study’s objectives, we linked them to the research questions and determined the metrics. The remainder of this study is based on the underlying protocol of this SM, as depicted in Figure 1, which describes the systematic mapping study’s workflow. We described the details in Section 4, Section 5, Section 6, Section 7 and Section 8.

4. Research Method

We chose to perform a systematic mapping study (SMS) to obtain an overview of the experience sampling method in stress detection and evaluation. We followed the guidelines provided in works, such as those of Kitchenham [44], Budgen et al. [45], and Petersen et al. [46].

4.1. Article Selection

The selection of the article actually forms the basis of the synthesis of all its conclusions. In this study, articles were selected using a three-step process, using the guidelines presented in the referenced systematic mapping article: (1) article identification using digital libraries and search engines, (2) exclusion criteria outside the scope of this study, and (3) definition and implementation of inclusion criteria targeting specific resources and locations that may be missed by digital libraries and search engines to manually select relevant articles. These steps are shown in Figure 1.

4.1.1. Step 1: Article Identification

We acquired the literature list through a keyword-based search in electronic databases: IEEE Xplore, ACM Digital Library, Google Scholar, Microsoft Academic Search, Science Direct, and Springer Link. Search terms were selected through an emulated primary research term assessment process. The final results were converted into the following search terms: “Experience sampling*” and “Stress*.” In this step, 587 articles were obtained that made up the initial data pool.

4.1.2. Step 2: Exclusion Criteria

Some eligibility criteria were established, and the following exclusion criteria were defined to eliminate articles from the initial repository.

• C1: Languages other than English;
• C2: Relevance to the topic;
• C3: Did not appear in the published proceedings of a journal, book, conference, symposium, magazine, or workshop.

The relevant criteria were applied sequentially. It was easier to apply criteria C1 and C2 than to apply criteria C3. Under the C3 criterion, each study was first reviewed by one author and then cross-checked by the other author to assess the relevance of the article. At the final stage of exclusion, 170 of the 587 articles were eliminated, and 417 articles remained for use in the study.

4.1.3. Step 3: Final Article Set

Figure 2 presents the distribution of 417 articles analyzed during the study. Finally, a total of 358 articles (Appendix A) were used within the scope of the study. A few articles whose full texts were not available were classified as ”Others”.

4.2. Iterative Development of the Systematic Map

A systematic map is a tool used to classify selected articles. Map development is time-consuming because of the size of the task and the complex process. Our map, which contains 358 article reviews, employs the GQM approach, which contains research questions and metrics used as the primary guide to the SMS. For RQ 1, we need to collect the following attributes: “data collection method,” “data type,” “number of participants,” “number of questions,” “experience sampling method,” “analysis method,” “stress type,” “methods to trigger stress,” and “mean time.” We defined these attributes and presented the map structure that can be identified as comprehensive. Similarly, for RQ 2, we need to collect the following metrics: “locations with the highest number of articles,” “authors with maximum articles,” “author memberships,” “number of articles per year,” “number of articles by venue type,” “number of citations by venue type,” and “number of citations by location.” This guide is used to reveal objective demographic and bibliometric data for the authors and articles.

Conclusively, we need to obtain the following metrics for RQ 3: “limitations,” “lessons learned,” and “future research directions.” This directed us to create our third map, which establishes the basis for clarifying RQ 3.

5. Mapping Research and Evaluation

RQ 1.1: What kind of data collection method was used in ESM?

Within the framework of experience sampling, demographic information and instant experiences can be collected via traditional paper-based methods, web-based methods [47,48], mobile devices-based such as mobile phone, personal digital assistant (PDA), wearable devices such as Siemens 3T scanner, Holter monitor, Empatica e4, Real Extraction DNA kit, Affectiva Q-Sensor, HealthPatch MD, Google Glass, Emotiv Insight EEG Headset, Microsoft Band 2, WatchMinder3, Philips Respironics [49], and/or medical devices such as a magnetic resonance (MR) device [50]. Figure 3 shows the number of articles in these categories. Each article is associated with at least one or more categories, for example, Ref. [51] collecting voice and questionnaire data with a mobile device while simultaneously collecting physiological signals with an MR device.

As shown in Figure 3, the most popular ES method used by 289 of the researchers was the mobile device. The second most widely used method was wearable devices (51), which provided new data sources for research.

RQ 1.2: What Type of Data Has Been Collected?

In Figure 4, all data collection methods used within the scope of 358 articles are divided into the following main categories: survey, physiological data, mobile data, audio, GPS, accelerometer, video, image, and computer-generated data. The data types collected in the researched articles have provided various data such as heart rate, skin temperature, and pulse using wearable devices [52,53]. We describe such data within the context of physiological data and show it in Figure 5. Mobile data and PC data track daily metrics of the type of frequency and clicks in stressful situations. GPS is used to obtain location information of a person in a stressful situation to search for a link between stress and geographic location. Physiological data are divided into the following categories: saliva, electrocardiography (ECG), electroencephalography (EEG), heart rate variability (HRV), temperature, blood volume pulse (BVP), respiration, eye movements, electrodermal activity (EDA), galvanic skin response (GSR), and photoplethysmography (PPG). The x-axes in Figure 4 show the number of articles and y-axis data types for each data type collected.

RQ 1.3: How many participants were studied? Within the scope of the articles examined in Figure 6, data from 82,798 participants were analyzed. The x-axis of the graph shows the number of articles, and the y-axis shows the participant range according to the number of participants in the study. The most preferred participant number range (150 articles) was detected as 0–50 participants. Two common reasons for conducting studies with fewer participants were found: one is the participants’ personal data security [54], and the other is the work in specific areas [55].

RQ 1.4: How many questions were asked of the participants? A total of 6998 questions were identified within the reviewed studies’ scope, as shown in Figure 7. As seen in the graph, the most preferred (38%) question range was determined as 0–20. It is emphasized that keeping the question texts short to not distract the participant’s attention ensures more correct answers and ease of implementation [S059].

RQ 1.5: What experience sampling method was used?

Types of ESM used in the articles are given in Figure 8. Random ESM [56] is used to sample the participant’s experience at unforeseen times. Time-based ESM [57] sends survey notifications at certain hours within the participant’s information. A semi-random, half-time-based ESM [58], random survey notification is made in a certain time frame within the participant’s information. The event-based method [59] is applied before and after certain actions are performed by the participants. In the signal-based method [S152,S288] a signal is sent to the participant to answer the questionnaire at unpredictable times. Each article is associated with at least one or more categories. For example, in [60], experience sampling data were collected from participants for four days based on the signal, four days based on the event, and four days based on time.

RQ 1.6: What analysis method was used?

We categorized the data analysis methods used in primary studies as regression, analysis of variance (ANOVA), multi-level models, random forest, and support vector machine. To rank data analysis methods based on the number of methods used in the published articles, the methods were extracted (Figure 9). The most popular method used for data analysis, as shown in Figure 9, is regression methods (41%). The second most popular method is ANOVA (23%), and the third is the statistical multi-level model (41%), also known as a random parameter model that contains hierarchical linear models, linear mixed-effect models, mixed models, nested data models, random coefficient, and random-effects models (17%).

RQ 1.7: What type of stress was studied?

Stress occurs when the body is affected by a number of negative events and shows physiological and psychological reactions to them. While physiological stress affects human physiology, psychological stress can affect social life, movement/behavior, cognitive, and emotional areas. In Figure 10, the physiological and psychological stress types of the participants were compared. Most participants in the studies (89%) felt psychological stress, and 11% felt physiological stress. The distribution of the psychological stress types on the participants can be displayed. While the most common type of psychological stress felt was behavioral (28%), the second was cognitive (25%), then social (20%) and emotional (27%) stress.

RQ 1.8: What specific focus was established on triggering events of stress?

The ten factors that most frequently cause stress in the studies reviewed are shown in Figure 11. The number of x-axis runs is triggering events that cause y-axis stress. Most of the studies aimed at coping with daily stress (78) and improving the current situation, while other important stress causes can be evaluated as psychotic problems (60) and work stress (46).

RQ 1.9: What was the average time spent on an experience sampling study?

The user experience data collection time is shown in Figure 12 with certain days; the x-axis reflects the number of studies, and the y-axis is the number of user experience collection days. The most preferred data collection period is 8 to 14 days. Some studies may require longer observations than others due to the subject matter. For example, [S227] is a study with an observation of three months or more.

Figure 13 shows the mapping results obtained from research sub-questions Q1.1 (Data Collection Method) and Q1.3 (Number of Participants) in comparison to research sub-questions Q1.4 (Number of Questions) and Q1.6 (Analysis Method). These results may indicate that:

• Most ESM studies are designed to target a maximum of 100 participants in the experiment, and mobile devices are the most preferred data collection method, among others, especially in cases where the number of participants increased, where the use of mobile devices was considered almost the only option;
• The number of questions asked of the participants was mostly limited to 40. Similarly, the mobile device appears to be the method that supports the greatest number of questions;
• Examination of the analysis method according to the number of participants shows that there is no pattern. In any case, statistical analysis is the most widely adopted technique.

Figure 14 shows the mapping results obtained from research sub-question Q1.7 (Research Focus) in comparison to research sub-questions Q1.2 (Data Type) and Q1.5 (ESM Approach). These results may indicate that:

• The most used ESM approach is random sampling, even when the research focus changes. This approach, which dominates the studies in the analysis of physical stress, is used in conjunction with other approaches in the analysis of mental stress;
• While the most used data collection method for physical stress is the acquisition of physiological signals, the other research focus is surveys.

6. Mapping Demographics

In this section, we address RQ 2 and examine the demographics of articles and authors.

RQ 2.1: Who are the authors with the most articles on experience sampling topics?

Inez Myin-Germeys is the most published and the most senior author, with 18 articles in this field. The second- and third-ranked authors were Jim Van Os and Marieke Wichers, with 15 and 10 articles, respectively.

RQ 2.2: Which countries produced the most articles?

The places that produced the most work in the field of stress and experience sampling were examined. As a result of this review, USA (84) ranks first among the places that conduct the most research in this field. The UK (20) is in second place, and Netherlands (15) is third.

RQ 2.3: What is the academia/industry ratio of the author affiliations?

It was investigated how much the academia and industry contributed to the studies in the field of stress and experience sampling. The Academy has made a great contribution to the literature with 173 studies. However, it has been observed that the studies in this field are not yet mature in the sector.

RQ 2.4: Which venues have the highest number of articles?

Figure 15 aims to present the top journals in the field because our main motivation is to raise awareness about which resources researchers should follow. Most preferred is IEEE Acsess (40). Motivation and Emotion (14) ranked second, and Psychological Medicen and PLOS ONE (8) ranked third.

RQ 2.5: What is the annual publication trend?

Table 1. Distributions over publication types.

Type of Publication	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020	2021	Total
Conference		4	2	5	14	10	8	20	9	16	3	2	93	25.97%
Journal	10	8	14	15	16	16	22	25	39	34	23	37	259	72.34%
Book Chapter				1	1		1	1	1			1	6	1.67%
Total	10	12	16	21	31	26	31	46	49	50	26	40	358	100%

shows the distribution over time for the 358 primary studies. With 50 articles published since 2010, it has been determined that the most active year is 2019.

RQ 2.6: What are the most influential articles in terms of citation count?

When the top three articles with the most citations are examined, the first place belongs to [S237] with 355 citations, and the second and third ones are [S140] and [S313], with 328 and 265 citations, respectively.

RQ 2.7: What is the number of citations by venue type?

According to the venue type, we classify the articles into three categories: conference, journal, and book. In Figure 16, the x-axis shows the number of articles in each category, and the y-axis shows the venue type. Figure 16 shows places with a high number of journals (260). When the number of citations and type of place is collected for each article, we observed that journal articles receive the most citations, with 7146 citations.

7. Mapping Limitations and Future Directions

Evidence from the reviewed primary studies indicates that applying EMS can be challenging from different perspectives. The research questions RQ 3 attempted to identify existing limitations and useful ESM approaches that can provide direction for future studies.

RQ 3.1: What limitations are reported in papers?

Limitations of the published studies are broadly categorized based on the participants, size of the dataset, analysis technique, algorithm, device/tool used, scaling, applicability, and causality perspectives.

• Participants: When the test is limited to a small group of participants, includes participants of similar socioeconomic status, only healthy individuals, only women or only men, or certain groups of participants, such as certain workgroups, generalization cannot be made [S002]. The need for a dataset to include all possible representative characteristics in a balanced structure may not be met in real life.
• Size of the Dataset: If the sample size is too small or large, it reduces the power of the work; for example, a large data size causes difficulties in processing data [S050]. On the other hand, lack of data negatively affects the accuracy of analysis models [S063]. Note that the use of simple classifiers such as Naive Bayes is recommended when working with a small dataset [S066]. As an alternative method, a large dataset belonging to the same or a close domain can be used as a reference with the transfer learning approach. Data augmentation, on the other hand, aims to synthetically reproduce existing data as the last proposed technique.
• Analysis technique: Possibly misleading situations may be encountered during the analysis, e.g., noisy data, feature extraction error, misleading data from participants, which will cause the analysis to fail [61].
• Algorithm: Known limitations of the algorithms presented, for example, Akaike information criterion (AIC) gives information about the quality of the model in an absolute sense. It will not give any warning if all candidate models are bad [S034].
• Device: Device and tool limitations, such as data accuracy and adequacy problems, can occur due to device difficulties such as battery power, consumption of a computational resource, and difficulty of the calibration [S177].
• Tool: Limitations of developed software, for example, collecting missing data due to software deficiencies, will result in incomplete deductions [S239].
• Scaling: The limitations of determining the methods to be used, for example, insufficient data collection time, cause the data size not to reach the optimum level [S241].
• Applicability: Limitations on usability under different environments, for example, the fact that its use appeals to an overly specific audience, cause it not to be preferred by the rest of the users [62].
• Causality: Objectivity, the limitation of unprovable, for example, subjective or misleading responses from participants, makes the study ungeneralizable [S072].

In this SM, the limitations are objectively stated as specified by each study author. Out of 358 articles, 103 articles reported one or more limitations of the research study. The obtained limitations are shown in Figure 17. The figure depicts the limitations of the research. The x-axis displays the number of articles in each category, and the y-axis recites categories. The most common limitation is the participant category.

RQ 3.2: What lessons learned are reported?

Most of the authors reported lessons learned from their studies. The lessons learned were reported in 81.84% (293/358) of the articles. Lessons learned vary greatly depending on the individual research and study context. That is why we conducted a qualitative analysis rather than a quantitative analysis. Readers should note that it must be interpreted in the context of the studies.

Apart from participant questionnaires, supportive data can be obtained with various devices such as Likert-scale, open-ended, or visual questions. More detailed studies were conducted with MR, Holter, or mobile device data to contribute to the literature [62,63]. Among these supporting data, photographs can be annotated with more than one tag, depending on the context. Following the participants longer and asking them to rate their mood on the photos provided more evidence for the temporal effect of happiness from different sources [S054].

Stress problems cause a decrease in employees’ performance at work [64], a decrease in the academic achievement of students at school [65], depression in healthy individuals in daily life [66], and diseases in individuals with genetic diseases [67]. It has been determined that individuals who have experienced childhood trauma [68] or war trauma [69] were more sensitive to stress. Due to the increasing stress in society, web [70] and mobile [71] health applications have been developed, and awareness studies have made it possible to cope with stress better [72].

RQ 3.3: What future research directions are being suggested? Most of the articles provided guiding advice for ongoing research. These can be broadly divided into the following categories:

• Participant: Participant-based improvements—Designing a study where participants can give direction to the study increases overall performance [S189].
• Dataset: Develop methods for collecting participant data, such as collecting large datasets [S082]. Big dataset analysis with a balanced structure always results in more meaningful inferences.
• Analysis: We observed that statistical methods are the most preferred in the analysis of the collected data. In the last quarter of the decade researched, there is a tendency to use more machine learning and deep learning approaches. Especially with the adoption of cloud computing and GPU-based processing technologies, analysis processes can be accelerated [S050].
• Algorithm: Build new models with different algorithmic approaches, such as semi-supervised deep learning approaches [S058] using ensemble models.
• Model: The goal is to improve the protocols used in the studies, such as a universal background model [S035].
• Various Context Indexes: Using data from multiple devices depending on the context, such as wearable device data [S147]. Models fed with data collected from different dimensions and perspectives have higher performance.
• Scaling: Scales used across the study, such as data collection time [S082]. Mostly, the time scale is used by default in studies. The contribution of analyses to be made with different scales should be investigated.
• Applicability: Developing useful and accurate tools, such as an application developed for individuals with severe cognitive impairment [S010]. Most studies use commodity systems to collect data and off-the-shelf business intelligence (BI) tools for analysis. Some situations do not accept these standard approaches. Software engineering approaches should be leveraged for problem-tailored tooling.

Future research directions outlined in the articles were identified. Figure 18 exhibits these data. Although these data contain directions for future research, they help us understand the thoughts of the researchers and what they perceived as missing parts at the time of their studies and publication.

In Figure 18, the x-axis shows the number of articles in each category, and the y-axis displays categories. Various context indexes, multi-device ESM work (23 articles) is perceived as the area that requires the most study. Model (21 articles) is perceived as the second area that requires the most study.

8. Discussion

This section summarizes the main findings of this systematic mapping study detailed in previous chapters. In addition, it highlights limitations that may represent threats to its validity and discusses implications for research and practice.

8.1. Principal Findings

The results provide an objective summary of trends in the stress-based experience sampling method. The collected data shows that context-dependent indexes, models, and analysis methods attract significant attention in the research community (Figure 18). The collection of new data types provided by the developing sensor technology, modeling with new method combinations, and obtaining meaningful information from these data due to various analyses was carried out by [S133] in the laboratory environment. However, real-life usability tests (Figure 18) are still ongoing. Psychological behavioral stress is the most researched topic for coping with depression, anxiety, panic disorders, and anger issues, which are some of the biggest problems of our time. Studies under this category aimed to identify the self-destructive or unhealthy behaviors of participants. These studies were considered to have a high potential for productization.

8.2. Limitations of the Systematic Mapping Study

The conducted systematic mapping study suffers from several limitations. The principal limitations are identified as selection bias, inaccuracy in data extraction, and misclassification. Selection bias refers to the distortion of a statistical analysis caused by the criteria used to select publications. To avoid this limitation, we aimed to determine the ideal query string for the selection of papers and include multiple repository searches, as explained in Section 4.

Inaccuracy and misclassification in data extraction refers to the possibility that information from a primary study and the information extracted will be interpreted differently by reviewers. To alleviate this threat, both authors independently reviewed all articles, and any discrepancies that emerged were resolved by consensus. We aimed to address different aspects of this research field by using several research questions; however, covering all aspects and the contributing factors is nearly impossible. Different researchers can consider addressing other aspects that we did not include in this research.

8.3. Implications for Research and Practice

The findings of our systematic mapping study have implications for researchers planning new studies of ESM on stress and for practitioners designing new products for the ecosystem. The most critical challenge is the absence of a product designed for use in stress analysis. Another important finding of our secondary study is that the papers in which statistical analysis and machine learning models were combined reached more comprehensive results with high accuracy. Therefore, we recommend that researchers apply accurate statistical data analysis, methods, and techniques that follow state-of-the-art machine learning approaches, especially deep learning.

Our findings show that the majority of the papers reported that participants are less motivated to engage in long-term experiments. In particular, the ergonomics of the mobile device/sensor system used allow more questions to be asked or more data to be collected. We, therefore, consider that there is an important shortage of dedicated equipment for such research projects. From a management perspective, designing a non-invasive wearable system that can be used without affecting daily life will be an effective solution to this limitation.

9. Conclusions

The aim of this study was to understand the stress-based experience sampling applications through a systematic mapping study of 358 research articles from well-known repositories. We aimed to explore the current major trends in the research area and how different ESM approaches are studied. As a result of the investigations on ESM and stress, we observed that although the cooperation between academia and industry is increasing, the industry has not had much one-to-one effect on research projects. With the widespread use of sensor technology, data collection has become easier, and its application has become practical. Secondly, we found that physical stress was less studied, with the majority of studies focusing on mental stress, especially cognitive and behavioral aspects. We also observed that random and hybrid sampling from ESM methods are preferred, and deep learning approaches have come to the fore in the establishment of the analysis model in recent years. The main reason the observation periods of the studies were mostly up to two weeks was understood as the limitations of the equipment used and the participants’ unwillingness to participate in long-term experiments. Finally, our aim for bibliometric analysis is to direct the researchers in this field to different studies that provide useful information. For instance, popular venues for publication, country-wise analysis, most cited research papers, and most active researchers are reported in this study. Despite the great emphasis placed on stress in the literature, none of the commercially available tools are geared toward stress-based remediation. The software of such systems needs to be tuned for a suitable use for stress analysis. Our survey revealed that the researcher who has performed the most work in this field has 18 publications and has contributed to the most cited work. These researchers generally conduct their research in the USA, and the UK has also been observed. This means that researchers who want to take part in studies in this field can take part in the study groups in the USA and UK, where the most studies in this field are performed. It is seen that the most preferred journal is the IEEE Access journal, which has been chosen 40 times by far.