Cognitive Modeling of Task Switching in Discretionary Multitasking Based on the ACT-R Cognitive Architecture

Abstract

Discretionary multitasking has emerged as a prevalent and important domain in research on human–computer interaction. Studies on modeling based on cognitive architectures such as ACT-R to gain insight into and predict human behavior in multitasking are critically important. However, studies on ACT-R modeling have mainly focused on concurrent and sequential multitasking, including scheduled task switching. Therefore, in this study, an ACT-R cognitive model of task switching in discretionary multitasking was developed to provide an integrated account of when and how humans decide on switching tasks. Our model contains a symbolic structure and subsymbolic equations that represent the cognitive process of task switching as self-interruption by the imposed demands and a decision to switch. To validate our model, it was applied to an illustrative dual task, including a memory game and a subitizing task, and the results were compared with human data. The results demonstrate that our model can provide a relatively accurate representation, in terms of task-switching percent just after the subtask, the number of task-switching during the subtask, and performance time depending on the task difficulty level; it exhibits enhanced performance in predicting human behavior in multitasking and demonstrates how ACT-R facilitates accounts of voluntary task switching.

1. Introduction

Many observational studies have demonstrated that multitasking is prevalent and important in everyday life, and especially in modern office environments where workers are frequently are immersed in information overloaded environments [1,2,3]. In particular, with the development of autonomous driving systems, task switching between awareness of driving situations and non-driving related tasks is a representative example of a discretionary multitasking environment [4]. As such, in the field of human–computer interaction, studies on human performance in multitasking environments to gain insight into and predict cognition and human behavior in multitasking environments are critically important.

In this way, over the past several decades, considerable progress has been achieved in the cognitive modeling of human behavior in diverse multitasking environments through various cognitive architectures—for example, executive-process interactive control (EPIC) [5], queuing network-model human processor (QN-MHP) [6], and adaptive control of thought-rational (ACT-R) [7,8]. Specifically, ACT-R is a cognitive architecture that provides a computational framework for the integrated modeling of various cognitive processes in the domains of various complex systems, such as menu selection [9], driving [10,11], and virtual games [12]. Thus, ACT-R has been used in several studies on human performance modeling in multitasking environments (e.g., [13,14,15]).

Therefore, the objective of this study is to develop a more general and accurate cognitive model based on ACT-R in order to provide a prediction of when and how humans perform task switching and gain additional insights into task switching in discretionary multitasking. For this purpose, we first identify the limitations of the current ACT-R in performance modeling in discretionary multitasking environments, based on related studies. Accounts of the cognitive processes involved in task switching by voluntary decisions in relevant literature are reviewed, and an ACT-R cognitive model based on such accounts is proposed to provide an integrated account of task switching by self-interruption in discretionary multitasking.

Related Work

ACT-R has been extended based on the threaded cognition theory [13], which can provide an account of concurrent human behavior in multitasking, in which a relatively brief period is spent on task switching [14]. Additionally, Borst, Taatgen, and van Rijn [15] proposed a computational method based on ACT-R for modeling human performance in multitasking, in which an ongoing task is interrupted and then resumed. In this manner, for the past decade, significant progress has been achieved in using ACT-R to model human performance based on the abovementioned studies, which focused on the interference produced by the cognitive bottleneck that occurs when multiple tasks are performed simultaneously and the effect of switching between tasks because of external interruptions, such that ongoing tasks are interrupted by other tasks.

However, studies on ACT-R modeling have focused on the manner in which humans decide to interrupt their task by internally motivating discretionary multitasking. Discretionary multitasking is one of the current general work environments in which workers have some discretion over what and when to perform among multiple tasks that need to be performed, in which task switching can occur because of external interruptions as well as internal interruptions, as evidenced by previous research [16]. Some observational studies have demonstrated that there are considerable instances of task switching due to voluntary decisions by internal interruptions, known as self-interruptions, which interrupt the execution of an ongoing task to focus on the execution of another task in discretionary multitasking [16]. For instance, Czerwinski, Horvitz, and Wilhite [17] reported that 40% of task switching was due to internal decisions. Dabbish, Mark, and González [18] observed that workers in open work environments would interrupt themselves more often. Moreover, discretionary multitasking situations present important practical concerns [19] in that discretionary multitasking does not necessarily lead to optimal switching between tasks and workers can make irrational decisions [20]. In particular, Katidioti, Borst, and Taatgen [21] conducted experiments to study participants switching to another simple task while playing a memory game, the goal of which was to find all pairs of matching cards, in order to compare self-interruption and external interruption. The results of their study indicate that the participants were significantly slower in the decision-making process of task switching on the self-interruption blocks than on the external interruption blocks; consequently, the voluntary decision to switch can be costly.

2. ACT-R Modeling for Task Switching

2.1. Theoretical Background

The cognitive processes involved in task switching can be characterized based on several studies focusing on human performance in discretionary multitasking conducted in the previous decade. We observed that accounts of task switching can distinguish several properties, as follows:

First, several studies have included accounts of task-switching based on cognitive resources. For instance, Salvucci and Taatgen’s [13] threaded cognition theory posits that task switching depends on the availability of cognitive resources. According to this theory, if a cognitive resource is no longer required in an ongoing task, the cognitive resource is released politely, which causes natural task switching. An account of task switching based on the availability of cognitive resources was also demonstrated in Katidioti and Taatgen’s [20] experimental study, in which participants performed a primary task and a secondary interrupting task, including responses to simple questions, and could freely switch from the primary task to the secondary task. The results of Katidioti and Taatgen’s study demonstrated that most subjects switch voluntarily to the secondary task when they are faced with a brief delay in the primary task, even though task switching degrades their performance, which indicates that the gap in the usage of cognitive resources results in task switching. Additionally, experimental studies [22,23,24] have also demonstrated that instances in which cognitive resources are released by subtask completion serve as natural points of task switching. In other words, most tasks consist of repetitions of subtasks, and points between subtasks represent natural points of task switching. In summary, the availability of cognitive resources by subtask completion is an important property that naturally leads to task switching in discretionary multitasking.

Second, according to Adler and Benbunan-Fich’s study [16], imbalances between the demands of an ongoing task and the level of ability of the person performing the task can result in the ongoing task being paused. In other words, when the ongoing task is too difficult compared to the person’s ability, negative emotions such as frustration and exhaustion are experienced. Additionally, when the ongoing task is too easy compared with the person’s ability, positive emotions such as stimulation are experienced. The emotions caused by an imbalance likely serve to trigger self-interruptions. In particular, a difficult ongoing task consumes cognitive resources more rapidly than an easy task, and unless a specific reward is associated with the ongoing task, people are more likely to look for an easier task to conserve their scarce cognitive resources. Baumeister and his colleagues [25,26] also provided similar accounts. They assumed that for more difficult tasks, the depletion of resources makes people give up easily on an ongoing task because of the reduced motivation to complete the task. Consequently, the emotions caused by the imbalance between the demands of an ongoing task and the performer’s ability to execute the task can trigger self-interruptions and increase the likelihood of task switching. In particular, difficult tasks are likely to cause self-interruptions in a typical work environment, in which rewards are not provided directly.

Third, several studies have reported accounts of task switching in which time spent on an ongoing task influences the likelihood of task switching [27,28]. These studies suggest that there is a relatively strong argument for an increasing number of switches away from an ongoing task as the task execution time increases. According to Kurzban et al. [27], the greater the difficulty of the task, the greater is the time spent on the task and the greater is the likelihood of switching when a break is needed because of the accumulated demands. This argument is similar to the abovementioned difficulty-based accounts. Accounts of the positive effects of the time spent on a task on task switching can be interpreted as implying that task switching can be rational when any subgoal in the task is not complete, despite the accumulated demands. Alternatively, other accounts have also been suggested in various studies [19,28,29], which stated that the task switching likelihood increases according to the declining rate of return as time is spent on the ongoing task. However, some accounts have demonstrated the opposite effect of time spent on an ongoing task. For instance, with vulnerabilities in memory, according to the memory-for-goal theory, task switching can be intentionally inhibited to avoid abandoning the accumulated memory of an ongoing task [28]. Therefore, the inhibition of switching can be increased when more time is spent on the ongoing task, because a longer time spent on the ongoing task will naturally accumulate more ongoing task-relevant memory. Combining the accounts of task switching in terms of the time spent on an ongoing task, it could be interpreted that the decisions to switch from the ongoing task can be made based on the demands of the task and the memory accumulated when performing it.

Finally, similar to the abovementioned accounts of task switching decisions based on attributes such as accumulated demands and memory, several studies have represented task switching in terms of a process of deciding which task to perform in discretionary multitasking. For instance, Wickens and his colleagues developed strategic task overload management (STOM) [30,31,32], which addresses multitasking behaviors in overloaded multitasking environments. The model focuses exclusively on the decision process of which task to perform or which task to continue performing, and it is based on multiple task attributes, including difficulty, interest, priority, salience, and time on the task [30]. In particular, in terms of difficulty, the STOM model assumes that easier alternative tasks can be more attractive than difficult ones; the validation study of the STOM model demonstrated that the greater the difficulty of an ongoing task, the more “sticky” is the inhibition to switch away [31]. Moreover, Gutzwiller et al. [28] presented the conclusion concerning the task attribute of difficulty as task end-expectancy. The greater the time spent on an ongoing task, the greater is the expectation of completing a subgoal of the task execution process, which could lead to a decision to intentionally increase the inhibition of task switching. In this manner, it can be determined that task switching is executed through the decision process in accordance with the attributes of the current tasks, such as the difficulty and time spent on the task in relation to task end-expectancy. Moreover, Katidioti et al. [21] examined changes in human pupil size during forced and voluntary task switching. The results showed an increase in pupil dilation for the voluntary condition, which could be attributed to the decision to switch.

In summary of the abovementioned accounts of task switching, in discretionary multitasking, imbalanced states can cause self-interruptions, which can indicate the point at which a decision is made regarding what task to perform. In addition, the point at which a subtask is completed can also be the time to decide whether to find another task to perform. The decision regarding what task to perform is made by comparing the task attributes of the ongoing task, where task switching is executed if it is rationally determined that a task other than the ongoing task needs to be performed. Accordingly, we herein propose an ACT-R modeling method to represent the process of task switching by self-interruptions based on the abovementioned accounts. Prior to presenting the proposed model, we first discuss the ACT-R cognitive architecture.

2.2. ACT-R Cognitive Architecture

The ACT-R (version 7.0) architecture consists of several core modules, and each module represents a part of the brain that independently processes different types of information (see Figure 1) [7]. Perceptual-motor modules, including the vision, motor, audio, and speech modules, interact with the real environment. The goal module stores the goal information required to complete a task. The declarative module stores and processes human declarative knowledge, which is represented in terms of structures called chunks. Declarative knowledge represents facts, such as 2 + 3 = 5. The imaginal module temporarily stores subgoal information or new information encoded by perceptual-motor modules. The procedural module, which is also referred to as a production system, stores and processes all procedural knowledge, which is represented as the rules of human cognitive behavior, called procedural rules, in an IF–THEN form. Each procedural rule includes a set of conditions and actions.

Each module behaves in a parallel manner and has its own buffer that can hold only one chunk associated with the module; therefore, ACT-R can represent a combination of both parallel and serial cognitive processing [11]. However, the procedural module does not have its own buffer. Instead, the procedural module performs a pattern-recognition function that continuously monitors all ACT-R buffers to identify procedural rules whose conditions match the state and content of the buffers. Following this, a conflict-resolution function is performed to select only one of the identified rules based on the highest expected utility value, and then, the prescribed actions of the selected procedural rule are executed [7]. The three functions in the procedural module represent neural functions in the basal ganglia structure, which is part of the brain. The basal ganglia evaluate the criterion satisfaction of the choice between multiple actions by interacting with the functional regions of the brain and then serve the execution of the prescribed action [33]. Thus, the basal ganglia implement the procedural rules in the procedural module by interacting with the buffers in the ACT-R modules and deciding upon the next action. In this manner, ACT-R can represent cognitive processes as symbolic structures.

According to the explanation presented on the ACT-R homepage, ACT-R is based on symbolic and subsymbolic structures. The symbolic structure relates to the symbolic representation of cognitive processes in terms of procedural rules of processing knowledge, called chunks, between functional systems in the brain. The subsymbolic structure includes mathematical equations concerning functional systems in each module, including an abstract representation of which chunks and rules are selected and how quickly the chunks and rules are processed. Therefore, a cognitive model of a specific task can be developed by establishing the chunks and the procedural rules associated with the task, in accordance with the symbolic structure of ACT-R [34], and by supplementing additional subsymbolic equations needed to represent human cognition in the task to ACT-R. The ACT-R model can then provide a prediction of a performer’s detailed cognitive process in the task in chronological order. In the next section, we describe our task-switching model in terms of symbolic and subsymbolic structures based on ACT-R.

2.3. Symbolic Structure of Proposed Task Switching Model

According to the abovementioned accounts of task switching, a symbolic representation of the cognitive process of task switching by self-interruption was framed based on ACT-R, consisting of three stages, as shown in Figure 2.

The first stage is based on the property of task switching related to the account that self-interruptions can be triggered by emotions experienced when there is an imbalance between the difficulty of an ongoing task and the performer’s ability [16]. Additionally, several neuroscience studies [35,36] have shown that the experienced emotions have affective values that influence the cognitive process in decision making under situations where the orbitofrontal cortex plays a key role in neural functions between emotion and decision making and has a neural interconnection with the basal ganglia, represented by the ACT-R procedural module (see Figure 3). In particular, the procedural module includes the role of the striatum in the basal ganglia, which performs the pattern-recognition of information required to determine the next action [7], and the striatum to which the cortical regions corresponding to ACT-R buffers project is known to have a direct interconnection with the orbitofrontal cortex. Together, the emotions experienced during the imbalanced state can be interpreted as directly influencing the cognitive process that determines the next action in the basal ganglia via the neural functions in the connection between the orbitofrontal cortex and the basal ganglia. In other words, the orbitofrontal cortex can affect the selection of the next action associated with self-interruption by projecting the affective value of emotion experienced by the imbalance in the basal ganglia. Therefore, in this study, based on the procedural module representing the basal ganglia, which has a direct interconnection with the orbitofrontal cortex in cognitive decision making, a procedural rule was defined; the rule is selected when there is an affective value derived from the imbalanced state and includes the action related to self-interruption. This rule is depicted in the interrupt-task stage in Figure 2, and a verbal description of the rule in pseudo-English form is given as Algorithm 1:

Algorithm 1 INTERRUPT-TASK rule

if goal is to perform the ongoing task,         and there is an affective value from an imbalance state then         clear the imaginal buffer end if

The interrupt-task rule is implemented when there is an affective value for which the imaginal buffer is cleared to indicate that the ongoing task is self-interrupted. The imaginal module is used to store intermediate information that is necessary for performing a task [37]; thus, clearing the information necessary for the ongoing task interrupts the task. The affective value projected to the basal ganglia from the orbitofrontal cortex is represented by several subsymbolic equations that quantitatively evaluate the imbalance between the difficulty of an ongoing task and the performer’s ability. Additionally, in this model, the subsymbolic equations are implemented in the procedural module, which has a direct connection with the orbitofrontal cortex, because the existing ACT-R has no module that directly represents the orbitofrontal cortex associated with emotion.

In the second stage, it is decided whether another task is to be performed or the ongoing task is to be continued, where the imaginal buffer is cleared to represent the self-interruption in the previous interrupt-task stage. A procedural rule is thus required to change the current goal to determine what task to perform next, and the rule is defined as Algorithm 2:

Algorithm 2 DECISION-TO-SWITCHING rule

if imaginal is empty then         change goal to “decision-to-switch” end if

The absence of any chunks in the buffer of the imaginal module implies that no information is currently required to perform the ongoing tasks, which indicates that no cognitive resources are currently required. Therefore, the decision-to-switching rule can be implemented based on the empty state of the imaginal buffer, not only by the interrupt-task rule but also when cognitive resources are available because the subtask of the ongoing task has just been completed. This is based on the abovementioned accounts indicating that the availability of cognitive resources can induce task switching.

Finally, if the current goal is changed to the “decision to switch” in the previous stage, the next stage involves continuing the ongoing task or requesting a goal of an alternative task to perform task switching, as shown in Figure 2. An ACT-R single model may include procedural rules to allow several strategies for the same task and use the utility mechanism to represent the selection of different strategies over the course of problem solving [38]. Therefore, the model in this study also uses the utility mechanism to select the next task to be performed, and two procedural rules are defined; a verbal description of the rules is given as Algorithms 3 and 4:

Algorithm 3 DO-TASK-SWITCHING rule

ifgoal is “decision-to-switch” then         request a goal of the althernative task end if

Algorithm 4 DECISION-TO-SWITCHING rule

ifgoal is “decision-to-switch” then         request the imaginal buffer to execute the ongoing task end if

Of the two rules above, the one with the highest utility value is selected; the utility value can be determined based on several task attributes according to the aforementioned STOM model.

2.4. Subsymbolic Structure of Proposed Task Switching Model

Several subsymbolic equations must be added to develop a computational model based on the symbolic structure proposed in the previous section. First, considering the interrupt-task rule in the symbolic structure, a subsymbolic equation is needed to represent the timing of starting the rule. In a situation where the interrupt-task rule begins when there exists an imbalance between the difficulty of the ongoing task and the performer’s ability, the subsymbolic equation must be able to evaluate the level of imbalance quantitatively in real time. According to Wickens et al. [31], difficulty is associated with the mental workload imposed by a task. The level of ability indicates the speed of cognitive processes [39]. According to these perspectives, the level of imbalance at time T (LI_T) is proportional to the ratio of the value of the imposed mental workload to the speed of the cognitive processes in the ongoing task at time T.

Jo et al. [40] developed a methodology to quantitatively predict the mental workload over time using ACT-R, in which the accumulated mental workload over a time interval [T1, T2] (AW_[T1,T2]) is represented as the sum of the activated times of ACT-R modules during the time period, as indicated below:

$$A{W}_{\left[T1, T2\right]}={\displaystyle \sum }_i\left(W_i{{\displaystyle \int }}_{T1}^{T2}{A}_i\left(t\right)dt\right)$$

(1)

where W_i is the weight assigned to module i (W_i ≥ 1), and A_i(t) is a function that determines if module i is activated at the given time t (0 or 1 according to inactive or active, respectively), and A_i(t)dt represents the activated time of module i during the infinitesimal time dt, such that each module in ACT-R is activated to process relevant information associated with the execution of a task [40]. Additionally, because affective experiences such as frustration, which can be caused by mental workload, are affected by the temporal dimension that includes recovery time [41], we can assume that the imposed mental workload decays over time. Therefore, the general decay function is applied to Equation (1) for the accumulated mental workload, as follows:

$$M{W}_T={\displaystyle \sum }_j{\left(AW\right)}_j·t_j^{-d}$$

(2)

where MW_T is the imposed mental workload at time T, (AW)_j is the accumulated mental workload imposed by the jth subtask in the ongoing task, t_j is the time since the jth subtask has been executed, and d is the decay parameter and can be decimal number between 0 and 1. Furthermore, it can be assumed that the speed of cognitive processes is related to the number of subtasks in the ongoing task that have been processed, which can be represented as the number of accomplished subgoals in the ongoing task. Consequently, LI_T can be denoted as the ratio of MW_T to the number of accomplished subgoals at time T, N(Subgoals_T), as expressed below:

$$L{I}_T=M{W}_T / N\left(Subgoal{s}_T\right)$$

(3)

In Equation (3) above, LI_T can be interpreted as the amount of mental workload imposed per subtask. Finally, the scaled value of the level of imbalance, as in Equation (4), can be obtained by dividing LI_T by the standard value (MW_std), which is calculated using the accumulated workload imposed when performing a basic subtask.

$$Scaled L{I}_T=L{I}_T/M{W}_{std}$$

(4)

In Equation (4), if the scaled LI_T value is greater than unity, it means that the subtask being performed at time T is more difficult than the standard subtask in the ongoing task. Therefore, in the proposed model, the interrupt-task rule is implemented when the scaled LI_T value is greater than the sum of unity and the imbalance threshold value, as expressed below:

$$Scaled L{I}_T>1+\epsilon \left(imbalance threshold value\right)$$

(5)

where the imbalance threshold value, $\epsilon$, can be a decimal number between 0 and 1.

Next, the utility values concerning the do-task-switching rule and the do-not-task-switching rule must be determined to select the implementation of the next rule after the decision-to-switching rule. To determine the utility values, although several task attributes could be used, attributes related to difficulty and task end-expectancy were used in this study. In other words, if the degree of difficulty of an ongoing task is greater than the degree of task end-expectancy, task switching will be performed, and if not, the ongoing task will continue to be performed. Therefore, the subsymbolic equations associated with difficulty and task end-expectancy were used to indicate the utility values of the do-task-switching and do-not-task-switching rules, respectively. For the utility value concerning difficulty, the accumulated mental workload value, MW_T, was used to denote the demand imposed by the ongoing task until the current time T. Additionally, by dividing the utility value by the above-used standard value, MW_std, the scaled utility value associated with difficulty can be obtained. The utility value concerning task end-expectancy is represented as the number of chunks associated with the subgoal currently being performed in the ongoing task in the retrieval module. It is expected that the greater the amount of information associated with the ongoing task, the sooner the task will be completed, which is consistent with the abovementioned account indicating that task switching can be inhibited to avoid abandoning the accumulated information concerning the ongoing task [28].

2.5. Discussion

As mentioned earlier, considerable efforts have been devoted toward accounting for the cognitive process of task switching in discretionary multitasking. The proposed model contributes toward such efforts using an ACT-R-based approach for integrating different accounts of task switching. ACT-R includes several significant features that facilitate the establishment of an integrated account of voluntary task switching.

One of the significant aspects of ACT-R in the proposed model is the production system. Task switching can be affected by several task attributes; even the same task attribute can have different effects on task switching depending on the situation. For example, the task attribute of difficulty is considered to have positive effects on task switching under self-interruption [16], but during the decision process of task switching, the difficulty of the ongoing task is considered to have negative effects on task switching [30]. The production system in ACT-R facilitates an integrated account of the conflicting circumstances of task switching by providing a rule-based representation of cognitive control to adapt to different cognitive states. Additionally, the utility system in the production system can provide a rational criterion for selecting one of the procedural rules and facilitates the representation of a rational decision process in humans. Through the utility system, the proposed model can make rational decisions by considering several task attributes without unconditionally switching to alternative tasks.

Another significant aspect of the ACT-R in the proposed model is its symbolic and subsymbolic structures. The symbolic structure facilitates the representation of an ACT-R model for each part of the brain as a module and describes the process of performing a task through a series of processes in which the modules are activated. The subsymbolic structure provides predictions of the time related to the duration for which the modules are active. Considering these structures together, ACT-R can predict the extent to which the brain’s parts are activated, which implies that ACT-R facilitates the quantitative prediction and explanation of the mental workload exposed by the execution of a task. The quantitative prediction of mental workload plays an important role in determining the point of task switching in the proposed model.

In this manner, ACT-R is beneficial for developing an integrated model of task switching in discretionary multitasking. We developed a task-switching model by using the features of ACT-R to represent an integrated account of task switching. However, the question of whether the proposed model can quantitatively represent the cognitive processes of task switching in real humans when dual tasks are performed simultaneously still remains unaddressed. Therefore, an experimental study was conducted to validate the proposed model.

3. Model Validation

3.1. Design

The model in this study represents the cognitive process of task switching as self-interruption according to the imposed mental demands, followed by a decision as to whether to switch tasks based on the task attributes. To validate the model, an illustrative model of a dual task was developed based on the modeling method, and the simulated results of the model were compared with the results of an empirical test. The dual task used in this study includes a primary task that has several mental demand levels and a secondary task that is easy in comparison to the primary task. The primary task involves a memory game, usually referred to as a concentration or pair game. In the memory game, a deck of cards that includes pairs of matching images is used. The goal of this game is to match all the pairs. All cards are laid face down at the start of the game. The players then flip two cards sequentially. If the flipped cards match, they are kept facing upwards; if not, they are flipped to face downwards.

The memory game has been altered in various ways in certain studies to meet their respective goals [21,42]. Therefore, this game was slightly altered for the experiment in this study, such that there were 20 cards consisting of ten pairs arranged in a 5 × 4 matrix, as shown in Figure 4a. The cards depicted four types of images, depending on the level of difficulty, as shown in

Table 1. Four types of images on the cards in the primary task.

Difficulty	Image Form	Alternative Image Form	Instance
Level 1	Diamond at playing card	Diamond at playing card	(see Figure 4a)
Level 2	Three-letter word	Reverse order of the word	DOG, GOD
Level 3	Simple calculation	Simple calculation with the same answer	2 × 3, 4 + 2
Level 4	Simple equation	Simple equation with the same answer	X + 2 = 3, 2X + 3 = 5

.

The secondary task in the experiment involved a visual attention task called a subitizing task [43]. In this type of task, several visual objects are presented to the subject, and the subject must identify the number of objects present [44]. In the experiment, the subject must identify the number of target cards with only one diamond on the screen and then click on the card corresponding to the number of identified target cards (see Figure 4b). The number of target cards varied between five and nine. Because this secondary task is less demanding than the easiest version of the primary task, it can result in a switch to the secondary task when the primary task is too difficult. Additionally, at least five visual objects must be identified in this task, which prevents the rehearsal of the primary task when performing the secondary task. In other words, the subitizing task was chosen for the experiment.

The primary task, i.e., the memory game, includes a subgoal of matching a pair of cards by repeating a small unit of a task to select and click on a card, and the primary task is to perform the repetition of the subtask associated with the subgoal ten times. According to the characteristics of the memory game, three results concerning task switching can be expected based on the abovementioned accounts of task switching. First, most task switching will be performed just after achieving a subgoal, that is, when cognitive resources are free because of the completion of the subtask of matching a pair of cards, in accordance with accounts based on the availability of cognitive resources. Second, according to difficulty-based accounts, the greater the difficulty of the memory game, the greater the task switching that occurs during the execution of a subtask. If the level of difficulty of the memory game is high, each time a card is identified, it will consume considerable cognitive resources and increase the likelihood of switching to another task. Third, because task switching occurs through decision processes, if task switching occurs during the execution of a subtask, the subtask will involve a longer performance time than the other subtasks, including the time for the decision process. In addition, a small unit of a task involving the selection and clicking of a card in the subtask in which switching occurred will involve a longer performance time than the other subtask. This was also demonstrated in an experimental study by Katidioti et al. [21], in which the mental workload increased before voluntary task switching occurred, implying that a cognitive process is performed prior to task switching.

Therefore, the proposed modeling method was validated by confirming whether the results of both the subjects and the model of the dual task, including the memory game and subitizing task, demonstrate the abovementioned phenomenon. The hypotheses are as follows:

• Both the model and subjects are more likely to perform task switching just after the completion of the ongoing subtask.
• The greater the difficulty in the ongoing task, the greater the task switching significantly increases when a subtask is executed.
• A small unit of a task involving the selection and clicking of a card in the subtask where task switching occurs involves a longer performance time than the small unit task in the other subtask.

3.2. Subjects and Apparatus

Twelve subjects were recruited for this study, including nine males and three females with a mean age of 24.9 years (σ = 2.36). All subjects had adequate experience in performing multiple tasks using a computer, and informed consent was obtained from each subject.

A personal computer featuring a 42-inch LCD monitor with a resolution of 1024 × 768 and a standard mouse were used in the experiment. The experimental tasks for the subjects were programmed in Python, and the ACT-R model was programmed in Allegro Common Lisp.

3.3. Procedure

Before beginning the experiment, the subjects were provided with instructions regarding the experimental procedure and the experimental program from the experimenter, and the subjects were provided time to familiarize themselves with the experimental program at each level of difficulty. In particular, to prevent the subjects from assigning a high priority to a specific task or focusing too much on the performance of the task, it was emphasized to the subjects that there was no reward and that they were free to perform the dual task. The subjects then began with one of the levels of difficulty in the primary task according to the guided experiment sequence. To perform the primary task, subjects had to click on a card with the left button of the mouse, remember the value on the card by mentally determining the answer of the simple calculation or solving the equation in accordance with the level of difficulty, and continue by clicking on another card. If the clicked cards matched, the cards disappeared. The subjects could voluntarily switch to the secondary subitizing task at the desired time while performing the primary task. To switch to the secondary task, the subjects had to click the right button of the mouse. When the secondary task is completed, the primary task is automatically resumed, and the subjects execute the secondary tasks ten times through task switching. A trial ended when the secondary task was completed ten times and all cards in the primary task disappeared. The experiment was performed repeatedly under four different conditions for each level of difficulty.

3.4. Model

An illustrative model for the experiment was developed based on the proposed modeling method. The symbolic structure of the model consists of nine procedural rules concerning the primary task, eight procedural rules concerning the secondary task, and the abovementioned four procedural rules concerning task switching, as shown in Figure 5.

The primary task, i.e., the memory game, consists of ten repetitions of a subtask, and the goal of the subtask, i.e., the subgoal, is to find a pair of matching cards. The nine procedural rules represent the accomplishment of the unity subgoal. Therefore, by starting the memory game, the start-memory-game rule is fired, and a request is made to the imaginal buffer to initialize the subgoal. Each subtask repeats the process of selecting and clicking on a card until it achieves its subgoal, that is, until it finds a pair of matching cards. If the imaginal buffer is initialized with information concerning the subgoal, the choose-random-card rule or the choose-target-card rule is fired. The choose-random-card rule requests the vision module to find a visual-location chunk, including location information concerning an arbitrary card, and requests the vision module to move the visual attention to the location based on the found visual-location chunk. Next, the open-the-card rule requires the visually attended card to be clicked to flip the card, and subsequently, the re-attend-the-card rule requests the card to be visually attended again. Under higher-difficulty conditions, the procedural rules associated with mentally determining the answer to simple calculations or solving equations are additionally fired after the execution of the re-attend-the-card rule. Through this series of rules, the model can identify the content of a card, which, in turn, leads to the inspect-the-opened-card rule, which requests the declarative module to retrieve information concerning the card that was just identified. This represents the cognitive process used to determine whether the location information of the other card that matches the card that has just been identified needs to be recalled. Therefore, if the retrieval fails, it means that the content of the card that has just been identified is not in the declarative module; thus, the choose-random-card rule is fired again. Conversely, if the retrieval succeeds, it means that the declarative module includes the location information of the card with the same content as the card that has just been identified; in this case, the choose-target-card rule is fired and a request is made to the vision module to identify the location. If the clicked cards match and are cleared, the check-the-matched-cards rule is fired instead of the inspect-the-opened-card rule. The check-the-matched-cards rule clears the imaginal buffer to represent the completion of an ongoing subtask.

Both the continue-memory-game rule and the decision-to-switching rule are executed when the imaginal buffer is empty; however, because the utility value of the decision-to-switching rule is set to exceed the value of the continue-memory-game rule by 0.5, the decision-to-switching rule is usually executed. Additionally, the interrupt-task, choose-random-card, and choose-target-card rules are executed when the visual buffer is empty. However, because the interrupt-task rule includes an additional condition for the level of imbalance, the rule is more likely to be executed when the scaled LI_T value is greater than the sum of unity and the imbalance threshold value. The imbalance threshold value was set to 0.3, as shown in

Table 2. Parameter setting values of the model in the experiment.

Parameter	Setting	Value
Level of imbalance	Estimated	0.3
Simbalance	Estimated	0.1
MWstd	Empirical (preset)	207.23
Decay	Estimated	0.5
Latency factor	Estimated	0.5
Visual-num-finst	Estimated	9
Visual-finst-span	Estimated	8

. In addition, to represent variability, the scaled LI_T value is added to the value obtained from the noise generation function provided by ACT-R. The ACT-R noise function generates a value from a logistic distribution with a mean of 0 and an s value of 0.1 (see S_imbalance in Table 2). Moreover, the scaled LI_T is obtained by dividing MW_std, and MW_std in this model is obtained by using the pre-simulated result of the memory game model including only nine procedural rules concerning the memory game in the first level of difficulty. The decay parameter needed to obtain the value of the accumulated mental workload was also set to 0.5, which is generally used in ACT-R.

After the execution of the interrupt-task rule or the decision-to-switching rule, whether or not task switching occurs is determined by comparing the subsymbolic utility value in the do-task-switching rule with that in the do-not-task-switching rule. If the secondary task, i.e., the subitizing task, begins with the do-task-switching rule, then the find-target-button, attend-target-button, and encode-target-button rules are repeatedly executed until all targets are visually attended to represent the search for all playing cards with one diamond. Additionally, the encode-target-button rule records the number of targets detected in the imaginal buffer each time it is executed. Therefore, if all targets are visually attended, the next rules are fired in order, and subsequently, the primary task is resumed again by the restart-memory-game rule. In this study, two parameters, visual-num-finst and visual-finst-span, were adjusted as shown in Table 2, such that the model could find up to nine visual targets. Finally, the latency factor determines the amount of time taken to retrieve a chunk, and it is estimated to have a value of 0.5.

3.5. Results and Discussion

Figure 6 and

Table 3. Task switching percent (%) for subjects and model data just after the ongoing subtask, when the alternative task still remained.

Level of Difficulty	Mean (Standard Deviation)
	Subjects	Model
Level 1	95.29 (5.91)	95.74 (6.78)
Level 2	91.30 (12.40)	99.17 (2.89)
Level 3	96.88 (5.65)	100 (0)
Level 4	88.36 (6.68)	99.07 (3.20)

demonstrate that most of the subjects and the model performed task switching just after the completion of the ongoing task, when the alternative task still remained. Additionally, the root mean square error (RMSE) value between the subjects and model data was 6.791. Separate analyses of variance (ANOVAs) were performed to examine the trends in the results of task switching percentage just after a subtask in each of the subjects and the model data, and the statistical results included p-values of 0.058 and 0.062, respectively. The results demonstrate that most of the task switching in both the subjects and model occurred just after the completion of the ongoing subtask, regardless of the level of difficulty in the ongoing task, which confirms Hypothesis 1. However, Figure 6 demonstrates that the subject data are somewhat lower and more volatile than the model data, which could be interpreted to indicate that humans are more conservative in task switching than the model, and individual differences in human behavior have a somewhat significant effect on task switching.

In the context of Hypothesis 2, Figure 7 and

Table 4. Mean of task switching during the execution of a subtask for subjects and model data.

Level of Difficulty	Mean (Standard Deviation)
	Subjects	Model
Level 1	0.417 (0.67)	0.75 (0.86)
Level 2	1.08 (0.99)	1.25 (1.42)
Level 3	1.17 (0.94)	1.33 (0.98)
Level 4	2.0 (1.13)	2.33 (1.55)

present the results of the subject and model data. Separate nonparametric ANOVAs were performed to examine the trends in the results of the average number of task switching occurrences during the execution of the ongoing subtask of each subject and the model. In addition, the statistical results include p-values of 0.049 (subjects) and 0.005 (model), which implies that the number of task switching occurrences during the execution of a subtask in the ongoing task significantly increased for both the subjects and the model, in accordance with the increased difficulty level of the ongoing task. However, Figure 5 also illustrates that the subject data were somewhat lower than the model data (RMSE = 1.09). The results could also be interpreted to indicate that humans are more conservative in task switching than the model.

The results related to Hypothesis 3, which indicate that there exists a significant difference in the performance time of a small unit of a task for selecting and clicking a card in a subtask in the memory game, depending on whether or not task switching occurs while performing the subtask, are shown in Figure 8 and

Table 5. Performance time (s) in a small unit task for selecting and clicking a card according to the presence or absence of task switching in the memory game for subject and model data.

Level of Difficulty	Mean (Standard Deviation)
	without Task Switching		with Task Switching
	Subjects	Model	Subjects	Model
Level 1	1.2 (0.28)	1.04 (0.27)	1.95 (0.23)	1.25 (0.27)
Level 2	1.24 (0.37)	1.37 (0.30)	1.86 (0.62)	1.79 (0.32)
Level 3	1.36 (0.62)	1.34 (0.32)	1.93 (0.66)	1.76 (0.24)
Level 4	1.99 (0.68)	2.20 (0.70)	3.00 (1.08)	2.75 (0.46)

. We performed a three-way ANOVA to examine whether Hypothesis 3 is statistically correct, and the results illustrate that the difficulty level of the memory game (p < 0.0001) and whether or not task switching occurred during the subtask (p < 0.0001) had a significant effect on the performance time in a small unit of the task of selecting and clicking a card. However, the condition of the subject or model (p = 0.085) had no significant effect on the performance time in the task of selecting and clicking a card. Therefore, we can infer that the model appropriately represents the phenomenon in which the performance time increases in the subtask where task switching occurred, which can be observed in the subject data. However, as shown in Figure 8, there is a difference between subject data and model data, whereby the performance time of the subjects was longer than that of the model at difficulty level 1. According to Adler and Benbunan-Fich [16], task switching can occur not only when the task is too difficult but also when the task is perceived to be too easy. Thus, the difference in results at difficulty level 1 could be attributed to the self-interruption caused by perceiving the ongoing task as being too easy.

Finally, Figure 9 and

Table 6. Total performance time (s) of the illustrated dual task for subjects and model.

Level of Difficulty	Mean (Standard Deviation)
	Subjects	Model
Level 1	107.65 (14.80)	115.31 (5.94)
Level 2	126.97 (26.68)	130.19 (8.93)
Level 3	142.77 (14.02)	129.25 (6.09)
Level 4	189.91 (25.57)	162.02 (12.27)

present the total performance time for the subject data and model data. According to a goodness-of-fit analysis [45], the correlation coefficient and RMSE were calculated as r = 0.97 and RMSE = 18.83 for the total performance time. The results indicate that our task-switching model can predict the total performance time at different levels of difficulty and provide a representation of task switching in discretionary multitasking.

In summary, the results in the experiment demonstrate that the cognitive model in this study can provide a relatively accurate representation, in terms of task-switching percent just after the completion of the ongoing subtask, the amount of task-switching during the execution of the ongoing subtask, and performance time depending on the task difficulty level. It exhibits enhanced performance in predicting human behavior in discretionary multitasking environments and demonstrates how ACT-R facilitates accounts of voluntary task switching. However, since the experiment in this study was conducted with a somewhat small number of subjects and the conclusions should not be interpreted to statistically represent the general population.

4. General Discussion

The objective of this study was to develop an ACT-R model that could provide a quantitative representation of task switching in discretionary multitasking. For this purpose, we extended ACT-R by adding several subsymbolic equations and proposed four procedural rules related to task switching. Additionally, the proposed model was validated through an experimental study, confirming that the model can provide a relatively accurate representation of when and how task switching is performed. However, the model is yet to provide a complete representation of task switching in humans.

According to the results of the experiment performed in this study, the model has limitations in terms of representing the individual differences and conservative aspects of human task switching. Therefore, in future works, relevant parameters of this model must be appropriately readjusted based on human data. For example, by slightly increasing the value of the noise parameter in ACT-R, individual differences can be described. If the weights of the utility value of task end-expectancy were appropriately specified based on human data, the conservative tendency of a person could be represented. Additionally, it is suggested that additional task attributes such as priority, interest, and salience be included based on the STOM model [32], in order to determine the utility value, through which the model can more accurately predict the decision processes involved in task switching.

In conclusion, this study developed an ACT-R model for integrated accounts of task switching in discretionary multitasking. The model can represent cognitive processes of task switching by using several features in ACT-R, including a symbolic structure associated with interrupting ongoing tasks and the decision to switch between tasks, subsymbolic equations concerning mental workload and ability, and a refined utility system. Additionally, the model was validated through an experiment, and the results suggested directions in which the model can be further improved to obtain a more robust model of task switching. Therefore, future studies can investigate accounts of individual differences and additional task attributes related to task switching, and build an integrated ACT-R model based on these accounts.

Practical and Theoretical Implications

The ACT-R-based task-switching model used in this study has several practical and theoretical implications. First of all, one possible application of our model is as a practical tool for evaluating aid systems in discretionary multitasking environments. Several studies on practical applications related to multitasking [46,47] have developed systems that support multitasking performance for optimally managing the mental workload involved in task switching through an approach to aid in memory retrieval or decision making. In the task switching model, task switching occurs at two points: when the ongoing task is too difficult relative to the performer’s ability and when a subtask in the ongoing task is completed. Task switching at the former point occurs during the execution of a subtask, which can have a negative impact on the management of mental workload. Therefore, the proposed model can be used as a tool to evaluate the degree to which the frequency of task switching at the appropriate time improves when using the abovementioned aid systems. Moreover, because the proposed model includes additional decision-making processes associated with voluntary task switching, it can also assess the manner in which aid systems manage the additional demand from these decision processes.

Second, in terms of the ACT-R theory, a general ACT-R model continues to maintain the goal related to the task, and task switching in multitasking environments is represented through a pre-scheduled goal change. However, most task switching phenomena are difficult to predict and occur dynamically in accordance with the manner in which the task is processed. Therefore, Salvucci et al. [14] proposed a modeling approach that can describe adaptive task switching in concurrent multitasking environments. Furthermore, in our study, we proposed a task-switching model that can represent adaptive task switching in sequential multitasking environments.

This implies that the proposed model could apply ACT-R to expanded domains of more complex multiple tasks in discretionary multitasking, in terms of theoretical implications. For instance, in discretionary multitasking, autonomous driving is a domain in which multiple tasks exist, including a primary task associated with vigilance of the driving situation to prepare for a take-over and a secondary task such as reading a book, using a smart phone, or conversing with another person. Task switching is usually performed voluntarily in the autonomous driving domain. Therefore, in the proposed model, integrated accounts of task switching can be of significant assistance in developing the ACT-R model in complex multitasking domains such as autonomous driving.