Usability of an Affective Emotional Learning Tutoring System for Mobile Devices

Abstract

To echo the United Nations formulated Sustainable Development Goals (SDGs), SDG 4 is to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. Furthermore, high-quality education is the base on which human lives can be improved and sustainable development can be accomplished. Therefore, the affective emotional tutoring system established in this study enables learning via mobile devices, which are indispensable in daily life. The real-time interactive agent in the system guides learners to turn negative emotions into positive ones. We explored the usability of and user satisfaction with the affective emotional tutoring system. Sixty-two students participated in the study which used a quantitative research design to explore a learning situation. The overall usability of the system was evaluated with the System Usability Scale (SUS), and the Questionnaire for User Interaction Satisfaction (QUIS) was used to evaluate user satisfaction with the different elements of the system. The results showed that both the usability of and satisfaction with the affective emotional tutoring system were high. The emotional feedback mechanism of the system can help learners turn negative emotions into positive ones.

1. Introduction

1.1. Research Background and Motivation

Online learning can help students flourish due to its advantages, its sustainable development, and the high utilization rate of mobile phones. With advances in mobile technology, mobile devices such as cellphones and tablets have become indispensable in daily life. In the field of education, emphasis is increasingly being placed on mobile learning systems that integrate mobile technology into the teaching environment [1,2,3,4]. Mobile learning has several advantages facilitating the sustainable development of education, including convenience, flexibility, repetition, a low threshold, high teaching resources, and autonomous learning [5]. A system capable of reacting to physiological and psychological changes in students, to reduce negative feelings during the learning process, would be highly beneficial.

Affective computing involves the use of sensors to monitor users’ emotional state, along with related physiological and psychological changes, allowing the system to respond appropriately [6]. The affective tutoring system (ATS) proposed by Kort [7] recognized, analyzed, and provided feedback throughout the learning process, thus establishing an intelligent tutoring system (ITS) that could be combined with affective computing. Recognition of user emotions and corresponding system responses, as well as guidance from the “emotional agent” on how to maintain positive emotions, facilitated learning goals [8,9,10,11]. Therefore, interactive mobile learning systems using mobile devices and affective computing can help learners achieve their learning goals.

1.2. Research Purpose

This study established an affective emotional tutoring system which combined mobile devices with the technique of affective computing. Subjects input the sentences. The Chinese semantic emotional system recognized subjects’ emotions which were expressed in the learning process and provided appropriate feedback to learners to enhance learners’ learning interests. When the system received users’ negative emotions, it would feed positive emotions back correspondingly to guide users to turn negative emotion into positive emotion in the learning process. According to system flow, this study would be divided into three points: (1) Recognition of subjects’ emotions. (2) Record history of learners’ emotional state. (3) Feedback mechanism of interactive agent with high interactivity.

This study used the affective emotional tutoring system to explore whether the system could bring the influence of positive emotion to learners. The research questions are below.

• How do learners feel about the usability of the affective emotional tutoring system which is established on mobile devices?
• How satisfied are subjects with the interaction of the affective emotional tutoring system?

2. Literature Review

2.1. Mobile Learning

As mobile devices are becoming more and more popular, besides the usage in daily life, the property of being unlimited by time and space causes mobile learning to become an important issue in the field of education. Mobile learning is to conduct e-learning through assistive devices, such as a personal digital assistant (PDA), mobile phone, portable notebook, tablet, and so on, which displays learning content and provides the wireless communication channel of bilateral interaction for instructors and learners [12,13]. Mobile devices are taken as the assistive tool of e-learning materials in mobile learning. Learners could learn in any time and place, as if creating a new learning environment and recording the whole process of learning [14,15].

The range of research on mobile learning is wide. For example, the interactive English learning system designed by Hung and Young [1] for six graders integrated the advantage of wireless handheld devices to lead learners to better immersion and interactivity. Due to the urban–rural gap, there is an obvious difference between the educational environment of city and country. Researchers have made use of the property of mobile technology that being the fact it is unlimited by time and space to explore if mobile learning could connect the environment of city and country more closely. The research indicated that mobile learning could shorten the gap of education, and children from different social backgrounds could benefit a great deal [3]. In the research on higher education, Kumar and Bervell [4] took Google classroom as the tool of mobile learning to explore learners’ usage intentions. The research found that the factors which influenced learners to accept Google classroom were habit, hedonic motivation, and performance expectation.

2.2. Intelligent Agent

In 2012, Chen proposed that the basic features which an intelligent agent should possess were autonomous, interactive, and reactivity. Therefore, an intelligent agent is able to make a simple judgment and interact simply with learners. Norman [16] indicated that it was difficult to make a learner form a conceptual model without feedback from the system. Learners had obvious emotional ups and downs based on whether an intelligent agent existed or not. Simultaneously, learners could improve learning effectiveness through the interaction with an intelligent agent and maintain a positive mental state to cope with continuous learning through compassion and encouragement of the agent. The function of an intelligent agent could influence students’ emotions on learning, guide learning, and detect their learning situation [17]. The research also showed that through the operation of an intelligent agent, it was able to strengthen learning motivation, learning interest, and learning attitude, leading to the improvement of learning effectiveness [18]. Mao [8] put forward a digital learning system which had an emotional agent. The emotional agent analyzed users’ emotions through the function of facial recognition, speech recognition, and character recognition to guide them to learn.

2.3. Emotional Tutoring System

In the process of learning, learners’ emotional change would influence learning behavior and even learning effectiveness. Emotion triggers diverse mental process in individuals due to specific external stimulus and generates continuous change as time goes on [19,20]. Through affective computing, an inanimate machine could analyze a human’s emotional message and feed corresponding results back. Picard [21], who is the representative person for affective computing, proposed the idea of affective computing and explained that related research on emotion and computers was divided into four directions, including emotion recognition, emotion expression, having emotions, and emotional intelligence. So far, affective computing has gradually attracted attention. Scholars carried out research through ontology to predict students’ emotions that were generated in the process of learning and pointed out that positive emotions allow it to go smoothly [22]. Through computer analysis, intelligent tutoring systems (ITS) let subjects learn in individualized courses and provide feedback to users immediately [6]. Much research related to emotional tutoring systems has indicated that system interfaces had positive emotional relevance to learners and could improve learning effectiveness [8,9,10,11,23].

Emotion classification is a basic issue in affective computing. There are two main classification methods: one is discrete categories, and the other is continuous vector space [21]. In discrete categories, Ekman, Friesen, and Ellsworth [24] defined six expressions of human faces based on facial muscles and facial features: happiness, anger, sadness, surprise, disgust, and fear. Metri, Ghropade, and Butalia [25] used Ekman’s FACS feature regulations to develop the Facial Expression Recognition System (FERS). In terms of the state of learning, learning emotions are divided into eight types, which include: pleasure, hope, pride, anger, anxiety, shame, hopelessness, and boredom. Positive emotions in learning (pleasure, hope, etc.) and negative emotions in learning (anger, anxiety, etc.) are all multiple emotions that will be experienced in learning [26]. In the emotion classification of continuous vector space, the most common one is James Russell’s Circumplex Model [27,28]. In this classification method, emotions are divided into two axes, “Arousal” and “Valence”, to form all possible emotions. The various discrete categories mentioned above are located in the quadrants formed by the two axes. The PAD three-dimensional emotional model is proposed by Mehrabian and Russell [29] based on environmental psychology and is mainly used to describe and quantify emotional states. Among them, the degree of pleasure (displeasure–pleasure) represents the positive and negative degree of the individual; the degree of arousal (nonarousal–arousal) represents the physiological arousal state of the individual; the degree of submissiveness (submissiveness–dominance) represents the degree of personal dominance over others; and the values of each degree are between from −1 to 1, so that the researcher can clearly define different emotions by the numerical value of the three dimensions. Most of the past studies only focused on the degree of pleasure and the degree of arousal, but some emotions cannot be effectively distinguished using two dimensions (such as fear and anger), so the degree of strength plays a very important role in the PAD three-dimensional emotional model. More advanced, Mendez-Alegria, Y. et al. [30] demonstrated the Wheel of Emotions of Geneva + instrument, which aims to evaluate the emotional state of the user during the execution of tasks in the interactive system. In this method, the author takes “Pleasant” and “Control” as the two axes and forms four facets: satisfied, dissatisfied, enthusiastic, and unenthusiastic. The above-mentioned various discrete categories are located in these four facets [30,31,32,33]. In this study, in order to comply with Figure 1, the process chart of emotional feedback guiding, a relatively simple emotion classification method is adopted, and we only divide it into positive and negative emotions. The decision tree composed of these two emotions constitutes the design of each emotion feedback. Judgment of affective factors can help learning. The system obtains learners’ emotions through facial and semantic emotion recognition, and guides learners to continue teaching activities with positive emotions through interactive agents. The research results show that the assistance of interactive agents can increase learning motivation [34,35,36]. In Figure 1, the system interface is three-column web page layout. The left side is course menu, the center is course information, and the right side is the control button of emotional agent interaction area and the questionnaire link area. The system displays the interactive learning question and learners can input the feedback in the emotional agent interactive area.

3. Methods

The 62 students participated in an interactive technology course. The description of the experimental process was announced on the course. Students with the willingness to participate in the experiment and contacted us. The 62 participants were students from a university in Taiwan, including 33 males and 29 females. The participating students come from two classes in the same school and the same department, taking the same subject, “Interactive Design”. The experimental materials watched by the 62 students were all the same. The main content is interactive technology such as technological art, interactive vision, digital performing arts, wearable devices, light sculpture projection, network media, etc., combined with related online videos on Youtube to increase course richness. They operated the affective emotional tutoring system on tablets in a networked environment. The course was designed such that the subjects could interact freely with the affective emotional agent while they learned. The subjects use the affective emotional tutoring system while learning the content of the interactive technology course in the experimental period, after which the subjects completed a questionnaire regarding system usability and satisfactoriness. They interacted with the emotional agent through a user-operated floating frame. The system interface had a three-column web page layout with the course menu on the left, course information in the center, and emotional agent interaction area and questionnaire on the right.

3.1. Interface Design

The experimental devices were tablets. The function of interacting with the emotional agent was independent, which provided a floating frame for the user to operate. The system interface was a three-column web page layout. The left side was course menu, the center was course information, and the right side was emotional agent interaction area and questionnaire area.

As Figure 1a shows, area A was the course menu; area B was the content of the course; area E was the questionnaire; and area F was the control button of the agent. As Figure 1b shows, when users pushed the button of the agent, it would show area C1 of emotional agent and area D1 of Chinese semantic interaction. Area F was the control switch to show or hide area C1 and area D1 to learn the course, which was in a translucent frame to avoid crowding the page and influencing the learning of the course.

3.2. Course Model

The content of the teaching material in this experiment was interactive technology. The main courses were interactive technology such as wearable devices, projection mapping, and so on, which combined online videos related to interactive technology on Youtube to make the courses rich in content. Subjects watched the course for 5 to 10 min, which included online teaching materials and different learning units.

3.3. Emotional Feedback Guiding Model

Through affective computing, the emotional tutoring system judged users’ emotions and provided feedback via the emotional agent to achieve real-time interaction with users. By analyzing users’ positive or negative emotions, the final goal of affective computing was to maintain users’ positive emotions and guide negative emotion to positive emotion by the emotional feedback model.

This study designed different response models for the emotional feedback guiding model. The system could recognize users’ emotions and give a response of positive emotion, negative emotion, no emotion, or unrecognized. Later, it would show 150 related questions to positive and negative emotion in the text response area to maintain users’ positive emotions and guide negative emotion to positive emotion.

The process of the emotional feedback guiding model was that users answered the first preset emotional question in the course by inputting text, then the system recognized positive or negative emotion in the statement and fed the result of the change of the interactive emotional agent’s emotion and text recognition back to the user. When the result of the recognition was completed, if it presented positive emotion, the system would redisplay related questions to positive emotion for users to answer after 3 s and recognize emotion continuously to guide users to maintain positive emotions. The system continued to guide emotion until there appeared a positive tone or users ended up emotional recognizing themselves. If the system recognized that users had negative emotions, it would show related questions to positive emotion, and guide users to think about things related to positive emotion in order to turn their emotions from negative to positive. Figure 2 was the process chart of emotional feedback guiding.

3.4. Agent Model

The virtual agent could influence learners’ learning emotion, sensed subjects’ learning situation, and give corresponding emotional guidance. Therefore, this system set up the emotional agent as a bridge between learners and the system. In course, users input emotional sentences for the system to recognize. The system would feed corresponding sentences back and change the icon of the agent to strengthen users’ willing to operate. For example, when the system recognized positive emotion in learners’ sentences, the emotional agent would show motion graphics of positive emotion. On the contrary, when the system recognized negative emotion in learners’ sentences, the emotional agent would show motion graphics of negative emotion. If the sentence did not include emotional key words or the system could not recognize the input sentence, it would show image feedback of no emotion or being confused. Figure 3 below is the icon of the emotional agent’s positive and negative emotions.

4. Data Analysis and Results

4.1. Experimental Data Collection and Analysis

Users’ Emotional Data Collection

In the experiment of users testing the emotional semantic recognition of the system, subjects’ emotional data that the system collected were 363 messages in total, as

Table 1. Users’ emotional data.

Emotion Relevance		Number of Answers (Piece)	Percentage
Positive Emotion		189	52%
Negative Emotion		69	19%
	No emotion	36	10%
Unidentified	English	25	7%
	Number	22	6%
	Punctuation Marks	22	6%
Entirety		363	100%

shows.

In the process of the experiment, since there was no limited range of sentences that users and the interactive agent could input, the system could not recognize some words. Table 1 is sorting data of users’ emotion. There were 189 pieces of data where users input positive emotional words, which accounted for 52% of all emotional data. There were 69 pieces of the data where users input negative emotional words, which accounted for 19% of all emotional data. For the unidentified words that the system could not recognize, there were 36 no emotion words, which accounted for 10% of all emotional data. There were 25 related words to the English alphabet, which accounted for 10% of all emotional data. There were 22 number and punctuation marks, which accounted for 6% of all emotional data.

4.2. System Usability Analysis

In order to analyze users’ usability of the system, this study conducted a statistical analysis through collecting a system usability scale that was filled out by the subjects. There were 62 subjects in total. However, there were two subjects who did not fill out the scale, which were determined as invalid scales.

4.2.1. System Usability Scale—Reliability Analysis

Reliability analysis of the system usability scale is as shown in

Table 2. Reliability analysis of system usability scale.

Cronbach’s Alpha	Number of Items
0.809	10

. The higher Cronbach’s Alpha, the higher the reliability of the scale. The minimum acceptable reliability is Cronbach’s Alpha > 0.7 [37]. Cronbach’s Alpha of this scale was 0.809, higher than 0.7, which represented that there was high reliability of internal consistency in this scale.

4.2.2. System Usability Scale—Descriptive Statistics

The statistical result of scores of every question in system usability scale is as

Table 3. Descriptive statistics of system usability scale.

			Percentage of Each Question in 5-Point Likert Scale (%)
Question	Mean	Standard Deviation	1	2	3	4	5	4 + 5
Q1: I think I would often use the system.	3.25	0.84	3.3	11.7	45.0	36.7	3.3	40.0
Q2: I think the system is too complicated.	3.95	0.79	0.0	5.0	18.3	53.3	23.5	76.8
Q3: I think the system is easy to use.	4.17	0.67	0.0	1.7	10.0	58.3	30.0	88.3
Q4: I think I need a technician’s help to use the system.	3.61	0.90	3.3	6.7	26.7	51.7	11.7	63.4
Q5: I think all functions of the system were integrated well.	3.45	0.87	3.3	6.7	40.0	41.7	8.3	50.0
Q6: I think there was too much contradiction in the system.	3.57	0.89	1.7	10.0	30.0	46.7	11.7	58.4
Q7: I think most people could learn how to use the system fast.	4.33	0.68	0.0	0.0	11.7	43.3	45.0	88.3
Q8: I think the system is very difficult to use.	4.28	0.58	0.0	0.0	6.7	58.3	35.0	93.3
Q9: I think I am very confident of using the system.	4.28	0.83	1.7	0.0	13.3	38.3	46.7	85.0
Q10: I think I have to learn something to use the system.	3.90	0.82	0.0	6.7	18.3	53.3	21.7	75.0
All	3.87	0.78	1.3	4.9	22.0	48.2	23.7	71.9

, which showed that subjects’ feeling on using the system was 71.9%. Table 3 below is the statistical result of every question.

The sum of the highest and the second highest percentage of each question in the 5-point Likert scale represented Agree; the third highest percentage of each question in the 5-point Likert scale represented Neutral; and the sum of the lowest and the second lowest percentage of each question in the 5-point Likert scale represented Disagree. The analysis result is as below. For Q1, 40% of the subjects thought that they would often use the system; 45% of the subjects chose neutral; and 15% of the subjects disagreed. For Q2, 76.8% of the subjects thought that the system was not complicated; 18.3% of the subjects chose neutral; and only 5% of the subjects thought that the system was too complicated. For Q3, 88.3% of the subjects thought that the system was easy to use; 10% of the subjects chose neutral; and only 1.7% of the subjects disagreed. For Q4, 63.4% of the subjects thought that they did not need a technician’s help to use the system; 26.7% of the subjects chose neutral; and 10% of the subjects needed a technician’s help. For Q5, 50% of the subjects thought that all functions of the system were integrated well; 40% of the subjects chose neutral; and 10% of the subjects disagreed. For Q6, 58.4% of the subjects thought that there was not too much contradiction in the system; 26.7% of the subjects chose neutral; and 11.7% of the subjects thought that the system was contradictory. For Q7, 88.3% of the subjects thought that they could learn how to use the system fast; 11.7% of the subjects chose neutral. For Q8, 93.3% of the subjects thought that the system was not very difficult to use; only 6.7% of the subjects chose neutral. For Q9, 85% of the subjects thought that they were confident in using the system; 13.3% of the subjects chose neutral; and only 1.7% of the subjects disagreed. For Q10, 75% of the subjects thought that they did not have to learn other skills to use the system; 18.3% of the subjects chose neutral; and only 6.7% of the subjects thought that they had to learn other skills to use the system.

4.3. User Interaction Satisfaction Analysis

In order to analyze subjects’ interaction satisfaction on the usability of the system, the study collected the questionnaire for user interaction satisfaction that was filled out by the subjects. There were 62 subjects participating. Among them, there were two subjects who did not fill out the questionnaire, which were determined as invalid questionnaires.

4.3.1. Questionnaire for User Interaction Satisfaction Analysis—Reliability Analysis

Reliability analysis of system usability scale is as shown in

Table 4. Reliability analysis of questionnaire for user interaction satisfaction.

Cronbach’s Alpha	Number of Items
0.941	28

. The higher Cronbach’s Alpha, the higher the reliability of the scale. Cronbach’s Alpha value of this scale was 0.809. The minimum acceptable reliability is Cronbach’s Alpha > 0.7 [37]. Cronbach’s Alpha of this scale was 0.941, higher than 0.7, which represented that there was high reliability of internal consistency in this scale.

4.3.2. Questionnaire for User Interaction Satisfaction Analysis—Descriptive Statistics

The study analyzed the six dimensions of the questionnaire for user interaction satisfaction, including overall using reaction, screen presentation, terms and system information, learning factor, system performance, and availability and user interface. Percentages of five points to seven points in the 7-point Likert scale were added up as positive scores. The description of the results is below, as shown in

Table 5. Descriptive statistics of the questionnaire for user interaction satisfaction.

Dimension	Quest-ion	Mean	Mean Dimension	Standard Deviation	Percentage of Each Question in 7-Point Likert Scale (%)
					1	2	3	4	5	6	7
Overall using reaction	Q1	5.20	5.15	1.05	1.7	0.0	1.7	18.3	36.7	35.0	6.7
	Q2	5.83		1.22	0.0	3.3	1.7	5.0	21.7	30.0	38.3
	Q3	5.20		1.27	1.7	3.3	3.3	11.7	38.3	28.3	13.3
	Q4	4.53		1.24	1.7	3.3	18.3	13.3	48.3	10.0	5.0
	Q5	5.05		1.15	0.0	3.3	5.0	21.7	30.0	33.3	6.7
	Q6	5.08		1.14	1.7	0.0	6.7	15.0	41.7	26.7	8.3
Screen presentat-ion	Q7	5.93	5.7	1.15	0.0	0.0	6.7	5.0	13.3	38.3	36.7
	Q8	5.52		1.19	1.7	0. 0	5.0	5.0	35.0	33.3	20.0
	Q9	5.68		1.16	1.7	0.0	1.7	6.7	31.7	31.7	26.7
	Q10	5.65		1.15	1.7	0.0	1.7	10.0	23.3	41.7	21.7
Terms and system information	Q11	5.7	5.45	1.21	1.7	0.0	0.0	13.3	26.7	26.7	31.7
	Q12	5.37		1.09	0.0	0.0	3.3	21.7	25.0	35.0	15.0
	Q13	5.77		1.11	1.7	0.0	0.0	6.7	30.0	33.3	28.3
	Q14	5.77		1.2	1.7	0.0	3.3	5.0	25.0	35.0	30.0
	Q15	5.33		1.27	1.7	0.0	5.0	16.7	30.0	26.7	20.0
	Q16	4.77		1.47	3.3	3.3	8.3	25.0	35.0	8.3	16.7
Learning factor	Q17	6.25	5.88	0.77	0.0	0.0	0.0	1.7	15.0	40.0	43.3
	Q18	5.62		1.04	0.0	0.0	1.7	15.0	25	36.7	21.7
	Q19	5.88		0.96	0.0	0.0	1.7	5.0	26.7	36.7	30.0
	Q20	6		0.86	0.0	0.0	0.0	3.3	26.7	36.7	33.3
	Q21	5.67		1.04	0.0	0.0	0.0	18.3	20.0	38.3	23.3
System perform-ance	Q22	6.07	5.8	0.95	0.0	0.0	0.0	8.3	16.7	35.0	40.0
	Q23	5.52		1.11	1.7	0.0	1.7	11.7	26.7	43.3	15.0
Availabil-ity and user interface	Q24	5.92	5.02	1.09	1.7	0.0	0.0	5.0	23.3	36.7	33.3
	Q25	5.49		1.24	1.7	0.0	3.3	13.3	30.0	28.3	23.3
	Q26	4.88		1.3	1.7	1.7	8.3	28.3	25.0	25.0	10.0
	Q27	5.2		1.35	1.0	0.0	8.3	23.3	16.7	33.3	16.7
	Q28	3.63		1.84	11.7	20.0	23.3	13.3	10.0	13.3	8.3
Entirety		5.44	5.5	1.17	1.5	1.4	4.3	12.4	26.9	31.3	22.3

.

1.

Overall using reaction

In Q1, 78.4% of the subjects considered that overall using reaction was good. In Q2, 90% of the subjects considered that overall using reaction was easy. In Q3, 79.9% of the subjects considered that overall using reaction was satisfactory. In Q4, 63.3% of the subjects considered that overall using reaction was high-efficient. In Q5, 70% of the subjects considered that overall using reaction was interesting. In Q6, 76.7% of the subjects considered that overall using reaction was flexible.

2.

Screen presentation

In Q7, up to 88.3% of the subjects considered that the text on the screen of the system was easy to read. In Q8, up to 88.3% of the subjects considered that the message presentation on the screen of the system was helpful. In Q9, up to 90.1% of the subjects considered that overall planning of the system screen was clear. In Q10, up to 86.7% of the subjects considered that the presented style of the system screen was clear.

3.

Terms and system information

In Q11, up to 85.1% of the subjects considered that the technical terms that the system used possessed consistency. In Q12, up to 75% of the subjects considered that the described position on the system screen was consistent. In Q13, up to 91.6% of the subjects considered that the prompt message on system screen was very clear. In Q14, up to 90% of the subjects considered that the prompt message on system screen was very clear. In Q15, up to 76.7% of the subjects considered that the message in which the system noticed users was consistent with actual operation. In Q16, up to 60% of the subjects considered that the prompt message of error could help improve operational problems.

4.

Learning factor

In Q17, up to 98.3% of the subjects considered the system easy to use. In Q18, 83.4% of the subjects considered that they could learn new functions through trying to explore. In Q19, only 93.4% of the subjects considered that the name and the use of commands were easy to remember. In Q20, up to 96.7% of the subjects considered that the execution of tasks was easy to understand. In Q21, up to 81.6% of the subjects considered that the explanatory message on the screen was helpful.

5.

System performance

In Q22, 91.7% of the subjects considered that the execution speed of the system was fast enough. In Q23, 85% of the subjects considered that the system was reliable.

6.

Availability and user interface

In Q24, 93.3% of the subjects considered that the color matching of the system was good. In Q25, 81.6% of the subjects considered that the information and feedback of the system was good. In Q26, 60% of the subjects considered that the response to errors of the system was good. In Q27, 66.7% of the subjects considered that the information of the system was enriched enough. In Q28, only 31.6% of the subjects considered that the system was chaotic and that there was too much interference in user interface.

Then, this study analyzed the correlation of the six dimensions in the questionnaire for user interaction satisfaction. It used the Pearson correlation coefficient to analyze overall using reaction, screen presentation, terms and system information, learning factor, system performance, and availability and user interface.

Table 6. Correlation coefficient analysis of every dimension in user interaction satisfaction.

Variable	Overall Using Reaction	Screen Presentation	Terms and System Information	Learning Factor	System Performance	Availability and User Interface
Overall using reaction	1
Screen presentation	0.76 **	1
Terms and system information	0.72 **	0.83 **	1
Learning factor	0.31 *	0.42 **	0.52 **	1
System performance	0.71 **	0.67 **	0.76 **	0.47 **	1
Availability and user interface	0.62 **	0.63 **	0.71 **	0.38 **	78 **	1

Note. ** p < 0.01; * p < 0.05.

shows that the correlation coefficients of the six dimensions were all significant positive correlations. The highest correlation coefficient was screen presentation on terms and system information, which was 0.83. It represented that the better the screen presentation, the faster the user could get the hang of terms and system information. The lowest correlation coefficient was learning factor on overall using reaction, which was 0.31. On the whole, the system was highly satisfactory in user interaction satisfaction.

5. Discussion and Conclusions

This study established an interactive affective emotional tutoring system based on a web page for use with a mobile device. Real-time interaction of the user with the emotional agent provided emotional feedback and guidance. The system increased user interest in learning and provided a deeper learning experience. This study analyzed the usability of the system through the SUS and explored system satisfaction using the QUIS. The analysis showed that most users were satisfied with the usability of the system and considered it easy to operate. They learnt how to use the system quickly and could operate it confidently. Therefore, the interactive affective emotional tutoring system had good usability on a mobile device. The overall satisfaction of the learners was high. The users considered the system design to be clear and simple. Additionally, the learning process was engaging, which increased willingness to learn. Thus, this study certified the factors that influence learners’ continuous use of mobile devices in online education and benefit the sustainable future of mobile online learning in Taiwan.

Emotional balance represents an important state of mind. The balance means that there is a certain degree of constant energy in the human body, which tends to be evenly distributed. This volume represents the moderate tension in the human body, and the restoration to a moderate state represents balance. Many studies have included this dimension in the evaluation methods (such as questionnaires) to measure it [31,38] and although this study did not evaluate emotional balance, we try to achieve emotional balance through emotional feedback from the system. Norman [39,40] pointed out that for users to form a conceptual model, it is difficult to achieve it without systematic feedback. The user will have obvious emotional ups and downs due to the presence or absence of the intelligent agent. At the same time, the user can increase the efficiency of learning through interaction with the intelligent agent, and the more the agent’s compassionate and encouraging sentences can be used the more the mental state of the person remains positive and can cope with continuous learning. The function of intelligent agents can affect students’ learning emotions and can guide learning as well as detect students’ learning conditions [10,11]. An Affective Tutoring System (ATS) is a system that can provide users with appropriate teaching content through computer calculations and analysis and can directly give users instant feedback. The users give appropriate feedback, so the smart tutoring system has more advantages compared with traditional teaching [41,42].

We have made observations in the experiment and have the following findings: (1) Students from the field of science and engineering in high school have a higher acceptance of interactive technology. However, students from the humanities field in high school are more creative in design thinking. (2) Although boys have a high acceptance of interactive technology, girls show higher interest and curiosity in emerging technologies. (3) Since they come from the same experimental body and similar learning experience, we did not conduct sentiment analysis in terms of age, knowledge level, etc. (4) At present, only observation is used, and we expect to conduct quantitative analysis again. Regarding this part, we have added them into our future works.

6. Future Work

When faced with e-learning systems, the main task of users is to learn. This is a rather abstract process. Traditional user research focuses on objective parameters that are measurable and verifiable and ignores the hedonic component, and therefore cannot reflect the motivations of learners when using e-learning systems. Therefore, the evaluation expert user experience (UX) should significantly increase their efforts in such an environment when dealing with [31,32,33]. ISO/IEC 25010 provides a definition that is fully adapted to the user experience of human–machine systems, including computer systems in use and software products in use. In this standard, satisfaction is divided into four sub-features, which are designed to identify the needs of users when using the system in a specific use environment: utility, confidence, pleasure, and comfort. In the research of Aguirre, Villareal-Freire1, Gil, and Collazos [43], ISO/IEC 25010 was cleverly adopted to design the best UX experience evaluation method [31,32,33]. We will use this as a basis and explore UX evaluation in future research.