Next Article in Journal
Pilot-Based Adaptive Channel Estimation for Underwater Spatial Modulation Technologies
Previous Article in Journal
Some Identities of Fully Degenerate Bernoulli Polynomials Associated with Degenerate Bernstein Polynomials
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Symmetry
Volume 11
Issue 5
10.3390/sym11050710
symmetry-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Applying Event-Related Potentials to Measure Perceptual Experience toward the Navigation Interface of a Mobile Game for Improving the Design

by
Fu Guo
,
Jun-Yi Jiang
*,
Xiao-Hui Tian
and
Jia-Hao Chen
School of Business Administration, Northeastern University, No. 195 Chuangxin Road, Hunnan District, Shenyang 110167, China
*
Author to whom correspondence should be addressed.
Symmetry 2019, 11(5), 710; https://doi.org/10.3390/sym11050710
Submission received: 8 April 2019 / Revised: 17 May 2019 / Accepted: 21 May 2019 / Published: 24 May 2019
Browse Figures
Versions Notes

Abstract

:
High-level user experience has become the key factor that one game can be successful in the game market. The home page of mobile games, especially the design of the navigation interface, has a significant impact on users’ initial experience, which is an important determent to users’ preferences and purchase decision. Hence, measuring users’ perceptual experiences of the navigation interface can help designers understand real demands from users. Previous studies primarily used self-report scales or interviews to measure gamers’ perceptual experiences. However, it may not reflect gamers’ real perceptions that they are feeling as most of time the feeling is short-lived and implicit. To fill this gap, the current study attempted to combine subjective evaluation with event-related potentials (ERP) to objectively measure gamers’ perceptual experience evoked by the navigation interface of the mobile game. The navigation interfaces of mobile games with low, medium, and high perceptual experience were developed and the ERP experiment was conducted to detect the differences in users’ electroencephalograph (EEG) components when subjects were exposed to the different design levels of navigation interface. The results showed that N1 reaction showed asymmetry in brain regions, and P2 and N2 showed symmetry, and relative to the navigation interface with low and medium perceptual experiences, the high level of navigation interface induced a larger amplitude of N2 in the anterior scalp and P2 in the frontal scalp. These EEG components can, therefore, be regarded as significant indicators reflecting gamers’ perceptions of the navigation interface. The findings benefit game companies of navigation interface designs.

1. Introduction

In recent years, with the popularization of smart mobile devices, the number of mobile games and mobile gamers has increased rapidly, and playing mobile games has become an important leisure and entertainment activity in people’s daily lives. Today, there are more than 20,000 mobile games operating online, but few of them have achieved great success. A survey of gamer retention by Appsee, a mobile-user data analytics firm, found that mobile games had a low average first-month retention rate of 22%. Nearly 50% of gamers choose to remove mobile games because their poor initial impression [1]. The navigation interface is the first channel for gamers to interact with a game, which directly affects gamers’ perceptual experiences and their subsequent purchase decisions [2,3]. In the past, the design of the game interface was mostly based on the designers’ own preferences and knowledge, which could not meet the real needs of gamers. Therefore, game designers began to pay more and more attention to the gamers’ subjective feelings. Gamers’ behaviors and decisions in the game are influenced by their perception of the game, but game designers are often unaware of gamers’ real needs. It is impossible for the gamers to fully understand all aspects of the game when they first started the game [4]. Usually, they will form a general feeling in a very short period, and accept the external information selectively by visual attention [5]. The decision-making behavior of gamers is more likely to take place under the condition of perceptual decision-making, which is also what the game companies want to achieve [6,7]. Therefore, games design is also more concerned with gamers’ emotional demands, such as emotional design, pleasure experience, and other concepts proposed. Gamers will be limited by the reception of information, affected by the flow of attention, and intuitively or sensibly make choices when confronted with a large number of game choices [8]. For example, when the user unwittingly watches the person beside him is playing games, he will be fascinated with visual impact on the screen, or be attracted by the first impression on the home page of the game and arouses the intention to play it, which can be seen as perceptual experience. This stage is mainly because the gamers have the intention to use the game or the sense of the visual experience, and then bringing about the corresponding changes in behavior, psychology, and physiology of the further experience. The appearance of the game interface motivates the experience of the gamer’s perceptual thinking, which is the key factor to determine the gamer’s choice and action. Owing to the incomplete acquisition of product information, this stage is more sensitive to the product’s perception and evaluation so that it was called the perceptual experience. However, due to the perceptual experience of mobile game possessing subjectivity, dynamic, and environment-dependence, how to measure it scientifically has become a significant and difficult issue.
Existing studies mainly focused on applying the subjective method, such as self-reporting, interviews, and questionnaires, to measure gamer experience toward navigation interface of games. These methods cannot synchronously reflect gamers’ mental state and perceptual process in the game. Meanwhile, gamers often feel it is difficult to express their true feelings due to poor language, resulting in unconvincing measured results. Especially, when subjects are at low attention, self-reporting will destroy subjects’ game experience, and questionnaires also have the problem of excessive subjectivity [9]. The physiological signal generated by internal organs of the human body is a kind of biological electrical signal varied with emotional changes. Physiological data can be obtained continuously by specialized instrument, to evaluate the emotional state of gamers in real time [10]. Compared with subjective methods, although physiological signals may be affected by external factors, such as subjects’ physical movement, sweating, and fatigue, it can more objectively reflect subjects’ emotional changes. Previous physiological instruments used to measure the emotion of gamers include facial action coding system (FACS), galvanic skin reaction (GSR), and electrocardiogram (ECG), etc. However, the limitations of these methods are gradually exposed with the deepening of research. For example, mild stimulation often fails to cause observable changes in subjects’ facial expressions, failing to fully reveal the user’s continuous emotion changes [11]. Subjects’ age, gender, body temperature, skin humidity, breathing rate, etc., all of these factors can cause the certain effect to GSR results, and the differences in personality will also affect the results [12]. Therefore, it is difficult to compare the GSR results of gamers among groups or within the group in different test periods [13]. While, ECG can only make differences at the 6th second after the presentation of a positive stimulus and negative stimulus, and cannot distinguish emotional changes within the first five seconds [14]. Therefore, it is necessary to explore more accurate physiological means for measuring perceptual experience of mobile games.
Neuroscience has been widely used in the field of user experience, among which event-related potentials (ERP) provide a method to access and record neural activity from the scalp [15]. It is a reliable and high-resolution physiological means, allowing the computer to retrieve and analyze the spectral characteristics of the EEG (electroencephalograph) component generated when the user is thinking [16]. Compared with other methods of brain nerve (positron emission tomography, functional magnetic resonance imaging, etc.), non-invasive ERP can directly reflect gamers’ psychological activities with low cost, therefore becoming a common technology to explore brain activity gradually. The principle of ERP experimentation is to record subjects’ electrophysiological responses to external stimulation, which is mainly reflected in the change of the amplitude of brainwave in different brain regions and the incubation time causing change. Frontal N/P1, for example, represents a negative/positive potential with peak latency, approximately 100 ms after stimuli are displayed and distributed on the frontal scalp. The brain regions being analyzed commonly include a frontal region (F), the central region (C), parietal region (P), occipital region (O), and temporal region (T). Previous studies showed that ERP are reflecting early visual attention mainly occurred between 100 and 300 ms, and when the attraction of stimulus was higher, a smaller N1 amplitude would be induced [17]. The ERP has been widely used in various disciplines. For example, Ding (2017) applied ERP to explore the influence of smartphones on user’s aesthetic experience and found that N1 could be used as an aesthetic index for smartphones design [18]. A visual aesthetic scale was applied to determine the aesthetic level of the smartphone. ERP results showed that smartphones with higher aesthetic level would cause subjects eliciting lower N1 amplitude, with a significant difference compared with lower aesthetic smartphones. Luck et al. (2000) found that compared with neutral pictures, subjects watching positive or negative pictures would cause a larger N2 amplitude, meanwhile, subjects had longer annotation duration and more annotation points [19]. Cudo et al. (2018) found that higher arousal images would cause the subject more negative N2 amplitude, while the valence of the images did not significantly affect the change of N2 amplitude [20]. Recio et al. (2014) applied ERP technology to explore the brain cognitive mechanism when the user perceived word images with different levels of arousal and valence [21]. The results showed that words with higher arousal evoked larger P2 amplitude significantly, but the emotional valence of the words did not cause significant changes. In addition, some scholars showed that P2 was also related to aesthetics and preference. The subject could induce P2 with larger amplitude when observing the preferred brand logo, showing that P2 could reflect the aesthetic experience of stimulus.
At present, there is little research on the application of ERP technology in game evaluation and game design. The perceptual experience refers to a perception and response that the user does not fully interact with products, and more is the willing and desire to further interact with products. In this process, the product appearance and other experience that stimulates the user’s perceptual thinking is the most important factor determining the user’s decision and behavior [22,23]. Since people cannot fully explain their cognition and psychology, especially at low attention level, studies on ERP can provide information that cannot be obtained by traditional methods (interviews, questionnaires, and focus groups). Therefore, the research on the user’s neural response can not only help us understand the internal formation mechanism of perceptual experience but also provide a new method to measure perceptual experience. Based on previous studies on user experience and brain cognition [24,25], this paper aims to explore the differences in ERP induced by navigation interfaces of the mobile game with different levels, to fulfill for the shortcomings of traditional user experience methods.
The self-developed navigation interfaces of mobile games were selected as the stimulation material in the ERP experiment. A perceptual experience scale of mobile games was developed for determining the levels of stimulation material, being divided into three groups (low level, medium level, high level). In this paper, the perceptual experience refers to the perception and reaction caused when browsing the navigation interface of mobile games, that is, the first visual impression to the user, and does not contain any interaction process between gamer and navigation interface. This was a bottom-up process of visual information processing, where gamer’s attention was usually guided by stimuli such as visual saliency (color, contrast, etc.), object size, and visual position. An oddball paradigm was used in the ERP experiment, with navigation interface pictures as frequent non-target stimuli and landscape pictures as rare targets. Finally, the EEG components induced by high, medium, and low-level navigation interfaces were compared. This study expects that navigation interfaces of mobile games with different perceptual experiences can cause gamers differences in attention allocation and perception. Subconsciously, gamers will show certain preferences or tendentiousness to navigation interfaces with high perceptual experience, mainly manifested in the differences in EEG components. According to previous studies on visual ERP, it is presumed that these differences are mainly reflected in the changes of N1, N2, and P2 potentials.

2. Methods

This study combined scale and ERP technology to measure gamers’ perceptual experiences of navigation interface on mobile game. Before the ERP experiment, we have developed a scale to measure gamers’ perceptual experience of navigation interface on mobile game. Then, the scale was used to test the stimuli that were created to conduct the ERP experiment. According to the results, the stimuli were divided into three perceptual experience levels (low, medium, and high). After that, The ERP experiment was conducted to further explore subjects’ neural response when they were watching the three levels of stimuli.

2.1. Construction of the Perceptual Experience Scale

2.1.1. Stimuli

The selection of experimental materials included two channels: downloading ready-made mobile games on mobile APP platform and developing games independently. Choosing ready-made mobile games was more convenient and less costly, but this way was hard to control the differentiation of stimulus with the same game type, and subjects’ proficiency to the selected game would also interfere with the evaluation. Considering that the level control of stimulus would directly affect the correctness of the study and, hence, developed games independently as ERP stimulus. Following the principles of game development, six versions of mobile games with the same theme were created by using Unity3D engine (Unity Technologies, UK), which were named CrazyLily1.0 to CrazyLily6.0, respectively (Figure 1). The game type is the most popular casual mobile game (the gamer needs to estimate the length of the bridge to get through a random obstacle). The major differences of these six navigation interfaces were in the logo, controls, layout, and background, and the brightness and contrast were set as the same in all conditions.

2.1.2. Indexes of Perceptual Experience

According to the definition of perceptual experience given above, the user’s visual perception will be in a dominant position. According to the studies on the visual aesthetic of product appearance, the usability of APP navigation interface and playability of the game, the scale includes five dimensions: (i) emotional perception [26], (ii) gameplay [27], (iii) usability [28], (iv) attractiveness [29], and (v) tendency for further experience. Interface attributes can trigger gamers’ emotional responses, and gamers’ emotions before playing mainly consist of amazement, satisfaction, pleasure, and expectation. Gameplay usually refers to the immersion and flow experience brought to users by the theme, content, and mechanism of the game, and for the navigation interface where the user’s visual perception is dominant, the gameplay is closer to the visual aesthetics inspired by the navigation interface. Design of controls, logo, background, and collocation of these elements will directly affect the aesthetics of the navigation interface. Visual usability mainly refers to the degree to which the navigation interface realizes the game functions and satisfies the gamers, and is evaluated by measuring layout clarity, usability, and friendliness. Perceived attractiveness caused by the appearance of the navigation interface refers to gamers’ subjective impression at a low attention level and consists of creativeness, impression, and capacity to keep the gamer’s attention and interest. Finally, there is an additional question reflecting the gamer’s tendency to experience the mobile game. The evaluation is based on a seven-point Likert scale (1, completely disagree, to 7, completely agree).

2.1.3. Questionnaire Design

As shown in Table 1, the questionnaire includes demographic information (i.e., gender, age, education background, the time and frequency of playing mobile game), a screenshot of navigation interface, and items concerning gamers’ emotional perception, usability, gameplay, attractiveness, and tendency to future experience. To validate the effectiveness of the questionnaire, we sent the questionnaire to 386 students and employees from colleges in the north of China by e-mail. Each participant only evaluated one of the six navigation interfaces. Eventually, a total of 335 questionnaires were collected, of which 14% of respondents used mobile games for 0.5‒1 years, 26% for 1‒2 years, 58% for more than two years, and less than 2% for less than half a year. It can be seen that all respondents are familiar with mobile games, which ensures the reliability of the sample. The results of reliability test suggested that the questionnaire has a high reliability with Cronbach alpha = 0.968 > 0.7 [30].

2.2. ERP Experiment

2.2.1. Stimuli and Apparatus

The ERP stimuli were composed of three screenshots of navigation interfaces and three landscape pictures. Three screenshots of navigation interfaces, including Crazylily1.0, Crazylily3.0, Crazylily6.0, were selected as the stimuli by the expert panel. The three landscape pictures were downloaded from web pages. All stimuli were displayed on the center of a 19-inch LCD (1440 × 900 pixels, 60 Hz) with a gray background. To get closer to the real phone screen, the sizes of the picture were adjusted to 700 × 460 pixels, and the pictures were centrally presented by E-prime 2.0 professional software. The experiment used a Neuroscan electroencephalograph (EEG) system to record and analyze the EEG signals. The electrodes position was constructed based on an international 10‒20 system, and the Ag/AgCl64 conductive electrode cap was used.

2.2.2. Subjects

A total of 22 undergraduates, graduates, and doctors from Northeastern University (12 males; ages range: 20‒30 years old) were recruited for ERP experiment. They were all right-handed, had a normal or corrected-to-normal vision, and had no history of neurological or mental illness. The study complied with departmental ethics committee regulations. Before participating in the experiment, all the subjects signed informed consent. All subjects had been playing mobile games for more than two years and were familiar with casual mobile games. Moreover, they had never experienced or learned about the mobile games used as stimuli before the experiment.

2.2.3. Procedure

Figure 2 presents the paradigm of the ERP experiment. The experiment adopted the oddball paradigm. The pictures were randomly presented, in which the targets occurred 25% of the time, and the non-target was occurred 75%. Moreover, the three navigation interfaces of mobile games that were non-target stimuli were presented randomly one at a time with equal frequency, in which the one with the highest perceptual experience is a high group, the one with the lowest perceptual experience score is a low group, and the other is a medium group. In the experiment, the subjects were required to click the left mouse rapidly when the picture of the landscape was presented. Therefore, the brain nerve reaction to the navigation interface of mobile game was generated at a low attention level. After the ERP experiment, subjects need to evaluate the perceptual experience of three navigation interfaces of mobile games, which provided a reference for the ERP analysis.
The ERP experiment was conducted in a quiet and soft-lit laboratory. Before the experiment, the participants need to adjust their sitting posture and sat comfortably. The distance between the eyes and the screen was about 90 cm, and the horizontal and vertical viewing angles were 12.9° and 6.8°, respectively. The participants were required to click the left mouse button as soon as possible when the picture of the landscape appeared. The display time of each navigation interface was 1200 ms; then a blank screen was shown for 1200~1500 ms. During the experiment, two rest periods were set, and the rest time was decided by the subjects themselves. After the rest time, participants needed to click the left mouse button to continue. Each navigation interface picture was presented in 50 times, and the experiment lasted about 15‒20 minutes. The entire experiment, including the preparation time, took about 35 minutes.

2.2.4. Electrophysiological Recording and Analyses

Figure 3 depicted the scalp location of the electrode points collected in the experiment. The EEG data of 24 electrode points (FP1, FPZ, FP2, F7, F3, FZ, F4, F8, FC3, FCZ, FC4, T7, C3, CZ, C4, T8, P7, P3, PZ, P4, P8, O1, OZ, O2) were recorded using Ag/AgCl conductive electrode caps. During the recording process, a reference electrode was placed on the left mastoid, and an electrode placed on the right mastoid was used to record EEG offline data, and the FPZ and FZ were used as grounding electrodes. Vertical electro-oculographic (VEOG) and horizontal electro-oculographic (HEOG) were recorded with additional electrodes placed 1.5 cm above and below the left eye and 1.5 cm outside the outer canthi of both eyes. The resistance at each electrode was hold under 5 kΩ. The EEG signals were filtered with a band-pass of 0.05~100 Hz, and the sampling frequency was 500 Hz per conductor.

2.3. Data Processing and EEG Component Selection

2.3.1. Data Processing

Before data processing, the EEG data should be converted into a bilateral mastoid reference. To ensure the purity of brain signals, a series of data processing was carried out by Curry7.0 SBA. First, the brain signal fragments with obvious drift were removed. Then, the EEG from −200~0 ms before stimuli presentation to 800 ms after stimuli presentation was intercepted, and the baseline was corrected with an average amplitude of −200~0 ms. The threshold for eliminating vertical eye movement artifacts was set to 200 μV, out-of-range signals would be eliminated, and the filter band pass was 1~30 Hz. The collected data were divided into three groups, and event-related potentials were superimposed and averaged in combination with subjective or behavioral data of related experiments. (Curry 7.0 SBA). Finally, statistical analysis and topographic map analysis were carried out. The principles of ERP acquisition and data processing was as follows, as shown in Figure 4.

2.3.2. EEG Component Selection

ERP was originally called evoked potentials (EP). The word ERP was put forward by Vaughan in 1969. ERP is defined as when external stimulus can induce the sensory system or cause corresponding psychological activities of the subject, this stimulus or certain psychological activities can produce potential changes in relevant brain regions. In the process of EEG signal acquisition, ERP is submerged in spontaneous potential (EEG). Although EEG contains relevant physiological and psychological information, it is not the waveform itself that is caused by the information. ERP amplitude induced by one stimulus is normally 2~10 μV, which is submerged in the EEG. Hence, the ERP component needs to be extracted from EEG. ERP has two constant basic characteristics: constant latency and constant waveform [31]. Therefore, through superimposing EEG segments induced by the same stimulus or psychological activity, ERP will be highlighted in the continuous superimposition due to two constant characteristics (latency and waveform), while spontaneous EEG will cancel each other out due to its own disorder. After n times of superposition, the ERP amplitude increases by n times. However, the EEG is added in the form of random noise, and its amplitude is only increased by n times. The amplitude of ERP after superposition is n times of the EEG, which is revealed from the background of the EEG. After averaging, the ERP value of one stimulus is obtained. ERP usually uses amplitude, distribution and latency to reflect related psychological activities. Among them, N-negative and P-positive represent the polarities of components. For example, if N200 appears in the frontal region, it represents a negative wave with a peak latency of 200 ms, and N200 in the anterior part of brain is related to cognitive matching, etc. The perceptual experience stage mainly receives information through visual senses, and then forms certain psychological perceptions and decision-making behaviors through high-level cognitive processing. Therefore, the components obtained in the experiment are mostly related to vision. Figure 5 illustrates the process of EEG superposition. The EEG components induced by the same stimulus are superimposed n times to form the following ERP waveform on the electrode point of FZ.

3. Results

Two subjects were excluded due to the data missing at FZ electrode point. Therefore, the remaining 20 subjects (10 males, average age = 24.85 years, SD = 2.36) were included in the subsequent analysis.

3.1. Results of the Subjective Evaluation

Table 2 presents the average scores of each dimension of perceptual experience and the total evaluation of perceptual experience of the selected navigation interfaces. The ANOVA and post-hoc analysis showed that there were significant differences in each dimensions of perceptual experience and the total evaluation of perceptual experience among the three levels of navigation interfaces (F(2, 57) = 3670.772, p < 0.001).

3.2. ERP Results

The EEG data of three navigation interfaces of mobile games with high, medium, and low perceptual experience were superimposed and averaged, respectively. The total waveform diagram of ERP of 20 subjects at each electrode point was shown in Figure 6. According to the total waveform diagram, brain topography (Figure 7) and relevant studies, the EEG components of N1, P2, and N2, respectively, in the time window of 80~110 ms, 140~170 ms, and 220~260 ms selected the prefrontal (FP) area (FP1, FPZ, FP2), frontal (F) area (F3, FZ, F4), frontal-central (FC) area (FC3, FCZ, FC4), and central (C) area (C3, CZ, C4) for analysis. Within each time window, the mean potentials were analyzed by a two-tailed paired sample t-test. The results of paired sample t-test are shown in Table 3 and Table 4.
From the scalp topographic map, the activation area of N1 was mainly concentrated in the left-anterior scalp, the activation area of P2 was mainly concentrated in the frontal scalp, and the activation area of N2 was mainly concentrated in the anterior scalp. The N1 reaction showed asymmetry in the brain regions, and the P2 and the N2 showed symmetry. It could be seen that after 140 ms, the F and FC areas were positively activated and, after 220 ms, the anterior scalp was negatively activated. Moreover, in these two-time windows, the activation degree was obviously enhanced with the improved perceptual experience.
For N1 (80‒110 ms), the result of average amplitude showed that the N1 became more negative for the stimuli with the lower perceptual experience. The paired sample t-test showed that there were significant differences in N1 amplitude between the lower perceptual experience and the medium perceptual experience on F and FC areas (p < 0.05), and marginally significant differences on FP and C areas (p = 0.073, 0.060). In other conditions, there were no significant differences across various electrodes. For P2 (140‒170 ms), the high perceptual experience would evoke more positive P2 amplitude. The paired sample t-test showed that there were significant differences between low perceptual experience and medium perceptual experience on FP, F and FC areas (p < 0.05), and a marginally significant difference on C area (p = 0.052). There were significant differences between the low perceptual experience and the high perceptual experience on FP and F areas (p < 0.05), while a marginally significant difference in the FC area. For N2 (220‒260 ms), the high perceptual experience would evoke more positive P2 amplitude. The paired sample t-test showed that there were significant differences between high perceptual experience and low perceptual experience (p < 0.01), and between high perceptual experience and medium perceptual experience (p < 0.05) on FP, F, FC, and C areas.

4. Discussion

The experiment investigated the differences in ERP responses caused by the navigation interfaces of mobile games with different perceptual experiences under an oddball paradigm, in which the navigation interfaces of mobile games serve as non-target stimuli and landscape pictures serve as target stimuli. The simulation of gamers’ perceptual experience process at a low attention level was realized, and the processing mechanism of gamer’s visual information for stimulus was bottom-up. It could be seen from the results that there were significantly differences in the EEG components of gamers in the process of perceiving game interfaces with different perceptual experience. Specifically, navigation interfaces of mobile games with higher perceptual experience evoked high amplitudes of P2/N2 than those with the low perceptual experience. For N1, there were significant differences between low perceptual experience and medium perceptual experience on F and FC, while there were no significant differences between low perceptual experience and high perceptual experience. Therefore, it could not be fully proved that the changes in N1 were due to changes in perceptual experience.
The statistical results showed that as for the N1 amplitude, there existed significant differences between low and medium perceptual experiences, while there were no significant differences between medium and high and between low and high perceptual experiences. According to previous studies on N1, the N1 changes were caused by the activation of the early visual area of the brain due to the stimulation of basic visual characteristics. Moreover, these changes belonged to the neural activity stimulated by “sensory immersion”, mainly induced by the low-order attributes of stimulation, such as profile, shape, color, etc. Navigation interfaces of mobile games with different levels differ from background, layout, and color. This bottom-up cognitive process was mainly reflected in the attention induced by stimulation, and in the neurological response, it was mainly reflected in the difference of N1, that is, the N1 amplitude caused by the navigation interfaces of mobile games with lower perceptual experience should be more negative. The reason why the N1 amplitude did not exist significant differences induced by low-high perceptual experience may be that these two pictures both caused the user’s emotion with higher valence. The picture with low perceptual experience induced the user’s negative emotion, and the picture with high perceptual experience induced the user’s positive emotion, while the picture with medium perceptual experience induced similar emotion with picture with medium perceptual valence. Keil et al. (2001) showed that the medium stimulus could induce larger N1 amplitude compared with the positive and negative emotional stimuli, which was exactly consistent with the conclusion in this experiment [32]. In addition, although existing significant differences in the subjective evaluation of these three navigation interfaces, the interface with high perceptual experiences were composed of more elements, including rich colors, animation decorations, and superposed backgrounds, therefore, having a stronger stereoscopic impression. However, the interfaces with low and medium perceptual experiences were both composed of a simple backgrounds and monotonous colors. According to Wijers et al. (1987), the higher visual load would make N1 amplitude more negative [33]. Hoversten et al. (2011) focused on early visual processing of users and showed that early visual processing would be affected by low-priority perceptive attributes, and at low attention, high-density images would induce more negative N1 amplitude of users [34]. Therefore, it was speculated that the insignificance of N1 amplitude between high and low perceptual experiences in this experiment might be caused by the differences in the complexity of the interfaces. The interface with high perceptual experience has higher complexity, which increased the visual load of the users and resulted in a decrease in N1 amplitude. The above discussion on the N1 needs further ERP experiment for verification.
The grand-averaged ERP showed that P2 occurs in the frontal scalp about 150 ms after the stimuli were presented. The statistical results showed that P2 induced by low-medium, low-high perceptual experiences showed significant differences, while medium-high was not significant. Luck et al. (1994) showed that the P2 appearing in the anterior scalp was related to the color and shape of the stimulus, and was useful in analyzing and reflecting the physical properties of the stimulus [35]. In this experiment, the aesthetic attribute was an important aspect reflecting perceptual experience. Accordingly, the differences in P2 might be induced by the users’ perceptual differences of interface aesthetics. In addition, Carretié et al. (1994) showed that P2 was associated with arousal and was more sensitive to low-arousal stimuli [36]. This was consistent with the results concluded in this study that there were significant differences in P2 induced by low-high, low-medium perceptual experiences, while no significant difference existed in medium-high in the experiment. From the overall trend, the experimental results of P2 were consistent with previous studies, that is, navigation interfaces of mobile games with higher perceptual experience had better aesthetic experience and higher arousal of users so that it could induce larger P2 amplitude.
The paired sample t-test showed that low-high, medium-high perceptual experience induced significant differences of N2, which occurred around 220 ms after stimuli onset in the anterior scalp, while low-medium did not. N2 was the endogenous component of ERP, and N2 in the anterior scalp was related to cognitive processing and evaluation activity [37]. The significant differences in mean amplitude of N2 could reflect the differences in cognition and evaluation of navigation interfaces with the different perceptual experience. In early visual processing, the users’ neural responses to the stimuli were mainly reflected in C1 and N1 components, which could be used to reflect the differences of attention [38]. With the deepening of visual perception, the stimuli would induce higher-order cognition of the users. When navigation interfaces with different perceptual experience were cyclically presented to the subjects, the subjects would compare and analyze the interface pictures under the influence of their own knowledge, memory, etc. [39], thus inducing brain nerve response of the subjects in relevant evaluation and decision, and then potentially showing a tendency to like a certain navigation interface. Moreover, the higher-order cognitive change and neural response were reflected in a change in N2 in the anterior scalp. According to Tommaso et al. (2008), a Go-NoGo paradigm of ERP (subjects need to make choices) was applied to study the aesthetic evaluation of art paintings and geometric figures, and the experimental results showed that more beautiful figures could induce more negative N2 amplitude of users [40]. In this study, there was no task of preference selection for the navigation interfaces. Although there were differences in experimental paradigm, the similar neuropsychological response should be elicited by the stimuli regardless of whether subjects were asked to estimate the stimuli. The conclusion of N2 in the ERP experiment was consistent with the previous studies. The navigation interfaces of mobile games with higher perceptual experience attracted more attention, made users have a preference tendency, and the neural response showed a larger N2 amplitude in the anterior scalp.

5. Conclusions

In this paper, the ERP technology is applied to measure the perceptual experiences of the navigation interfaces of mobile games and explores the brain’s cognitive processes of the user to the navigation interface. The study of perceptual experience holds that the user’s preference for the products is based on the measurement of individual difference among products. The preference is a direct reflection of the user’s feelings and attitudes and affects the user’s behavior intention and purchase decision. Therefore, the research on the neural mechanism of users when contacting with game interfaces in the initial stage is helpful to improve the design of the mobile games. Moreover, it can grasp the real demands of users for the first time. Relevant research has indicated that it only takes 50 ms for users to form the first impression of the product [41], while there is lack of in-depth studies on the perception and experience of this short-term process. Therefore, the study on ERP with a more accurate measurement is helpful to better understand the formation of perceptual experience.
The ERP experiment shows that users’ initial experience is affected by the basic characteristics of the navigation interface and their cognition. There are significant differences in the subjective evaluation of perceptual experiences and EEG results. The superposition analysis of EEG data is carried out according to the subjective evaluation results. It is showed that the navigation interfaces of mobile games with a significant difference in perceptual experience could induce P2 and N2 amplitudes with a significant difference, specifically, game interfaces with higher perceptual experience levels can induce larger amplitudes of P2 in the frontal scalp and N2 in the anterior scalp. The study proves that the ERP technique can be used to measure the visual perception and the neural reflection caused by the difference of the game interface. Even at low attention, the game interface with different perceptual experience can still induce the difference of the EEG component. The future work should explore the design characteristics of game interfaces that cause the cognitive differences in users, thus providing technical support for game design and, moreover, study the interactive process between user and game, to overall enhance the user experience of mobile games.
The main limitations of this study have the following two aspects. First, the participants is college students. Future research can recruited more non-student participants to improve the generalization of the findings. Second, the experiment ignores the influence of gender on the cognition and neural response of game products. Bandura A, an American neuropsychologist, has found that the structure and activity patterns of cerebral cortex between males and females are quite different [42]. Female responses, in a visual, auditory, and tactile sense, are usually more sensitive than male, so it is necessary to deeply analyze the emotional needs of females.

Author Contributions

Conceptualization, F.G. and J.J.; methodology, J.J. and J.C.; statistical analysis, X.T. and J.C.; investigation, J.J. and X.T.; data collection, J.J. and X.T.; writing-original draft, J.J., X.T., J.C.; Writing-review and editing, J.J., X.T., J.C.

Funding

This research was supported by the National Natural Science Foundation of China (grant no. 71471033, 71771045) and the “Double First-Class” Disciplines Construction Project of Northeastern University (grant no. 02050021940101).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Su, Y.S.; Chiang, W.L.; Lee, C.T.J.; Chang, H.C. The effect of flow experience on player loyalty in mobile game application. Comput. Hum. Behav. 2016, 63, 240–248. [Google Scholar] [CrossRef]
  2. Merikivi, J.; Tuunainen, V.; Nguyen, D. What makes continued mobile gaming enjoyable? Comput. Hum. Behav. 2017, 68, 411–421. [Google Scholar] [CrossRef]
  3. Yoon, H.; Park, S.H.; Lee, K.T.; Park, J.; Dey, A.; Kim, S. A case study on iteratively assessing and enhancing wearable user interface prototypes. Symmetry 2017, 9, 114. [Google Scholar] [CrossRef]
  4. Nagai, Y.; Georgiev, G.V. The role of impressions on users’ tactile interaction with product materials: An analysis of associative concept networks. Mater. Des. 2011, 32, 291–302. [Google Scholar] [CrossRef]
  5. Clement, J.; Kristensen, T.; Grønhaug, K. Understanding consumers’ in-store visual perception: The influence of package design features on visual attention. J. Retail. Consum. Serv. 2013, 20, 234–239. [Google Scholar] [CrossRef]
  6. Ho, C.H.; Lu, Y.N. Can pupil size be measured to assess design products. Int. J. Ind. Ergon. 2014, 44, 436–441. [Google Scholar] [CrossRef]
  7. Borsci, S.; Kuljis, J.; Barnett, J.; Pecchia, L. Beyond the User Preferences: Aligning the Prototype Design to the Users’ Expectations. Hum. Factors Ergon. Manuf. 2014, 26, 16–39. [Google Scholar] [CrossRef]
  8. Beatty, S.E.; Ferrell, M.E. Impulse buying: Modeling its precursors. J. Retail. 1998, 74, 169–191. [Google Scholar] [CrossRef]
  9. Jennett, C.; Cox, A.L.; Cairns, P.; Dhoparee, S.; Epps, A.; Tijs, T.; Walton, A. Measuring and defining the experience of immersion in games. Int. J. Hum. Comput. Stud. 2008, 66, 641–661. [Google Scholar] [CrossRef]
  10. Guo, F.; Cao, Y.; Ding, Y.; Liu, W.; Zhang, X. A multimodal measurement method of users’ emotional experiences shopping online. Hum. Factors Ergon. Manuf. Serv. Ind. 2015, 25, 585–598. [Google Scholar] [CrossRef]
  11. Hazlett, R.L. Measuring emotional valence during interactive experiences: Boys at video game play. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Montréal, QC, Canada, 22–27 April 2006; pp. 1023–1026. [Google Scholar]
  12. Chen, F.; Zhou, J.; Wang, Y.; Yu, K.; Arshad, S.Z.; Khawaji, A.; Conway, D. Robust Multimodal Cognitive Load Measurement; Springer International Publishing: Cham, Switzerland, 2016. [Google Scholar]
  13. Sim, H.; Lee, W.H.; Kim, J.Y. A Study on Emotion Classification utilizing Bio-Signal (PPG, GSR, RESP). Art Cult. Game Gr. Broadcasting Dig. Contents 2015, 87, 73–77. [Google Scholar]
  14. Isbister, K.; Schaffer, N. Game Usability: Advice from the Experts for Advancing the Player Experience; CRC Press: Boca Raton, FL, USA, 2008. [Google Scholar]
  15. Ding, Y.; Guo, F.; Zhang, X.; Qu, Q.; Liu, W. Using event related potentials to identify a user’s behavioural intention aroused by product form design. Appl. Ergon. 2016, 55, 117–123. [Google Scholar] [CrossRef]
  16. Ocklenburg, S.; Peterburs, J.; Mertzen, J.; Schmitz, J.; Güntürkün, O.; Grimshaw, G. Effects of emotional valence on hemispheric asymmetries in response inhibition. Symmetry 2017, 9, 145. [Google Scholar] [CrossRef]
  17. Cona, G.; Kliegel, M.; Bisiacchi, P. Differential effects of emotional cues on components of prospective memory: An ERP study. Front. Hum. Neurosci. 2015, 9, 1–15. [Google Scholar] [CrossRef]
  18. Ding, Y.; Guo, F.; Hu, M.; Cao, Y. Using event related potentials to investigate visual aesthetic perception of product appearance. Hum. Factors Ergon. Manuf. Serv. Ind. 2017, 27, 223–232. [Google Scholar] [CrossRef]
  19. Luck, S.J.; Woodman, G.F.; Vogel, E.K. Event-related potential studies of attention. Trends Cogn. Sci. 2000, 4, 432–440. [Google Scholar] [CrossRef]
  20. Cudo, A.; Francuz, P.; Augustynowicz, P.; Stróżak, P. The effects of arousal and approach motivated positive affect on cognitive control. An ERP study. Front. Hum. Neurosci. 2018, 12, 320. [Google Scholar] [CrossRef]
  21. Recio, G.; Conrad, M.; Hansen, L.B.; Jacobs, A.M. On pleasure and thrill: The interplay between arousal and valence during visual word recognition. Brain Lang. 2014, 134, 34–43. [Google Scholar] [CrossRef] [PubMed]
  22. Skadberg, Y.X.; Kimmel, J.R. Visitors’ flow experience while browsing a Web site: Its measurement, contributing factors and consequences. Comp. Hum. Behav. 2004, 20, 403–422. [Google Scholar] [CrossRef]
  23. Creusen, M.E.; Schoormans, J.P. The different roles of product appearance in consumer choice. J. Prod. Innov. Manag. 2005, 22, 63–81. [Google Scholar] [CrossRef]
  24. Aggarwal, A.; Niezen, G.; Thimbleby, H. User experience evaluation through the brain’s electrical activity. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, Helsinki, Finland, 26–30 October 2014. [Google Scholar]
  25. Bos, D.P.O.; Duvinage, M.; Oktay, O.; Saa, J.D.; Guruler, H.; Istanbullu, A.; Van Vliet, M.; Van De Laar, B.; Poel, M.; Roijendijk, L.; et al. Looking around with your brain in a virtual world. In Proceedings of the 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB), Paris, France, 11–15 April 2011. [Google Scholar]
  26. Little, L.M.; Kluemper, D.; Nelson, D.L.; Gooty, J. Development and validation of the interpersonal emotion management scale. J. Occ. Organ. Psychol. 2012, 85, 407–420. [Google Scholar] [CrossRef]
  27. Nacke, L.; Lindley, C.A. Flow and immersion in first-person shooters: Measuring the player’s gameplay experience. In Proceedings of the 2008 Conference on Future Play: Research, Play, Share, Toronto, ON, Canada, 3–5 November 2008; pp. 81–88. [Google Scholar]
  28. Phan, M.H.; Keebler, J.R.; Chaparro, B.S. The development and validation of the game user experience satisfaction scale (GUESS). Hum. Factors 2016, 58, 1217–1247. [Google Scholar] [CrossRef]
  29. Rizzi, A.; Fogli, D.; Barricelli, B.R. A new approach to perceptual assessment of human-computer interfaces. Multimed. Tools Appl. 2017, 76, 7381–7399. [Google Scholar] [CrossRef]
  30. Field, A. Discovering Statistics Using SPSS, 3rd ed.; Sage: London, UK, 2009. [Google Scholar]
  31. Luck, S.J. An Introduction to the Event-Related Potential Technique, 2nd ed.; The MIT Press: Cambridge, MA, USA, 2014. [Google Scholar]
  32. Keil, A.; Müller, M.M.; Gruber, T.; Wienbruch, C.; Stolarova, M.; Elbert, T. Effects of emotional arousal in the cerebral hemispheres: A study of oscillatory brain activity and event-related potentials. Clin. Neurophysiol. 2001, 112, 2057–2068. [Google Scholar] [CrossRef]
  33. Wijers, A.A.; Okita, T.; Mulder, G.; Mulder, L.J.M.; Lorist, M.M.; Poiesz, R.; Scheffers, M.K. Visual search and spatial attention: ERP in focussed and divided attention conditions. Biol. Psychol. 1987, 25, 33–60. [Google Scholar] [CrossRef]
  34. Hoversten, L.J.; Brothers, T.; Swaab, T.Y.; Traxler, M.J. Early processing of orthographic language membership information in bilingual visual word recognition: Evidence from ERP. Neuropsychologia 2017, 103, 183–190. [Google Scholar] [CrossRef]
  35. Luck, S.J.; Hillyard, S.A. Electrophysiological correlates of feature analysis during visual search. Psychophysiology 1994, 31, 291–308. [Google Scholar] [CrossRef]
  36. Carretié, L.; Hinojosa, J.A.; Martín-Loeches, M.; Mercado, F.; Tapia, M. Automatic attention to emotional stimuli: Neural correlates. Hum. Brain Mapp. 2004, 22, 290–299. [Google Scholar]
  37. Folstein, J.R.; Van Petten, C. Influence of cognitive control and mismatch on the N2 component of the ERP: A review. Psychophysiology 2008, 45, 152–170. [Google Scholar] [CrossRef]
  38. Flynn, M.; Liasis, A.; Gardner, M.; Towell, T. Visual mismatch negativity to masked stimuli presented at very brief presentation rates. Exp. Brain Res. 2017, 235, 555–563. [Google Scholar] [CrossRef]
  39. Basiul, I.A.; Aia, K. Dependence N200 and P300 ERP in P300-based brain-computer interface on the variations of voluntary attention. Zhurnal Vyssheĩ Nervnoĩ Deiatelnosti Imeni I P Pavlova 2014, 64, 159–165. [Google Scholar] [PubMed]
  40. De Tommaso, M.; Pecoraro, C.; Sardaro, M.; Serpino, C.; Lancioni, G.; Livrea, P. Influence of aesthetic perception on visual event-related potentials. Consciousness Cogn. 2008, 17, 933–945. [Google Scholar] [CrossRef] [PubMed]
  41. Lindgaard, G.; Fernandes, G.; Dudek, C.; Brown, J. Attention web designers: You have 50 milliseconds to make a good first impression. Behav. Inf. Technol. 2006, 25, 115–126. [Google Scholar] [CrossRef]
  42. Bandura, A. Social cognitive theory: An agentic perspective. Asian J. Soc. Psychol. 1999, 2, 21–41. [Google Scholar] [CrossRef]
Symmetry 11 00710 g001 550
Figure 1. Six versions of the navigation interface.
Figure 1. Six versions of the navigation interface.
Symmetry 11 00710 g001
Symmetry 11 00710 g002 550
Figure 2. The experimental paradigm.
Figure 2. The experimental paradigm.
Symmetry 11 00710 g002
Symmetry 11 00710 g003 550
Figure 3. Electrodes applied in the experiment.
Figure 3. Electrodes applied in the experiment.
Symmetry 11 00710 g003
Symmetry 11 00710 g004 550
Figure 4. The procedure of ERPs collecting and processing.
Figure 4. The procedure of ERPs collecting and processing.
Symmetry 11 00710 g004
Symmetry 11 00710 g005 550
Figure 5. The sketch map of ERP epochs.
Figure 5. The sketch map of ERP epochs.
Symmetry 11 00710 g005
Symmetry 11 00710 g006 550
Figure 6. Grand averaged event-related potentials at each electrode point.
Figure 6. Grand averaged event-related potentials at each electrode point.
Symmetry 11 00710 g006
Symmetry 11 00710 g007 550
Figure 7. Scalp topographic maps for three levels in three-time windows.
Figure 7. Scalp topographic maps for three levels in three-time windows.
Symmetry 11 00710 g007
Table
Table 1. Perceptual experience questionnaire.
Table 1. Perceptual experience questionnaire.
Demographic information:
  gender; ( ) age; ( ) education background; ( ) years of playing mobile games; ( ) hours of playing mobile games per week; ( )
Emotional perception (completely disagree: 1 completely agree: 7)
  EP-1: This navigation interface makes me feel amazed.
  EP-2: This navigation interface makes me feel pleasure.
  EP-3: This navigation interface makes me feel content.
  EP-4: I am hopeful for the operation of this navigation interface.
Usability (completely disagree: 1 completely agree: 7)
  U-1: This navigation interface has a clear layout.
  U-2: This navigation interface looks easy to operate.
  U-3: This navigation interface looks friendly.
Gameplay (completely disagree: 1 completely agree: 7)
  G-1: The design elements of this navigation interface is delicate.
  G-2: This navigation interface has an integrated appearance.
  G-3: The layout of this navigation interface is innovative.
  G-4: This navigation interface is colorful.
Attractiveness (completely disagree: 1 completely agree: 7)
  A-1: Some parts of this navigation interface are novel in design.
  A-2: This navigation interface gives me a deep impressive.
  A-3: This navigation interface can attract my attention.
  A-4: This navigation interface can stimulate my interest in the operation.
The tendency for a further experience this mobile game
  (completely disagree: 1 completely agree: 7)
  T-1: I intend to experience this mobile game further.
Table
Table 2. Scores of subjective evaluation.
Table 2. Scores of subjective evaluation.
StimuliEmotional PerceptionGameplayUsabilityAttractivenessThe Tendency for Further ExperiencePerceptual Experience
Crazylily1.08.405.758.358.502.2533.25
Crazylily3.014.8512.6515.6516.204.0563.40
Crazylily6.023.7016.7523.5523.405.7093.10
Table
Table 3. Average amplitude of brain region for three conditions in three-time windows.
Table 3. Average amplitude of brain region for three conditions in three-time windows.
ERPLocationLow Perceptual ExperienceMedium Perceptual ExperienceHigh Perceptual Experience
MeanSDMeanSDMeanSD
N1FP−2.2623.098−1.4242.394−1.3702.601
F−1.6952.054−0.9412.039−1.1912.455
FC−1.5861.932−0.7562.114−1.1262.311
C−1.1521.758−0.3962.080−0.7932.051
P2FP−1.2414.9850.5594.8240.3794.450
F0.1013.4351.5273.5871.1143.286
FC0.3703.1951.5513.4271.1983.105
C0.2322.9141.0313.2260.8992.983
N2FP−6.4004.826−6.4335.154−7.9634.290
F−5.2543.410−5.2733.249−7.1443.512
FC−4.6873.051−4.7782.816−6.7193.299
C−3.9422.689−4.0762.437−5.8542.958
Table
Table 4. Results of two-tailed paired sample t-test.
Table 4. Results of two-tailed paired sample t-test.
ERPLocationLower-MediumLower-HigherMedium-Higher
tptptp
N1FP−1.8950.073−1.5510.137-0.1330.895
F−2.2170.039−0.9970.3310.6030.554
FC−2.2790.034−0.9890.3350.9470.335
C−2.0000.060−0.8570.4021.1090.281
P2FP−3.5730.002−3.2180.0050.3190.753
F−3.5330.002−2.4170.0260.8720.394
FC−2.9770.008−1.8590.0790.7940.437
C−2.0780.052−1.4400.1660.3090.760
N2FP0.0700.9453.1750.0052.7370.013
F0.0550.9575.1550.0004.2270.000
FC0.2590.7995.7240.0004.3320.000
C0.3910.7005.5960.0004.1310.001

Share and Cite

MDPI and ACS Style

Guo, F.; Jiang, J.-Y.; Tian, X.-H.; Chen, J.-H. Applying Event-Related Potentials to Measure Perceptual Experience toward the Navigation Interface of a Mobile Game for Improving the Design. Symmetry 2019, 11, 710. https://doi.org/10.3390/sym11050710

AMA Style

Guo F, Jiang J-Y, Tian X-H, Chen J-H. Applying Event-Related Potentials to Measure Perceptual Experience toward the Navigation Interface of a Mobile Game for Improving the Design. Symmetry. 2019; 11(5):710. https://doi.org/10.3390/sym11050710

Chicago/Turabian Style

Guo, Fu, Jun-Yi Jiang, Xiao-Hui Tian, and Jia-Hao Chen. 2019. "Applying Event-Related Potentials to Measure Perceptual Experience toward the Navigation Interface of a Mobile Game for Improving the Design" Symmetry 11, no. 5: 710. https://doi.org/10.3390/sym11050710

APA Style

Guo, F., Jiang, J. -Y., Tian, X. -H., & Chen, J. -H. (2019). Applying Event-Related Potentials to Measure Perceptual Experience toward the Navigation Interface of a Mobile Game for Improving the Design. Symmetry, 11(5), 710. https://doi.org/10.3390/sym11050710

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop