Measuring the Impact of Museum Architecture, Spaces and Exhibits on Virtual Visitors Using Facial Expression Analysis Software

Abstract

Since the launch of online video portals in 2005, museums have encouraged visitors to upload and share their visits online. Although much has been written about visitors’ experiences in museums, very little exists on the impact virtual visits have on viewers. In this qualitative pilot study, a total of 2035 emotional reactions were recorded and analyzed after visiting 14 online museums using a facial expression recognition software. Following open and axial coding techniques, themes and subcategories emerged. Findings showed that while the background of the participant mediated how one experiences a museum online, certain architectural and exhibit attributes, if present, triggered similar emotions to those experienced in an in-person visit. Findings suggest that experiencing museums through online video portals may be as engaging as visiting museums in person—only if the creator captures a significant proportion of architectural details, transitioning of spaces and exhibits details. Further findings showed that facial expression software reveals what captures virtual visitors’ emotions, and what architectural and exhibit features keep them curious and engaged.

1. Introduction

Museums have long been a source of education, information and recreation to the general public. In the United States, museums contribute over $50 billion to the U.S. economy [1] with 66% of leisure travelers willing to explore local art or cultural environments [2]. Therefore, it becomes important to continuously learn what keeps visitors motivated to go to museums, what makes them curious about what they see and what makes them satisfied with their visits. Since the launch of online video portals in 2005, many museums have encouraged the use of platforms such as YouTube in delivering museum experiences to online viewers in an attempt to reach larger audiences, increase awareness and loyalty, and most importantly, bring new forms of public engagement [3,4].

Studies specific to understanding museum visitors have spurred interest among curators and researchers since the 19th century, and museum professionals are always looking for new ways to meet these expectations. Phelan [5] provided an excellent overview of self-determination theory in explaining how museums meet one’s psychological needs for competence, autonomy and relatedness [6]. If these needs are met, then one may argue that visitors will be motivated to learn. To support one’s need for competence, museum professionals should make the design of their exhibits fun and easy for visitors to navigate freely [7]. Most museums offer free-choice learning, where navigation is guided by a visitor’s needs and interests [8,9]. While this is important to satisfy one’s need for autonomy, curators or exhibit designers should consider how to make use of a wide range of design options in presenting exhibits to visitors. The need for relatedness can be satisfied through visiting the museum with friends or family members, or by offering the visitor opportunities for social interaction with others [10].

There is a large body of literature that explores the role of architectural design and exhibits on visitors’ engagement and emotions using observations, surveys, focus groups or interviews. Many of these studies focus on museum layouts and the position of exhibits within galleries. Known as space syntax, scholars offered tools to identify spaces within the museum that gave exhibits higher levels of co-visibility, co-awareness and co-presence, thereby, increasing the probability of engagement [11,12,13]. Numerous other studies look at the role of narrations in creating a fertile area around exhibits and offer new dimensions to their meanings. However, these studies may be too restrictive in understanding the meaning of space [14]. In a well-cited study, Smith and Smith [15] looked at visitors coming alone and visitors coming in groups. They documented that the interaction between visitors generated a social event while experiencing an art exhibit. Others went in-depth in examining the role of architecture, especially those designed by signature architects such as Gehry, I.M. Pei or Hadid [16]. Further studies focus on the impact of adding sound to spaces and exhibits [17] or enhancing visitors’ experiences through the use of virtual or mixed reality technologies [18]. Most recently, studies show how the use of QR codes in delivering information can maximize visitors’ engagement [19].

To capture a higher resolution of data on visitors’ preferences, Rashed et al. [20] used a LIDAR pole human tracking system installed at shoulder level at a museum. This technology enabled the researchers to create heat maps representing four different visiting styles: The Ant visitor spent more time around all exhibits, the Fish visitor moved mostly around empty areas, the Grasshopper visited certain exhibits while skipping others, and the Butterfly visited most exhibits but not with an equal amount of time. Such information helps curators know which exhibits were preferred over others. It also helps curators understand how to arrange exhibits to avoid congestion.

Tseng et al. [21] used a different technology to capture visitors’ emotional reactions to exhibits. They developed an eyewear system and a microrecorder and asked six participants to wear the glasses during their visit. They were able to capture the visitors’ eye movements through wearable glasses with an attached camera and connected the eye movements to the exhibit being viewed by the visitor. Once data were extracted, participants were asked to explain the moments that caused their eyes to blink within the context of five experiences such as feel, sense, think, act and relate.

While these studies offer valuable insights to museum architectural designers and curators, the findings may only be applicable to in-person visits. To date, there has been insufficient literature that examines the behavior and motivations of visitors of museums online [4]. The purpose of this qualitative study is to analyze the impact of online videos of different museums on emotions using facial expression detection technology. The results of this study will help museum curators understand the impact different exhibits have on emotions and will help them understand what key architectural features attract and engage visitors. The results of this study will also help architects and exhibit designers understand the role their design has on users before executing the designs. Therefore, the study questions are: (1) To what extent does viewing museums via online videos impact a viewers’ emotions? (2) Can these emotions be captured accurately using facial expression detection technology? And (3) Using what is learnt from literature review, how similar are emotional reactions to in-person visits?

2. Literature Review

Museum visitors do not have infinite time to visit each exhibit. They visit museums of their free will and, therefore, there is an expectation that they will see, move, navigate and experience exhibits in their own way. This literature review is organized around three themes that address visitors’ engagement, reactions, and expectations. First, it presents a review of major design principles related to how exhibits and architectural features impact visitors’ engagement. Second, it summaries previous work that evaluated the effectiveness of these principles. Third, it outlines previous work that measured specific visitors’ emotions and how they impacted their satisfaction level with their visits.

Alt and Shaw [22] argued that it is important to classify museum exhibits in terms of their effectiveness on visitors. they generated and tested 53 attributes on participants who just visited a museum. They later examined these attributes on 45 exhibits and a total of 1980 subjects and were able to reduce the list to 23 attributes. Examples of these attributes included an overall evaluation of the exhibit such as “It presents information I’m already familiar with,” the attractiveness of the exhibit such as “Your attention is distracted from it by other displays,” an evaluation of the subject matter such as “It is a difficult subject to understand,” an emotional reaction such as “It is entertaining,” the visual effect of the exhibit such as “It looks like something of science fiction,” and the clarity of the exhibit such as “It is confusing.”

Another major contribution to museum visitor studies was Perry’s [23] development of six guiding principles based on visitors’ psychological needs. Perry found visitors expected to see objects that triggered their curiosity, increased their level of confidence in their ability to learn something new, challenged them when figuring something out, engaged their sensory experience, and encouraged their communication with others through exchanges of ideas. Perry [24] tested these principles and found that they increased visitor engagement with exhibits and helped museums better understand how to design exhibits that suit visitors’ psychological needs.

Interestingly, Forrest [25] proposed borrowing the concept of atmospherics in understanding the role of environmental psychology on visitors’ experiences. More specifically, the author borrowed the stimulus-organism-response (S-O-R) model to explain how the sensory inputs such as lighting, layout or color (stimulus), although depended on personal factors or past experiences, triggered emotional reactions such as pleasure or arousal (organism). This in turn produced behavioral outcomes such as approach or avoidance (response). This study shed further light on explaining the possibility that emotions do matter in understanding whether the visitor was curious or engaged, or would visit the exhibit again.

Building on prior works, Skydsgaard et al. [26] argued that the use of design principles may enhance visitors’ reflection and discussion. They examined four principles of design in museums developed earlier by the Steno Museum. These were curiosity (such as new knowledge or connecting new information to existing knowledge), challenge (such as exhibits that evoke emotions such as taboos), narratives (such as using narrative devices that stimulate reflection) and participation (such as physical interactivity that requires bodily interaction and dialogic interactivity where visitors share ideas with others). The research team developed a questionnaire and distributed it to pupils from four schools. Ninety-eight participants gave feedback through the questionnaires. Although their study showed that elements of curiosity and challenge were what attracted visitors the most, these four guiding principles were helpful in creating the desired experience among visitors to keep them engaged.

Museums have recently changed the way they deliver experience to visitors. Over the last decade, there has been more focus on presenting objects to visitors and a shift from product orientation to consumer orientation [27]. Therefore, it becomes reasonable to understand the motivations behind visits to the museum, what impacts visitors’ emotions, what makes them satisfied with the visit and what makes them return [28]. Visitors who expressed positive emotions from their visits were more likely to be satisfied [29]. Learning more about the specifics of these emotions could be helpful in enhancing the experience [30] since people are more influenced with the emotional values associated with objects—an area that still needs to be teased out [31]. Characterized by their intensity [32], emotions were usually formed as a response to consumption of an experience (e.g., seeing an exhibit) and may be measured by core emotions such as joy, anger, disgust and fear [33] or by bi-polar state of emotions such as pleasant/ unpleasant or calm/excited [34,35].

Due to lack of research on the role of emotions as shaped by experiencing objects or exhibits, Del Chiappa et al. [27] analyzed the behavior of 410 people who visited the National Museum of Archeology in Sardinia. The questionnaires assessed their sociodemographic data, their emotions and their overall satisfaction. Visitors also participated in focus groups to assess their emotions further. Findings confirmed that emotions were directly linked to visitors’ overall level of satisfaction with the museum. The authors argued that considering emotions in future studies examining visitors’ behavior was important to understand what increased their level of satisfaction.

Although studies around emotions were limited, other strands of museum visitor research selected to focus on specific elements that were believed to impact visitors’ emotions. One such example was integrating sound into exhibits to captivate the sensory experience of visitors. In historical museums, listening to sounds from the past to represent a certain episode was found to be more immersive as it lifted visitors out of reality and magically transported them to a different time period [17]. Working with the Danish Museum, Hjortkjær (2019) tracked what visitors remembered the most from their visit to the historical display through user surveys. Almost every fourth respondent referred to the different sound installations as the highlight of their experience. This study, however, captured one of many elements that could play a role in one’s own emotions.

Another strand of research looked at the power of social conversations on visitors’ engagement [36] and how exhibits create such interactions. Achiam et al. [14] recorded the vocalization of 12 visitors who visited two dioramas representing the ice age and a discovery room with objects that had no labels. The authors transcribed the dialogues between participants verbatim to capture their thought process as they interacted with the exhibits rather than capturing their assessments using post-hoc questionnaires. They discussed the findings within the concept of perceived affordances and how the unlabeled objects in the discovery room were successful in constructing different narratives depending on the visitors’ prior knowledge and experience.

Although numerous studies showed that visiting museums is largely social and most visitors go with friends or family members, one recent study surveyed 100 users of online platforms and semi-interviewed 10 of these users [4]. Their findings showed that viewers were motivated to visit museums through online video platforms because they were well-designed, immersed and interestingly, identified themselves with the author of the video. The educational part of the online museum visits also allowed viewers to escape reality by exploring something different and manage their moods. Although they were not accompanied by friends or family, most respondents expressed their feeling of belonging to a community who share similar interests by being able to use the comment section [4].

Although there is insufficient literature that examines the impact of visiting museums online on the viewer’s emotions, this study will test what is known from visiting museums using a facial expression analysis software.

3. Methods

In this pilot study, the authors reviewed over 100 online video tours of different museums captured by visitors. The authors qualified 14 museums because the videos were professionally taken, known for their unique architectural features, captured enough architectural details and offered high resolution close ups of the exhibits. All 14 video clips had a minimum of 13,000 views and a maximum of 2 million views between 2016 and 2020. Although the authors acknowledge that these videos were not immersive since they did not capture important details such as smell, the video clips had similar qualities recommended by Gladysheva, Verboom and Arora [4].

Other criterion included in the selection of the videos was to focus on two types of museum: art museums and dark museums. Art museums were selected because this category is not restricted to a geographical boundary. This allows the researchers to test art museums from different countries to focus on the space and the display of the exhibit. Paintings like those of Van Gogh or Monet are good examples of art pieces that are common across different art museums. Dark museums were selected because they were designed to trigger emotions.

The 14 selected museums representing art museums and dark tourism museums are as follows: the Tate Modern Museum in London, the Ulucanlar Prison Museum in Ankara, the Museum of Modern Art in New York, the Eastern State Penitentiary in Philadelphia, the Eli and Edythe Broad Art Museum in East Lansing, Alcatraz Penitentiary in San Francisco, the Ukrainian National Chernobyl Museum in Kiev, the Istanbul Art Museum in Istanbul, the Guggenheim Museum in New York, the Guggenheim Museum in Bilbao, the University of Michigan Museum of Art in Ann Arbor, the Toledo Modern Art Museum in Toledo, the Sinop Prison Museum in Sinop and the 9/11 Memorial and Museum in New York.

Table 1. Summary of museums included in this study.

Museums	Subject 1		Subject 2
	Yes	No	Yes	No
Type: Art
1. Tate Modern Museum	*			*
2. Guggenheim Museum	*		*
3. MOMA		*	*
4. Broad Art Museum	*		*
5. Istanbul Art Museum	*			*
6. Guggenheim Museum		*		*
7. UMMA	*		*
8. Toledo Museum of Art	*		*
Type: Dark Tourism
9. Ulucanlar Prison Museum	*			*
10. 9/11 Memorial and Museum		*		*
11. Ukrainian National Chornobyl		*		*
12. Sinop Prison Museum	*			*
13. Alcatraz Federal Penitentiary		*	*
14. Eastern State Penitentiary		*	*

* denotes whether the viewed museum was visited by the participant in person prior to study.

summarizes the types of these museums and indicates whether they were visited by the participants prior to the study.

After qualifying museums for this study, video clips were downloaded into an appropriate format. These short clips were then identified as stimuli inside a software called iMotions. iMotions is an advanced software that has the capability of analyzing 7 core emotions and 20 facial expression measures. Recent research tested the reliability of this software in detecting emotions by comparing its performance to other similar facial expression detection software [37,38]. In their research, Kulke et al. [38] compared the recognition software with EMG measurements of the zygomaticus mayor and corrugator superclii muscles—the muscles responsible for emotions like happiness and anger. Results showed that the values correlated highly.

The iMotions software computes different emotions based on analyzing the facial expression of the participant. When the participant views a stimulus (in this case, the video clip), the software captures the main facial features of the participant while looking at the clip through a high-resolution camera attached to the computer. The distance between the eyebrows, the distance between the nose and the upper lip, and the position of the chin are all examples of how the software computes the different emotions. If the angles of the lips are pulled up in certain ways along with other facial features because of a reaction from the zygomaticus major muscle that is responsible for the smile, then that is an indication of joy. The algorithms built into the software to register each of these emotions were computed based on scientific research on movements of facial landmarks and other types of muscle movement. The software registers these different emotions numerically and graphically (see Figure 1). The graphic output is similar to that of a heartbeat rhythm strip displayed on an ECG monitor. Each emotion has a strip of waves. Changes in facial expressions as a response to a stimulus in the video clip will cause the wave to peak (short, sharp peaks or long, curvy peaks). Within the context of museums, these peaks may correspond to an architectural space that suddenly opens to the view of the participant or an artwork with a high level of saliency (see Figure 2).

In this research, each museum clip was identified in the software as a stimulus. The investigators of this study were the two participants. The researchers purposefully used self-experimentation technique to be able to verify and explain each resulting emotion and reaction. Researchers within similar fields advocated for the use of this approach because when the experimenter becomes the subject of the study, it helps test new concepts and formulate new ideas [39]. A recent review of self-experimentation technique showed how this method had led to notable, including Nobel Prize, discoveries [40].

The decision to proceed with self-experimentation for this research was for the following reasons: First, the total viewing time requires a time commitment of about 4 h from each participant to watch and analyze each video. Second, the authors spent additional time to review the accuracy of all 2035 emotional peaks resulting from the stimuli. Third, this self-experimentation also allowed the authors to discuss, verify and code the results produced in iMotions software while following inductive and thematic analysis as proposed by Strauss and Corbin [41]. Were the emotional changes a response to seeing an architectural space? Were they the result of seeing an exhibit with certain features? Or were they the result of the subjective experience of the participant?

The software created a timeline that registered peaks of different emotions while timestamping the exact second in the video clip that created the change in emotion. The timeline for the two participants for each stimulus (video clip) was identical. This was helpful because it allowed the researcher to compare how similar or different emotional reactions were between the two participants. It was also helpful for the purpose of this research to select an existing video clip because the path remained constant for the participants as opposed to having a self-guided walk through.

In this study, there were 302 identical emotional reactions, and 1733 unique ones. The authors applied open coding techniques with thematic groupings and redundancy checks on all 2035 emotional peaks, and this was repeated until the authors agreed on conclusions [41]. The researchers continued to analyze and code the emotions until a theoretical saturation was completed. The three main themes generated were the architecture of the space, the exhibits, and the subjective background of the visitor.

After compiling the emotional reactions from watching the different stimuli, the authors created a table with rows and columns (

Table 2. Summary of findings with sample thumbnail photos representing different emotional reactions from each of the three themes.

The Architectural Space
Contempt—Surprise	Surprise—Engagement	Surprise—Engagement
9/11 Memorial and Museum	Ulucanlar Prison Museum	Guggenheim Museum
Exhibits and Elements
Surprise—Engagement	Surprise—Engagement	Surprise—Engagement
Ulucanlar Prison Museum	Toledo Museum of Art	Toledo Museum of Art
Background
Engagement—Fear	Engagement—Fear	Surprise—Engagement—Fear
Ulucanlar Prison Museum	Guggenheim Museum	Toledo Museum of Art

). Rows represent the three major categories: architectural space, exhibits, and subjective background of the visitor while the columns represent eight different emotions (anger, joy, disgust, contempt, fear, sadness, surprise and engagement). Further analysis and use of axial coding techniques was undertaken to explore connections and linkages between codes. This step enabled the researchers to identify subcategories that were eventually connected to each of the three main themes. These new subcategories represented something very specific that demonstrated a pattern across all 14 museums.

4. Results

This section demonstrates the three different categories discussed in the preceding section. Further findings were grouped into six subcategories representing architectural space features, five subcategories representing exhibit features and six subcategories representing how a viewer’s background could influence their experience.

4.1. The Architectural Space

When examining the results of facial expressions from watching video clips from all 14 museums, changes in different types of emotions were registered in the software when the following six conditions emerged. These findings were consistent across all 14 museums.

4.1.1. Long or Narrow Spaces

First, when experiencing spaces that were long or narrow, areas that felt claustrophobic, or spaces with an irregular height:width ratio, a consistent pattern of surprise, sadness, fear and engagement were observed. Examples of these spaces were noted when viewing the long narrow corridors of the Ulucanlar Prison Museum and the Guggenheim Bilbao Museum, the low ceiling height at the Sinop Prison Museum, and the narrow individual cells at Eastern Penitentiary Museums (see Figure 3).

4.1.2. Visitation Areas

Certain emotional reactions are expected in spaces such as the visitation area at a prison museum type. Nonetheless these emotions may be reinforced with the presence of original wire mesh that separates visitors from inmates. It is likely that the appearance of the rough and cold material played a significant role in reinforcing its intended effect. While visitation areas exist only in prison museums, the fact that both sides of the space can be seen at the same time but cannot be accessed had a negative effect on the human being. In this study, the software registered feelings of sadness, surprise and engagement (see Figure 4).

4.1.3. Space Conditions and Building Remains

In this study, the authors noted that one of the most successful ways to impact visitors and communicate a message effectively was to preserve and showcase the original remains of an exhibit. Adding context that simulates the original space will likely have a stronger impact. Corroded walls, signs of rust, and the presence of rough finishes on the surfaces of hallways and interior walls are good examples that triggered feelings of sadness, fear, anger, surprise and engagement. These emotions were consistently observed while viewing the Sinop Prison Museum, the Eastern Penitentiary Museum, the Ulucanlar and the Alcatraz Museum (see Figure 5).

These types of surface had a consistent effect on participants, but it is worth noting that emotions were amplified at the 9/11 Memorial. There, steel columns that were preserved in their original damaged state were referenced using an exhibit label that showed a map connecting the visitor’s location to the actual location of the bombing site. In this case, the memorial site was laid underneath the original site, and thus, triggered mixed emotions of sadness, surprise, and engagement (see Figure 6 and Figure 7).

4.1.4. Architectural and Spatial Features

Kaplan and Kaplan [42] made various references to the impact of spatial mystery on human cognition, exploration and curiosity. Within the context of museums, this theory holds true when viewing the curved walkway of the Guggenheim Museum in New York, where the feelings of both surprise and engagement were triggered (see Figure 8). The spiral ramp hugging a large well-lit atrium remained one of the strongest architectural features to have within a museum. Another example from this category was the impact of the slanted walls of the Broad Art Museum on perceived levels of curiosity that also triggered feelings of surprise and engagement (see Figure 9).

Museums with larger building footprints were found to be challenging. How would one maintain the same level of interest throughout the galleries? Despite its scale, at the Tate Modern in London, the floor-to-ceiling windows positioned between and around the perimeter of galleries gave a sense of orientation to the viewer and a quick reference to the exterior grounds while keeping the viewer engaged—in our case it also triggered feelings of surprise and engagement (see Figure 10). Other good examples were represented in viewing the original columns at the 9/11 Memorial. Such architectural features played a role in magnifying the emotions of viewers. Further analysis of emotions showed that architectural and spatial features that have a high impact on emotions were generally triggered when viewing slanted walls, columns, ramps, atriums and perimeter windows overlooking the exterior grounds. The scale of the element compared to the space, the materials used in the architecture of the space, the colors of the walls, transparency, and even angles can all magnify different emotions like surprise, engagement and in some instances, joy or sadness.

4.1.5. Simulation of Events

Consistent with literature, simulating events from the past through sound or narration was also found to have a powerful impact on emotions. For example, seeing handprints on a wall at the Tate Museum triggered emotions of engagement and surprise as the participants recalled seeing the exhibit during an in-person visit prior to this research. Additionally, the same types of emotion were also triggered when viewing the lighting effect that represented radiation at the Chernobyl Museum. Since Chernobyl was a well-known event, simulating the incident through means like architectural lighting was successful in capturing one’s emotions.

4.1.6. Architectural Lighting

Architectural lighting was the most significant contributor to orchestrating one’s emotions. In larger museums such as the Tate, transitioning from a darker gallery to a lit gallery that has skylights or other forms of architectural light will keep visitors surprised and engaged. For smaller museums such as the Broad Art Museum, the use of simple architectural lighting to complement the natural light coming through the large, slanted windows triggered the same emotions of surprise and engagement. Architectural lighting emphasizes exhibits in the space and transmits the theme to the viewer (see Figure 11). In dark tourism museums, however, it was found that architectural lighting played a minimum role in triggering the same types of emotion.

4.2. Exhibits and Elements

In this category, the authors grouped different exhibits and elements from different museums with shared attributes into five categories listed below.

4.2.1. Exhibits Narrating Events

A mix of emotions such as fear, sadness, surprise and engagement were all exemplified around exhibits with different narration styles. For example, an exhibit using mannequins sitting on inmates’ bunk beds in an old rusty room (see Figure 12), an exhibit displaying original newspapers around those who were executed at the Ulucanlar Museum (see Figure 13), and an exhibit reading confession statements at the Eastern Penitentiary Museum all produced mixed emotions of fear, surprise, sadness and engagement. Exhibits narrating events were more noticeable in dark museums, especially those connected to a well-known event, as they tend to increase the desire to be informed about the event and therefore trigger curiosity.

4.2.2. Sound

Consistent with literature, exhibits that play with our senses can be powerful. Using sound to narrate stories and news or to read poems, such as is the case at the Sinop Prison Museum, had quite an impact on keeping visitors engaged and surprised. There, poems that were written by a renowned novelist and a poet who had served in the same prison spoke about living within the prison and captured prisoners’ feelings. All three prisons visited during this study also staged the events well by preserving the rough finishes of its cells. At Sinop Prison Museum, observing the talking mannequins and listening to voices coming unexpectedly from different directions certainly kept the participants continuously engaged. In this study, sound triggered feelings of sadness and surprises across all dark museums. Unlike other elements, it was observed that sound prolonged engagement with exhibits.

4.2.3. Belongings

Another form of narration that triggered different emotions was exhibiting personal belongings. For example, viewing a white coat that once belonged to a physician or a baby’s crib at the Chernobyl Museum kept participants engaged and surprised. Similarly, the belongings of prisoners exhibited at Ulucanlar, Eastern Penitentiary and Alcatraz had an identical impact on participants’ emotions (see Figure 14 and Figure 15). These personal belongings range in scale, such as old spice cologne at Ulucanlar, a musical instrument at Alcatraz (see Figure 16), or an old typewriter at the 9/11 Memorial. These observations were expected because when people see personal belongings they immediately connect with the people they once belonged to, or even sympathize with them.

4.2.4. Original Exhibit

Well-preserved exhibits displayed in their original condition were found to be important in maintaining a high level of engagement and surprise. Depending on the context, these exhibits may also trigger other types of emotion. Viewing children with birth defects at Chernobyl Museum, seeing burnt elevator cables at the 9/11 Memorial (see Figure 17), or experiencing fully furnished cells with triple bunk beds at the Sinop Prison Museum were all examples of exhibits that triggered multiple emotions such as fear, sadness, surprise and engagement.

4.2.5. Artwork

Curating artwork might be the most challenging task at a museum. What attracts visitors to exhibits and what leads them to skip certain exhibits has been the core of research studies in museums. In this study, a prolonged sense of surprise and engagement was observed at exhibits with certain characteristics. First, exhibits that were large in scale triggered the same emotions. Examples include large 3D sculptures at the Istanbul Modern Museum and at the Broad Art Museum, a large pot at the Guggenheim New York, large canvas paintings at Tate Modern Museum, and large floating exhibits at the Istanbul Modern Museum. Second, exhibits with shiny reflective surfaces were also found to have an impact. Examples ranged from displaying jewelry to using mirrors or reflective surfaces on art pieces. Third, the color drop behind an exhibit should be carefully selected. In most of the museums visited, exhibits were intentionally set up against white surfaces to create contrast.

4.3. Background

The researchers found that certain elements triggered different emotions for participants because of prior life experiences. For example, the impact of viewing exhibits that demonstrated the execution method on inmates was subjective according to the viewer’s pre-existing notions and beliefs. Phobia triggers were also noted in places with specific characteristics. Also, establishing personal connection to belongings or historical events was dependent on the personal experiences of the participant. Generally speaking, events triggered through sensory stimuli, and memory recall were all examples that were subjective according to the viewer’s background. Since life experiences vary, participants’ reactions also varied during the experiment. The researchers grouped their findings into the following themes.

4.3.1. Methods of Execution in Prison

The gallows exhibit at the Ulucanlar Prison Museum impacted the feelings of participants differently. For one participant, it triggered fear and sadness, and upon discussing the reason behind these feelings, it became clear that this was dependent on the participant’s experience with historical events (see Figure 18).

4.3.2. Phobia Triggers

Phobias can emerge due to stressful events. Therefore, what triggers phobias depends on the viewer. For example, when one participant viewed the bugs exhibit at the Eastern Penitentiary, it triggered feelings of fear and discomfort. Another reacted when the author of the video clip moved through claustrophobic spaces at the Tate Modern Museum.

4.3.3. Establishing Personal Connection to Belongings

Because sculptures and paintings painted black at the Toledo Art Museum represented slavery, they were found to have a significant impact on the emotions of surprise and engagement. A different example from this category was seeing the popular Old Spice perfume exhibit at the Ulucanlar Museum. Although it had once belonged to a prisoner, the emotional reaction was connected to prior experiences with the same brand, which established a personal connection to this form of exhibit.

4.3.4. Establishing Personal Connection to Events

Similarly, connecting to events is also dependent on one’s own experiences. To elaborate, seeing prison cells with thick walls located on a single loaded corridor at the Sinop Prison Museum where communication among inmates is not possible was compared with a prior visit of Alcatraz Penitentiary (see Figure 19) where most prison cells had thinner walls. At Alcatraz, these cells were located along a double loaded corridor where communication among inmates facing each other was possible. Other examples of exhibits where emotions of surprise and engagement were triggered included seeing the Syrian 3D ship exhibit at the Istanbul Modern Museum, the magnitude radiation map of the Chernobyl site at the Ukrainian National Chernobyl Museum, and the pop art exhibits of the Campbell Soup at MOMA.

4.3.5. Events Triggers through Sensory Stimuli

In this study, sensory stimuli were also subjective to the experiences of the viewer. Exhibits that produced feelings of fear, engagement and sadness included listening to birds outside Alcatraz (sound), seeing black-red paintings, and looking at two exhibits with contrasting textures (wood vs. steel) at the Guggenheim Bilbao.

4.3.6. Memory Call

Even though visitors return to museums, successful exhibits may still have a similar impact on emotions. For example, feelings of engagement and surprise were still triggered in the same manner when virtually viewing the deformed, worn-off staircase at the Sinop Prison Museum, which referenced the number of people who walked up these stairs over the years.

5. Discussion

The aim of this study was to explore whether viewing museum spaces via online video portals would affect virtual visitors’ experiences the same way it would in person. Using a facial expression detection software, it was possible to expand on the review of literature and elaborate on how spaces and exhibits triggered human emotions in museums. Literature on visitors’ studies demonstrated strong links between visitors’ engagement and exhibits that used sound [17]; narration [26]; sensory stimulation like light and color [25]; attractiveness or complexity [22]; and curiosity and challenge [24]. Results of this pilot study showed that these concepts were also applicable to digital museums. Moreover, the findings extended our knowledge on how the architecture of space had more impact on one’s own feelings of sadness, surprise and engagement while the exhibits impacted other emotions, such as fear and joy. Anger was shown to be triggered based on one’s own background. Not surprisingly, dark museums had a higher impact on emotions like anger and sadness.

When exploring the impact of space on the participant’s experience, new themes emerged that enabled the researchers to create an inventory of different architectural elements that were most impactful. To summarize, long and narrow spaces, curved hallways, and complex spatial layouts represented spaces that triggered a sense of fear, perhaps due to their claustrophobic effects. Second, in prison museums, the visitation space had an equal impact on emotions. Giving attention to such spaces may offer opportunities of higher engagement. Third, it was an expected finding that architectural lighting played a significant role in shaping one’s emotions. Dark spaces evoked mystery and fear, while bright spaces gave further emphasis to the elements being exhibited. Curators may benefit from integrating architectural lighting around certain exhibits. If the layout of the museum is complex, then lighting may serve as a landmark to help visitors navigate and distinguish spaces or galleries with different themed exhibits. Visitors can easily be overwhelmed with the complexity of the museum layout and the number and positions of exhibits within the space. If the museum is housed in an existing space, such as in the case of a prison museum, a careful play of lighting might mitigate those experiences. However, maintaining the integrity of the original finishes of the spaces will enhance one’s experience. In sequential galleries, moving from one gallery space to another might make one nervous or curious about what they will see next. Therefore, the width of the opening, the careful positioning of exhibits within sequential gallery spaces and the use of contrasting wall colors will all keep one curious, engaged, and in some instances, full of joy.

The findings also recognize that every museum type has unique exhibits. This pilot study showed that the display of newspapers demonstrating historical events had a significant impact on emotions. Consistent with the findings of Hjortkjær [17], amplifying spaces with sound can also intensify these emotions. Another instance was displaying one’s belongings from the past. Whether it was old furniture or personal clothes, these types of exhibits always pique interest. In the prison type museum, fully furnished prison cells with inmates’ belongings had more impact on emotions than empty ones. This finding was not surprising, and a simple image search about these museums on Google, TripAdvisor or Instagram will show how most photos shared by visitors are about belongings. Exhibiting specific objects from the past had a powerful way of narrating those events and reaching out to visitors. When it comes to the exhibit itself, whether it is an artwork or a 3D exhibit, its scale, color, texture and ratio with the space were all important factors in drawing visitors into the space. Museum visitors always connect what they have seen to their past experiences. Therefore, each visitor’s experience and interpretation of the exhibition will be subjective.

6. Conclusions and Limitations

In conclusion, the findings validated the use of iMotions Software in capturing the correct emotions within the context of museums. After coding all the emotional peaks, the researchers were able to understand the qualities of the architectural spaces and exhibits that may have the most significant impact on a virtual visitor’s experience. The results of this study can help create impactful museum videos and help architects, interior designers, curators, and exhibition designers understand the qualities that keep museum visitors engaged and curious. Additionally, the use of this software makes it possible for architects to test computer generated designs of buildings and exhibits prior to execution. Using facial expression recognition software may also give instant feedback to curators before putting up an exhibit.

This study has some limitations. The authors acknowledge that this study was self-experimentation that was based on the investigators’ observations. However, because timing was not an issue, it did allow the authors to qualitatively analyze all the emotional reactions following Strauss and Corbin’s approach in inductive and thematic analysis [41]. The second limitation was not being able to capture senses like smell and touch through post-video analysis. The authors acknowledge that visiting museums online will never replace the experience of visiting museums in person. The quality of the colors of the artwork is also dependent on the technical specifications of the viewer’s screens. As Mintz (1998) puts it in her article Media and Museums: A Museum Perspective: “…On a video screen, a painting has no texture; something is always lost when three dimensions are collapsed into two. Distinctions of scale disappear; a tiny Renoir and a gigantic David are the same size. The electronic palette cannot perfectly match colors in the real world [43].” Additionally, since most visitors visit museums with family and friends, viewing museums through recorded videos limit such interactions to texts via the comments section. Nevertheless, opportunities for future studies could be extended to include the impact of social interaction during virtual visits. Lastly, the authors also acknowledge that pre-recorded videos do not extend the same autonomy that museum visitors have during their free exploration [8,9]. Since many museums are shifting their attention to integrate technologies that enhance visitors’ experiences, technologies like virtual reality or eye-tracking systems can easily address these limitations and can easily be analyzed using facial expression software to further support their studies.