A Study on the Factors Influencing Willingness to Use Virtual Reality Systems for External Evaluation of Buildings

Abstract

Integrating new visualization methods based on virtual reality (VR) in the design evaluation process remains a challenge despite its expanding use in architectural design in recent years. This article proposes a VR-based evaluation model to improve the efficiency and quality of the overall architectural design process. By adopting a structural equation model in conjunction with the Technology Acceptance Model (TAM), the study examines users’ intention and perceived ease of use of VR in the design evaluation process based on a questionnaire survey using the proposed VR model of architectural and non-architectural professionals and students. The findings of the study show that the output quality plays a significant role in increasing behavioral intention to use the system, and perceived ease of use has a positive effect on perceived immersion and perceived usefulness of VR in the evaluation process. The study illustrates the importance of understanding “user perspective” and “willingness to use” in the development of VR-based systems to increase their practical use in architecture and design fields.

1. Introduction

Research Background and Purpose

In recent years, with the development of ICT technology, virtual reality (VR) technology has been widely used in various fields and eminently in the field of architecture for various purposes, such as the presentation and evaluation of architectural designs. The use of VR technology has improved the quality and efficiency of design projects by providing a three-dimensional perspective to understand architectural drawings. Since the tools and methods used for design evaluation may affect the results, the use of VR technology in viewing architectural design proposals on a real scale has increased to obtain better design evaluation results.

Thus, VR evaluation methods are widely used today for the evaluation of building exteriors. In the design process, to evaluate design alternatives for building exteriors using multiple facade elements, a large number of architectural models have to be made based on different combinations of exterior facade elements. This process requires a significant amount of time and effort to create the models and choose a proper visualization method. However, even after various models are prepared, it may be difficult for the evaluator to differentiate as there are a large number of models involved in the evaluation process. Therefore, there is a need to improve the design evaluation process by using new visualization methods based on VR technology to evaluate buildings efficiently. To address this gap, this article proposes a new VR evaluation system by combining the advantages of BIM and VR technology to optimize evaluation and feedback methods in the building design process. By designing and developing the new VR evaluation system, an intuitive, comprehensive, and realistic evaluation of the architectural design idea is realized, thus improving the overall efficiency and quality of design projects. Also, since cognitive research related to the use of VR by architectural designers and the rate of acceptance of new technologies by users is yet to be explored, further research is needed on the effective utilization of VR technologies in architectural design and evaluation. Thus, the study illustrates an in-depth understanding of attitudes and behavioral willingness of architectural professionals to apply VR technology in design and evaluation processes to meet the current challenges of design visualization by using VR technology and promote advanced digitization in the field of architectural design.

Thus, the study illustrates an in-depth understanding of attitudes and behavioral willingness of architectural professionals to apply VR technology in design and evaluation processes to meet the current challenges of design visualization by using VR technology and promote advanced digitization in the field of architectural design. A VR-based exterior design evaluation model was developed for this study and used to examine the user experience of such new visualization methods. Following the use of the developed system, a questionnaire survey of the participants (architecture and non-architecture professionals and students) was conducted to evaluate the proposed model. The questionnaire survey responses were examined using the Technology Acceptance Model (TAM) in conjunction with structural equation modeling. Following the survey, post hoc interviews were conducted to gain an in-depth understanding of the users’ experience while using the VR system. The interview responses were further analyzed to understand the positive and negative implications of the proposed VR model and suggestions for improvement in the future.

2. Literature Review

2.1. Building Design Process and Spatial Cognitive Ability

Spatial cognitive ability is an individual’s ability to perceive, understand, and apply the structure, relationships, and organization of space in the physical environment. This includes the ability to perceive and understand elements of space such as shape, size, orientation, and location, as well as the ability to perceive interrelationships, connections, and separations between spaces, and likewise, the ability to encode, represent, store, recognize, combine, decompose, generalize, and abstract objects or spatial patterns in the human mind [1,2,3].

In architecture, spatial cognitive ability refers to one’s ability to perceive, understand, and express architectural space. This includes the perception and understanding of the form, structure, function, and aesthetic characteristics of architectural space, as well as the ability to design and create an architectural space [4,5]. Thus, architectural professionals possess spatial cognitive skills for creating a three-dimensional mental image while working on a two-dimensional layout [6]. However, communicating their ideas using such traditional representation techniques to clients and other professionals involved in the construction process often results in cognitive discrepancies, as such individuals lack the ability to transfer design drawings to a three-dimensional depiction [6]. In recent years, with the advancement of visualization techniques, architects are adopting new technologies such as virtual reality (VR) in the design process to reduce cognitive discrepancies and to enhance the level of understanding of architectural design ideas by upgrading traditional visual media in architectural education [6,7,8,9].

The use of VR has overwhelmingly increased in various fields ranging from arts and humanities to clinical research over the past decades [10,11]. Specifically, the potential of VR in improving learning experiences in diverse fields of education has been realized as enhancement of cognitive skills through such devices can help in gaining in-depth understanding and retainment of spatial and visual information [10,12,13]. In the field of architecture, the use of VR as a visualization technique in architectural practice and education has also increased to overcome the limitations of traditional media, such as CAD and 3D models, which cause difficulties in experiencing scale, context, and functionality [14,15]. Prior studies show that VR can improve communication between stakeholders and effective collaboration and communication in the early design stages [15]. However, little attention has been paid to the use of VR in human interaction with spaces and visualization of buildings using a virtual environment for understanding and assessment of design proposals [16].

Therefore, in order to understand the effectiveness of new visualization methods, a preliminary study was conducted prior to this research in 2023 on user preference analysis using different visual media such as photographs, videos, and VR for architectural design assessment by different groups, including undergraduate and graduate level architecture students and practicing architects. In this study, the three visual media were compared based on the evaluation scales of joy, reality, convenience, and usability. The results of the study showed that 82.7% of the users preferred using VR as a visual media for design evaluation, as more than 84% of the participants considered VR as the most realistic visual media. Regarding the reason for choosing VR, 48.1% expressed that the system was easy to operate, and 42.3% considered it to be the most convenient, which suggests that VR has a better sense of realism compared to the other two visual media but needs to be improved in terms of both operability and convenience [17]. Based on the level of spatial cognitive abilities of users, cognitive discrepancies can be reduced by replacing visual media with VR during the architectural design process. Also, the preliminary study results show that the level of effectiveness of the three media effectiveness during architectural evaluation, VR, video, and photographs, was in descending order, hence proving that VR can reduce spatial cognition discrepancies compared to traditional visual media. The findings of this research were limited to the examination of different visualization mediums for evaluating design proposals by architectural students and professionals and did not take laypersons’ perspectives into account.

Previous studies have indicated significant differences in the aesthetic evaluation of building facades by architectural and non-architectural professionals depending on their age, sex, educational level, and mood [18,19]. It has been noted that such differences in opinions are related to the different ways of visual conceptualization of building design as architects tend to contemplate aesthetic standards. Therefore, in order to improve layman’s understanding of design proposals and to expand their visual immersion for the conceptualization of buildings on par with architects, VR has been proven to be an effective visualization medium. However, there is a need to examine the usability of VR in design evaluation by conducting a survey based on a large participant set, including persons related to the architectural field and laypersons [20].

2.2. Virtual Reality Technology in Architecture

Architects have long utilized drawings, models, and visualization tools for communicating design ideas to clients and other designers. In recent decades, technology has provided architects with a variety of tools such as computer-aided drafting, photo-realistic rendering, and, currently, virtual reality. Realizing these technological advances, architects have increasingly started using a combination of traditional representational techniques such as hand-drawing, sketching, and model building and new technologies in practice to produce designs at various scales and for adopting different ideas through their projects. As a result, the evolution of digital tools has increased the complexity of the visual representation of ideas in both building and iterative design phases. For example, the introduction of advanced computational tools has enabled architects and designers to virtually generate complex forms, toggle between scales and views, and work at a faster pace to tangibly represent ideas on paper with unprecedented ease. However, these computer-aided design tools currently lack the ability to contribute to a deeper understanding of the user’s spatial experience that goes beyond the formal, functional, and structural aspects of architecture. The core of architectural practice revolves around the iteration and visualization of designed spaces and communication within the discipline [21].

Despite the overwhelming research on the use of VR in architecture from different perspectives,

Table 1. Use of VR in previous architectural studies.

Author	Contents	VR Software	Use of BIM
Lee (2018) [22]	Improvement in user understanding, participation and interest in design following VR experience data	Unreal Engine	No
Lim (2019) [24]	Using VR technology to improve smart city plans, allowing citizens to use VR to evaluate city scenes	Unreal Engine	No
Lee (2020) [25]	Evaluation of technology developed using VR in architectural design process	Unity	Yes
Park (2021) [26]	Observe and compare building exteriors	Unity	No
Muhammad (2022) [23]	Using different methods to search for design layers to verify the effectiveness of VR in evaluating buildings	Unity	No

shows a list of studies that have used VR for design evaluation in existing buildings. Lee (2018) conducted a survey to determine public design preferences using VR. Since the VR experience included multiple scene changes, the evaluation was conducted separately at the end of the VR experience, reducing the immersion of the participants during the evaluation phase [22]. Muhammad (2022) tested methods of assessing individual building designs and analyzed the differences in the assessment methods based on time spent by the users to detect design errors while exploring the virtual environment. The study also validates the effectiveness of using VR for building assessment [23]. Lim (2019) utilized VR technology to present smart city scenarios where participants could select and evaluate a desired cityscape using a VR system. The study also uses VR environments created by utilizing drone photogrammetry to create virtual urban environments close to reality [24].

Park (2021) illustrates an evaluation system based on VR using typical characteristics of existing evaluation systems. The paper verifies the advantages and possibilities of VR using an example of residential burglary by comparing images and VR as two experimental tools [26]. The results of the study show the advantages of VR in recording participants’ decisions based on specific judgments in a clear manner in comparison to other VR experimental research, considering various environmental conditions. This confirms the possibility of VR as a new experimental tool, especially in the field of environmental psychology. Mao (2020) improves the reliability of design evaluation using VR by creating a virtual interface in which VR users are able to walk freely and realistically observe the virtual building while measuring and designing the dimensions of the space based on their own scale [27].

However, the development and promotion of VR technology in the field of architecture is at a slower pace. Therefore, with the increasing use of VR technology in architecture, there is a need to examine the relationship between visual perception and behavioral willingness of the professionals working in the architectural design field using virtual reality technology to develop and use innovative VR evaluation systems for architectural design.

2.3. Technology Acceptance Model (TAM)

The Technology Acceptance Model (TAM) is currently the most mature and robust theoretical model in the field of information systems applications. It is derived from the theory of rational behavior and was first proposed by Fred D. Davis of the University of Michigan in 1989, with the initial purpose of explaining the determinants of the widespread acceptance of computers. With the continuous development of the TAM theory, it has been widely applied to the study of users’ use and acceptance of new technologies. Users’ use of information technology ultimately depends on users’ intention to use it, which in turn is influenced by users’ cognitive beliefs. Venkatesh and Davis (2008) refined the TAM theory in their further research on technology acceptance and proposed the Technology Acceptance Model Extended (TAM3), which integrates the role of social influences, the influence of cognitive tools, and other relevant elements, using moderating variables such as “voluntariness” and “experience” to analyze factors such as user’s acceptance and use of technology [28]. TAM3 is based on external and individual factors, cognitive beliefs, and intention to use, that determine the users’ system use, cognition, and intention to use the system, respectively [29]. This theory does not specifically refer to a particular type of information technology/system but rather refers to all technologies or systems that are characterized by information processing or transmission.

The VR experimental system proposed in this study is undoubtedly also an information system. Therefore, this study examines the proposed system theoretically based on the Technology Acceptance Model 3 (TAM3), which is widely applicable as an explanatory tool for understanding user behavior and usage status of various information systems. Using this model, the study explores factors influencing users’ willingness and intention to use VR evaluation systems in architecture.

3. Research Model and Questionnaire Design

This study explores the attitudes and behavioral intentions of architectural professionals toward the application of VR technology for design and evaluation. The study examines the technological and behavioral willingness of architectural design personnel to-ward the virtual reality system developed based on the Technology Acceptance Model (TAM) using a Revit model of the Kim Chung-up Museum of Architecture in Anyang, South Korea. The study attempts to construct a virtual reality-based Technology Acceptance Model from the perspective of architects and designers in the field of architectural design and validate the obtained data through structural equation modeling to analyze the TAM model from a theoretical perspective.

3.1. Hypotheses and Research Model

In TAM3, based on the development model of TAM, a large number of studies on external and individual factors related to cognitive factors proved that intention to use has a direct impact on the actual use of the technology system by users. The stronger the user’s intention to use a technological system, the greater the preference to use it. Therefore, this study explores users’ usage behavior by focusing on factors influencing users’ intention to use VR systems. Using TAM3 with the causal chain framework proposed by Abdelsalam (2020) [29], the variables in this study were divided into “Antecedents Variables”, “Mediator Variables”, “Moderators Variables”, and “Outcome Variables” (refer to Figure 1).

Based on the review of the TAM models (TAM, TAM2, and TAM3), perceived ease of use and perceived usefulness were identified as principal variables for predicting the intention to use information technology. Therefore, the TAM model in this study is also based on the modified TAM model proposed by Davis (1989), which employs the key variables of perceived usefulness, perceived ease of use, and behavioral intention, along with variables such as “Perceived Immersion”, “Output quality”, and “System Characteristics” identified through literature review [30]. Additionally, ”Usage experience” was included in the basic information section of the questionnaire survey conducted for this study [28,29,30].

3.1.1. Antecedent Variables

(1)

Output Quality

Generally, the output quality of VR mainly depends on the hardware related to picture quality. When the output quality of the VR system is higher, the user can be better immersed in the virtual environment and thus easily understand the contents of the system. In this case, users will be able to make better choices based on their own understanding. Therefore, the improvement of output quality directly affects users’ perceived usefulness and perceived immersion. Therefore, this study proposes the following hypotheses:

H1: 

Output quality of VR systems has a significant positive effect on perceived usefulness.

H2: 

Output quality of VR systems has a significant positive effect on perceived immersion.

(2)

System Characteristics

System characteristics (SC) refers to a user’s self-perception and self-judgment of the system capabilities (hardware and software) when using a VR system to accomplish a particular task. After the TAM model was proposed, many researchers attempted to introduce variables, such as system characteristics, into the overall model to explain the influence of system capabilities on the acceptance and use of information systems in different domains. Accordingly, a strong relationship between system characteristics and the perceived ease of use of a system has been proved. System characteristics are a key factor in the use of a VR system, and the users’ perception of their own actions when performing tasks through the VR system, which affects their perceived usefulness and perceived ease of use of the technical system, as well as whether or not the user accepts and uses the VR system. Therefore, this study proposes the following second set of hypothesis statements:

H3: 

System characteristics of VR systems have a significant positive effect on perceived usefulness.

H4: 

System characteristics of VR systems have a significant positive effect on perceived ease of use.

H5: 

System characteristics of VR systems have a significant positive effect on perceived immersion.

3.1.2. Mediator Variables

(1)

Perceived Usefulness

Perceived usefulness is the degree to which users perceive the effectiveness of the VR system in helping them accomplish their tasks and the users’ cognitive judgment that the users’ use of the VR system can improve their productivity. It is considered to be an important component of the Technology Architecture Model (TAM) [31]. In TAM theory, perceived usefulness serves as a connecting component between external variables and intention to use in the overall structure and can largely influence users’ intention to use the system [32]. On a similar basis, this study argues that the degree to which users’ perception of the VR system improves their judgment of the building facade can influence users’ intention to use the VR system. Therefore, the study proposes the following hypothesis:

H6: 

Perceived usefulness of VR systems has a significant positive effect on behavioral intention.

(2)

Perceived Ease of Use

Perceived ease of use refers to how easy users find it to use the system. It is another central variable in technology acceptance modeling. Numerous studies have shown that perceived ease of use is a key predictor after perceived usefulness [33,34]. In general, if a user perceives a system as easy to use during interaction with characteristic information and information systems, the user’s cognitive load will be reduced, and thus, the willingness to use the system increases. Since human information processing is limited, there are chances of resistance and abandonment due to the complexity of learning and using complex systems, making the task impossible to accomplish. Consequently, the easier the system is to use, the more effort the user can put into the task itself, and when the VR system is easier to use, the more the user can immerse themselves in completing the task in the VR environment. Therefore, this paper proposes the following set of hypotheses:

H7: 

Perceived ease of use of VR systems has a significant positive effect on behavioral intention.

H8: 

Perceived ease of use of VR systems has a significant positive effect on perceived usefulness.

H9: 

Perceived ease of use of VR systems has a significant positive effect on perceived immersion.

(3)

Perceived Immersion

Perceived immersion refers to the state in which the user is completely immersed in the system and forgets about the real world while performing a task using a VR system and is followed by pleasure. The state of immersion is characterized by concentration and pleasure. When the user is viewing and evaluating the building using the VR system, the composition of the levels in the system and the task of selecting the elements inside will contribute to the immersion, and in this state, the user can fully concentrate on the building evaluation task. The pleasant feeling brought by the VR system will also enable the user to complete the task in a better way. In general, the more immersed the user can be in completing the task in the VR system, the more efficiently the task will be completed. Based on such affirmations, the study included the following hypotheses:

H10: 

Perceived immersion of VR systems has a significant positive effect on behavioral intention.

H11: 

Perceived immersion of VR systems has a significant positive effect on perceived usefulness.

3.1.3. Moderator Variable

Usage Experience and Major Studied

Usage experience and major studied (educational background) were chosen as moderator variables. Usage experience refers to the user’s experience with the VR system or VR environment, while major studied refers to the user’s educational background to understand the differences between users from different majors. Usage experience tends to have an impact on each variable [28,34,35]. Each time a user uses VR, the usage of the system helps in familiarizing them with the characteristics and interaction methods of the VR environment. As the users become more familiar with the VR environment, they gradually accumulate the skills of using VR systems, unconsciously exercising their learning ability. There are many studies on VR that take experience as an important influencing factor. Users who have used VR many times are better able to familiarize themselves with the VR environment and use the VR system to complete tasks. On the contrary, for users who have no experience in using VR, it takes time to use the novelty brought by the VR environment, thus affecting the efficiency of task completion. In addition, it has also been shown in related studies that significant differences in cognitive and behavioral styles of individual users from different professions can occur in performing tasks related to the architectural profession. Therefore, to investigate the influence of different VR usage experiences and different educational backgrounds users on users’ intention to use VR systems, the following hypotheses are proposed:

H12: 

VR Experience plays a moderating influence in the mechanisms influencing users’ intention to use VR systems.

H13: 

Major studied plays a moderating influence in the mechanism influencing users’ intention to use a VR system.

3.1.4. Outcome Variable

Behavioral Intention

In the TAM model, intention to use is the degree of the user’s willingness to use this technology or system to accomplish a specific task and, is directly influenced by the individual’s perception of the technology and is the most direct determinant of the behavior intention of use. Intention to use is defined in this study as whether the user will start or continue to use the VR system proposed in this study in the future. Users have a clear purpose for using the VR system for building evaluation tasks, and the intention to use the VR system will likewise have a direct impact on their actual behavior. In this study, “behavioral intention to use” was identified as the outcome variable to consider the mechanism of action between other variables and users’ intention to use the proposed system.

Based on the characteristics of the VR system, perception theory, and immersion theory, the content quality, system characteristics and perceived immersion were added to this model to revise the Technology Acceptance Model (TAM) for drawing the research model for this study (refer to Figure 2).

3.2. VR Evaluation Model

Using Revit, a 3D model of the Kim Chung-up Architecture Museum in Anyang, Korea, was built and imported to Unreal Engine 5, a virtual environment building software, using the Datasmith plugin. Changeable exterior elements such as roof, window, and entrance types and materials for walls and doors were categorized and coded in terms of various categories, and the model was provided with a user interface (UI) with virtual settings including location, weather, time, and sun direction. In this study, since there might be first-time users of VR equipment among the survey respondents, “Teleportation”, which is comfortable to use and has a low degree of operating difficulty, was chosen as the mobility method for this system after comprehensive analysis and consideration. Figure 3 shows the VR environment scenes from the proposed VR evaluation model developed for this study.

3.3. Questionnaire Survey Design

3.3.1. Questionnaire Design

This study mainly relies on the questionnaire survey to collect, interpret, and analyze the data related to user experience. In order to ensure the rationality and scientific precision of the questionnaire survey, the relevant literature was reviewed to draw appropriate measurement scales related to TAM and integrated with the objectives of this study. The main part of the questionnaire survey designed in this study is divided into respondent characteristics and system-related questions (refer to Figure 4).

(1)

Respondent Characteristics

This section of the survey includes gender, age, occupation, field of study, and experience in using VR, used to perform descriptive statistical analysis. Experience in using VR was used as a moderating variable to test the effects between the main variables in the research model.

(2)

Evaluation Parameters

This section of the questionnaire survey includes individual test items reflecting the main variables in the research model shown in Figure 2. Each question was tested based on “Likert’s seven-point scale”, with the seven levels being strongly disagree, disagree, somewhat disagree, average, somewhat agree, agree, and strongly agree, corresponding to a score between one and seven.

According to the requirements of structural equation analysis chosen for the analysis of data obtained from the survey [36], a minimum of three questions are required for each variable that needs to be tested in the model to ensure the validity of the questionnaire survey. Therefore, in the questionnaire survey form for this study, a minimum of three question items were designed for each variable. A total of 20 question items were included in this questionnaire (refer to

Table 2. Questions framed for each variable selected in this study.

Variant	Number	Question
Perceived Ease of Use, PEOU	PEOU1	I can easily use this VR system for architectural evaluations.
	PEOU2	I can easily accomplish the tasks that appear in this VR system.
	PEOU3	I was able to quickly adapt to using this VR system for building evaluations.
Perceived Usefulness, PU	PU1	By using this VR system for architectural evaluations, I can increase my understanding of design.
	PU2	By using this VR system for architectural evaluation, I can learn more about the decorative elements of the design.
	PU3	By using this VR system for architectural evaluation, I can choose my favorite combination of elements.
Perceived Immersion, PI	PI1	When I use this VR system, I feel completely immersed in the VR environment.
	PI2	When I use this VR system, I feel immersed in the architectural evaluation.
	PI3	When I use this VR system, I feel that the building evaluation is more interesting.
Output Quality, OQ	OQ1	I think the quality of the graphics on this VR system is great.
	OQ2	I think the graphics on this VR system have good connectivity.
	OQ3	I believe that each stage of this VR system contributes to the evaluation.
System Characteristic, SC	SC1	When using a VR system, it feels like the objects on the screen are the same as in the real world.
	SC2	With the VR system, I can see the building in 360°.
	SC3	When using a VR system, I can interact with scenes within the VR system.
	SC4	When using the VR system, I was pleased with the ability to move freely around the scene.
	SC5	When using the VR system, I was pleased with the ability to change the weather.
Behavioral Intention, BI	BI 1	I will continue to use VR equipment for architectural evaluations in the future.
	BI 2	I would recommend the VR service to others.
	BI 3	I believe that this VR system has the potential to be sustained.

). These question items were selected based on their reliability and validity, assessed through the review of related studies, and were modified based on the VR system proposed in this study.

3.3.2. Selection of Research Subjects

Although a large amount of research on TAM has been conducted, the majority of the studies use either online questionnaires or web pages to obtain data. In this research, questionnaires and post hoc interviews of the users were conducted after experiencing the VR evaluation system. The sample size was determined from the ratio of the number of parameters (q) and the question items (N) (sample estimate (N/q)) in the model. According to Tinsley (1987), the ratio of the number of questions in the survey questionnaire to the number of samples should be 1:5 [17]. On a similar basis, at least five cases/observations per free parameter (N:q = 1:5) in the TAM model have been supported by many researchers [37,38,39]. Based on such assumptions, the sample size in this study was determined based on the number of indicator variables for each latent variable/factor. For models with three to four indicators per factor, the sample size was assumed to be at least 100 [40,41]. The survey participants included architectural designers, architectural students, and non-architectural professionals. Based on the sample size considerations, more than 100 responses were needed to obtain reliable results, and therefore, 105 participants were chosen for the study. The three sample groups included 35 architectural students, 35 architectural designers, and 35 commoners (architectural non-professionals), respectively.

3.3.3. Questionnaire Survey

The questionnaire survey was reviewed by the “Pusan National University Institutional Review Board Committee” on 20 June 2023. Following the initial review on 22 August 2023, the survey was revised based on the suggestions of the committee. The official questionnaire survey was organized in the first week of September 2023, spanning nearly two months. The survey was conducted offline as the user was required to experience the developed VR system before filling out the questionnaire survey form (see Figure 5 and Figure 6). On completion of the questionnaire survey, a total of 105 responses were obtained, and since all of them were filled offline, the data obtained were checked for validity in order to examine any omissions or extreme responses.

3.3.4. Data Analysis Tools

Following the questionnaire survey, the obtained data were analyzed using descriptive statistical analysis and reliability and validity analysis in Statistical Package for the Social Sciences SPSS 25.0. Following basic analysis of the data, validation factor analysis, measurement model fitness analysis, and analysis of paths and clusters of the model were carried out by the Structural Equation Modeling Software AMOS 28.0.

4. Data Analysis

4.1. Descriptive Statistical Analysis

As mentioned in the previous section, the questionnaire survey included 35 respondents for each participant group, including architecture students, non-architecture students, and architectural firm staff, accounting for a total of 105 responses. The basic data of the 105 samples collected from the questionnaire survey were interpreted, as shown in

Table 3. Descriptive statistics on basic data of the survey participants.

Variable	Type	Number	Percentage
Gender	Male	45	42.9%
	Female	60	57.1%
Age group	Below 19 years	2	1.9%
	20~29 years old	78	74.3%
	30~39 years old	21	20.0%
	40~49 years old	3	2.9%
	Above 50 years	1	1.0%
Profession	Office	35	33.3%
	Student	70	66.7%
Major	Architecture	70	66.7%
	Non-architectural	35	33.3%
VR use (no. of times)	Not at all	33	31.4%
	1~3 times	55	52.4%
	4~6 times	9	8.6%
	More than 7 times	8	7.6%

, using SPSS 25.0.

Despite the ratio of male to female respondents in this questionnaire survey being 4:6 (as shown in Table 3), gender had no effect on the subsequent analysis of the data. A total of 74.3% of the respondents were aged between 20 and 29 years, and 20% were aged 30–39 years, and it can be seen that the overall respondents were predominantly young (refer to Table 3). The question item “major studied” was considered as the sample included architectural and non-architectural professionals and students. The ratio of “architectural” to “non-architectural” among the three sample groups was 2:1. In terms of the question item of “VR experience”, 70% of the respondents had VR experience prior to the survey (refer to Table 3). Therefore, “major studied” and “VR experience” data show that the respondents of this questionnaire survey have different backgrounds, which meets the needs of the study. Also, the mean value of each question item shown in Table 3 is above 3, indicating that the respondents of the questionnaire survey have more than average skill in recognizing the system developed in this study.

4.2. Reliability and Validity Analysis

4.2.1. Reliability Analysis

Reliability analysis was conducted to examine whether the collected data were valid and reliable, using repeated measurements to obtain the degree of consistency of the results. For studies that measure items on a Likert’s five-point scale, Cronbach’s alpha coefficient is usually used as a reliability indicator for analyzing the reliability of various behavioral and opinion questionnaire surveys (scales). Therefore, this study will use Cronbach’s alpha coefficient as a measure of questionnaire reliability to analyze the reliability of the questionnaire survey data using SPSS 25.0 [42]. As shown in

Table 4. Descriptive statistics on basic data of the survey participants.

Variable	Number	Cronbach’s Alpha	Overall Cronbach’s Alpha
Perceived Ease of Use	3	0.880	0.873
Perceived Usefulness	3	0.869
Perceived Immersion	3	0.862
Output Quality	3	0.886
System Characteristics	5	0.890
Behavioral Intention	3	0.895

, the overall Cronbach’s alpha coefficient is 0.873, which lies within the degree of reliable data (0.80–0.89 is a good reliability level). The Cronbach’s Alpha coefficients of the subscales are all higher than 0.8, which indicates that the overall reliability of the questionnaire data in the current study is high and predicts good internal consistency.

In order to identify discrepancies in the question items, the total relevance of the question items was evaluated using the “Corrected Item-Total Correlation (CITC)” analysis, generally used to test whether the data reliability can be improved by deleting a certain value. If the value is less than 0.3, the corresponding item is usually excluded from the analysis. The analysis results showed that the CTIC values of all the items in the scale were greater than 0.3, indicating that deletion of any of the 20 items does not significantly improve the reliability of the scale and that there is a high degree of internal consistency in the variable items (refer to

Table 5. Questions framed for each variable selected in this study.

Variant	Number	CTIC	Cronbach’s Alpha	Overall Cronbach’s Alpha
Perceived Ease of Use, PEOU	PEOU1	0.390	0.870	0.88
	PEOU2	0.449	0.868
	PEOU3	0.300	0.872
Perceived Usefulness, PU	PU1	0.525	0.865	0.869
	PU2	0.530	0.865
	PU3	0.583	0.863
Perceived Immersion, PI	PI1	0.515	0.865	0.862
	PI2	0.478	0.867
	PI3	0.517	0.865
Output Quality, OQ	OQ1	0.461	0.868	0.886
	OQ2	0.479	0.867
	OQ3	0.502	0.866
System Characteristic, SC	SC1	0.468	0.868	0.89
	SC2	0.480	0.867
	SC3	0.447	0.868
	SC4	0.507	0.866
	SC5	0.505	0.866
Behavioral Intention, BI	BI 1	0.444	0.868	0.895
	BI 2	0.476	0.867
	BI 3	0.416	0.869

).

4.2.2. Validity Analysis

Validity analysis in this study refers to the extent to which the test method is used to explain the validity of certain psychological or action characteristics and responses of the participants and is described in terms of the degree of validity and accuracy of the measurement results [41,42] To test the validity of the research model and explore the correlations between the dimensions of the variables, as suggested in previous studies, exploratory factor analysis was conducted. First, to check whether the sample data meets the requirements of exploratory factor analysis, Bartlett’s test and Kaiser–Meyer–Olkin (KMO) test were measured (

Table 6. Bartlett’s test and KMO test results.

Kaiser–Meyer–Olkin Measure of Sampling Adequacy		0.768
Bartlett’s Test of Sphericity	Approx. Chi-Square	1309
	df	190.000
	Sig.	0.000

). If the chi-square distribution of Bartlett’s test reaches a significant level, it indicates that there is a relationship between the variables and the data suitable for factor analysis. The analysis results of the data in this study subjected to Bartlett’s test and KMO test show that the approximate chi-square reached 771.316 with p < 0.001 and the KMO statistic value greater than 0.75, indicating that the data were well in line with the requirements for conducting the subsequent factor analysis.

Through exploratory factor analysis (EFA), the data finally converged following seven iterations and yielded a total of six valid factors (eigenvalues > 1) with cumulative variance amounting to 78.940% (refer to

Table 7. Exploratory factor analysis (EFA) results.

Number	System Characteristic	Behavioral Intention	Output Quality	Perceived Ease of Use	Perceived Immersion	Perceived Usefulness
PEOU1				0.852
PEOU2				0.857
PEOU3				0.896
PU1						0.892
PU2						0.769
PU3						0.810
FE1					0.852
FE2					0.841
FE3					0.844
OQ1			0.886
OQ2			0.900
OQ3			0.797
SC1	0.859
SC2	0.792
SC3	0.851
SC4	0.832
SC5	0.773
BI1		0.856
BI2		0.860
BI3		0.911
Percentage	17.663%	30.327%	42.791%	55.121%	67.146%	78.940%

). The six factors extracted were the same as the six dimensions considered for the research model that coincided during the initial conceptualization of this study.

4.2.3. Structural Equation Model

A structural equation model (SEM) is a statistical technique and validation method used to test the applicability of a theoretical model or a hypothetical model. SEM integrates the two statistical tools of factor analysis and path analysis, through which the researcher can simultaneously test the explicit variables and latent variables as well as the error terms in the test model, which is an effective means for multivariate data analysis. A measurement model describes the relationships between indicators and their underlying variables, measured or conceptualized through the indicators. Before testing the structural model, the measurement model should be tested first to prove its validity and assure that the reliability and validity of the measurement model are well tested.

(1)

Goodness-of-Fit in Confirmatory Factor Analysis

In this study, AMOS 28.0 software was used to construct a theoretical model of ’users’ intention to use the VR evaluation system for building facades. The reliability and validity of the theoretical model were verified by maximum likelihood estimation (MLE) in confirmatory factor analysis (CFA). Based on the results of goodness of fit of the model as shown in

Table 8. Model fit indices criterion and estimates.

Fitting Indexes	Threshold		Estimate	Interpretation
	Excellent	Acceptable
CMIN/DF	1–3	3–5	1.257	Excellent
RMSEA, Root Mean Square Error of Approximation	<0.05	<0.08	0.05	Excellent
GFI, Goodness-of-Fit Index	>0.90	>0.80	0.855	Acceptable
IFI, incremental fit index	>0.90	>0.80	0.968	Excellent
TLI, Tucker–Lewis Coefficient	>0.90	>0.80	0.96	Excellent
PNFI, Parsimony-adjusted NFI	>0.50		0.703	Excellent

, it can be seen that CMIN/DF is 1.581 (range of 1 to 3), RMR is 0.072, (interval < 0.08), and RMSEA is0.075, (good range of <0.08), indicating that the model has an excellent fit. The additional GFI, IFI, and TLI reached good levels, having a threshold above 0.8, which is an acceptable or excellent threshold range. Therefore, the results of this analysis indicate that the CFA of the data obtained from the questionnaire survey is a good fit.

(2)

Confirmatory Factor Analysis

Under the precondition that the scale CFA model has a good fit, further evaluation of the proposed measurement model was carried out to provide in-depth data analysis in terms of reliability, convergent validity, and discriminant validity of the variables.

Reliability Evaluation

Reliability evaluations typically include assessments of the item reliability of the observed variables as well as the combined reliability of the latent variables. Composite reliability (CR) is also one of the criteria for evaluating the intrinsic quality of a research model, which is mainly used to assess the internal consistency of the observations corresponding to each latent variable, in which the larger the value of the composite reliability of a variable, the higher the internal consistency of its corresponding observation [42]. If the composite reliability values for variables are 0.9 or above, the data are “excellent”, 0.8 or above is “good”, 0.8 is “very good”, 0.7 is “moderate”, and 0.5 is acceptable in terms of reliability.

The results of the analysis show that the factor loadings of each observed variable in the model are as low as 0.746 and as high as 0.924 (as seen in

Table 9. Reliability analysis for CFA.

Variant	Number	Significance	Estimate	Composite Reliability; CR	Average Variance Extracted (AVE)
Perceived Ease of Use, PEOU	PEOU1	***	0.873	0.880	0.710
	PEOU2	***	0.846
	PEOU3		0.808
Perceived Usefulness, PU	PU1	***	0.867	0.874	0.699
	PU2	***	0.783
	PU3		0.855
Perceived Immersion, PI	PI1		0.837	0.864	0.679
	PI2	***	0.784
	PI3	***	0.850
Output Quality, OQ	OQ1	***	0.858	0.890	0.731
	OQ2	***	0.924
	OQ3		0.776
System Characteristics, SC	SC1	***	0.826	0.876	0.622
	SC2	***	0.755
	SC3	***	0.813
	SC4	***	0.801
	SC5		0.746
Behavioral Intention, BI	BI 1		0.873	0.896	0.743
	BI 2	***	0.836
	BI 3	***	0.876

Note: *** stands for p < 0.001, p-value stands for significance; and the reliability coefficient is the square of the factor loadings (i.e., item reliability).

). All the factor loadings are in the range of 0.5 to 0.95, which is in accordance with the standard requirements of item reliability. Also, the composite reliability of each latent variable of the measurement model should be higher than the lowest value of 0.864. The values in Table 8 indicate that all the values of the composite reliability are higher than 0.8, which represents that each indicator has a good contribution to the model, and the internal consistency of the measurement model is recognized.

Through the above analysis, it can be seen that the reliability of the measurement model corresponding to the theoretical model constructed in this study to verify the ‘Behavioral intention to use the building facade VR evaluation system’ meets the parameter evaluation criteria and the model has a certain degree of consistency.

Convergent Validity

Convergent validity refers to the ability to measure the same indicator that falls on the same common factor constructs and shows a medium to high correlation between the measured variables and is assessed using average variance extracted (AVE). The larger the AVE, the greater the ability of the indicator variable to reflect the characteristics of its underlying factors, in which case the aggregation validity of the model will be greater. The factor loadings of the observed variables of the measurement model proposed in this study reached a significance level of p < 0.001 (refer to Table 9). In addition, the average variance extracted (AVE) of each latent variable was higher than the evaluation criteria of 0.5, with 0.622 being the lowest. Based on the above analysis, it has been proved that the factor loadings and the average variance extracted from each variable in the model basically reached the evaluation level, and the measurement model has good convergent validity and meets the research needs.

Discriminant Validity

Discriminant validity (DV) is defined as the presence of a low correlation or significant difference between any two potential variables. In order to explore the model discriminant validity, the correlation coefficients of the variables in the measurement model and the square root of the AVE of each variable were calculated based on the results of the exploratory factor analysis (refer to

Table 10. Exploratory factor analysis (EFA) results.

Construct	BI	PU	PEOU	SC	OQ	PI
BI	0.743
PU	0.499	0.699
PEOU	0.146	0.405	0.710
SC	0.142	0.224	0.126	0.622
Q	0.342	0.303	0.377	0.221	0.731
PI	0.196	0.440	0.282	0.360	0.294	0.679
√AVE	0.862	0.836	0.843	0.789	0.855	0.824

). According to the analysis results in the next table, the number of standardized relationships between the variables is less than the square root of the AVE values corresponding to the variables in the discriminant validity test, therefore indicating that there is good discriminant validity between the dimensions.

Through the above tests and evaluations, the reliability, convergent validity, and discriminant validity of the measurement model meet the requirements of the study. Also, the measurement model of “Behavioral intention to use the building facade VR evaluation system” used in this study has good reliability and validity, and it is suitable for the next step of structural equation modeling based on this model.

(3)

Research Hypotheses Testing

Through the previous section, based on the goodness of fit test of the proposed model, all the variables have met the criteria for a good fit model. Thus, AMOS software was used to conduct a path analysis and multi-group analysis of the structural equation model proposed in the study to analyze the proposed hypotheses and establish a logical relationship between the variables.

Path Analysis

From

Table 11. Path analysis results.

Path	Initial Sample	Sample Mean	Standard Deviation	p	Hypothesis	Results
OQ → PU	0.203	0.200	0.123	0.098	H1	False
OQ → PI	0.359	0.421	0.086	***	H2	True
SC → PU	0.038	0.058	0.117	0.745	H3	False
SC → PEOU	0.389	0.418	0.076	***	H4	True
SC → PI	0.544	0.548	0.085	***	H5	True
PU → BI	0.433	0.456	0.119	***	H6	True
PEOU → BI	0.402	0.405	0.111	***	H7	True
PEOU → PU	0.490	0.478	0.131	***	H8	True
PEOU → PI	0.090	0.090	0.090	0.314	H9	False
PI → BI	0.176	0.155	0.157	0.261	H10	False
PI → PU	0.478	0.460	0.152	0.002	H11	True

*** indicates that the p-value is less than 0.001 and is highly statistically significant.

, it can be seen that the path coefficients of the four hypotheses, H1, H3, H9, and H10, have a p-value of more than 0.05 and are not significant, which means that the four research hypotheses are not valid. The rest of the hypotheses (H2, H4, H5, H6, H7, H8, and H11) were found to be significant, thus indicating their validity. The path analysis of the final structural equation model was used to obtain the structural model plot of the standardized estimates, as shown in Figure 7.

In path analysis, the relationship between variables was explored through “Direct Effects” and “Indirect Effects”. Direct effects refer to the direct influence of the dependent variable on the outcome variable, measured by the path coefficient between the two. Indirect effects are those where the dependent variable has an indirect effect on the outcome variable by influencing one or more mediating variables. Direct and indirect effects can co-exist, and the total impact of direct and indirect effects indicates the “Total Effects” of the dependent variable on the outcome variable. The value of the direct effect is the path coefficient that corresponds to the causal relationship, while the value of the indirect effect is the sum of the products of the path coefficients of each of the indirect effects, and the value of the total effect is the value of the direct effect with the indirect effect (refer to

Table 12. Multi-variate effects.

	Direct Effects					Indirect Effects Behavioral Intention	Total Effects Behavioral Intention
		PU	PEOU	PI	BI
Mediator variables	PU	-	-	-	0.433	-	0.433
	PEOU	0.490	-	0.090	0.402	0.018	0.42
	PI	0.478	-	-	0.176	0.207	0.383
Antecedent variables	OQ	0.203	-	0.359	-	0.149	0.149
	SC	0.038	0.389	0.544	-	0.271	0.271

).

Through path analysis, it was found that four hypotheses were found to be invalid due to two reasons. Firstly, the hypothesis that the output quality and system characteristics of VR systems will affect the perceived usefulness of users was rejected. When formulating the hypotheses, it was assumed that the overall characteristics of the VR system have a significant effect on the level of user immersion, thereby enabling the user to better understand the design of the system. Similarly, the analysis results showed that the overall characteristics of the system do indeed affect perceived immersion and indirectly affect perceived usefulness through perceived immersion. Secondly, it was assumed that perceived ease of use has a significant impact on perceived immersion and perceived immersion on the behavioral intention to use. The survey results proved that immersion in the system would increase if the VR system were easy to operate, which in turn would lead to an increase in the user’s intention to use. However, based on the data obtained, it was found that only the overall characteristics of the system affect the immersion of the system and have little to do with the difficulty of use. Also, perceived immersion does not have an impact on usage intention and was proved to contradict the hypothesis, indicating that although perceived immersion is important, it can only improve perceived usefulness and indirectly affect behavioral intention to use.

Multi-Group Analysis

In TAM3, “moderating variables” that would influence the model were proposed, and “experience of VR use” and “major studied” were set as moderating variables in this study. When the moderating variables are continuous variables, the significance of the path coefficients on the independent variables and the moderating variables on the dependent variable are used to determine whether the moderating effect is significant or not. However, when the moderating variable is a categorical variable, the interaction is not applicable, in which case it is necessary to use multi-group analysis, so the two hypotheses, H12 and H13, were tested using multi-group structural equation analysis (refer to

Table 13. Multi-group structural equation analysis results.

Path		Results		Results
System characteristics → Perceived ease of use	H12 VR Experience	False	H13 Major	False
Output quality → Perceived immersion		True		True
System characteristics → Perceived immersion		False		False
Perceived ease of use → Perceived immersion		True		False
Output quality → Perceived usefulness		False		False
System characteristics → Perceived usefulness		False		False
Perceived ease of use → Perceived usefulness		True		True
Perceived immersion → Perceived usefulness		True		True
Perceived immersion → Behavioral intention		False		False
Perceived ease of use → Behavioral intention		False		False
Perceived usefulness → Behavioral intention		False		False

).

Based on the estimates in Table 13, it has been proven that the H12 (number of VR uses) and H13 (major studied) moderated some of the paths in the proposed model (see Figure 8).

Based on the results of the above path analysis and multi-variate analysis, the research model was revised to produce the final simple model pathway coefficient diagram (refer to Figure 5). In terms of the influence of external variables on the mediating variables, the resultant quality has a significant positive influence on the perceived immersion of VR users. The system characteristics also have a significant positive influence on the perceived immersion and perceived ease of use of VR users. Regarding the effects between the mediating variables, perceived immersion has a significant positive effect on VR users’ perceived usefulness. Meanwhile, perceived ease of use also has a significant positive effect on VR users’ perceived usefulness. Regarding the relationship between mediator variables and outcome variables, perceived ease of use has a significant positive effect on VR behavioral intention. Similarly, perceived ease of use has a significant positive effect on VR behavioral intention.

4.3. Analysis of Post Hoc Interviews

4.3.1. Additional Interview Questions for Respondents

The previous section was used to analyze the data obtained through the users’ responses to the questions prepared based on the structural equation modeling. By doing so, behavioral intention to use of the respondents was obtained through a comprehensive questioning of each respondent specific to the developed system. Additionally, follow-up interviews were conducted to understand the different perceptions and shortcomings of the system from the respondents belonging to different professions and occupations about the system presented in this study.

Questions were asked in a semi-structured format during the interviews. For consistency, some of the questions were asked to all three groups of respondents, while separate questions were asked to respondents according to their major (educational background) and occupation. The questions were asked in the following manner:

• In the questionnaire XXX you have given a low score for XXX. May I ask why? (All three groups).
• If any government authority invites you to evaluate design proposals for building renovation as an architect, do you think this system is useful for such purposes? (Architect/Architecture student).
• If you were approached by the government to evaluate building renovations as a citizen’s representative, would you find this system useful as a commoner when evaluating buildings? (Non-architecture student).
• What other disadvantages or shortcomings do you see in this VR evaluation system? (All three groups).

4.3.2. Respondent’s Response to the Interview

• Responses received on low scores of two question items

The first interview question was about the “low scoring” on a few questions, analyzed in two parts. The first part is based on the low score in terms of “system picture quality”. The picture quality was identified as a problem since the hardware used for this study was a bit insufficient to create a realistic image, as expressed by a few of the respondents:

“… Although the sense of volume and form of the building can be well appreciated through VR, for decorative elements like materials and colors, a better picture is needed. Now, maybe because the computer’s graphics card is not very good, it is a bit lacking in terms of picture quality, which leads to a bigger gap between the virtual world and the real world…” (Architecture student 9)

“… Through the previous videos I saw or VR games I experienced I felt that VR was still very average, so I didn’t have high expectations for the effects of VR environments, and the system in this experience, although the picture quality wasn’t particularly excellent, did exceed my expectations…” (Architecture student 10)

“… I feel that it’s mainly because the device’s pixels are a bit insufficient, causing the scene’s picture quality to be a bit low, and after switching the external materials it feels like just the color changes, the specific material changes aren’t very obvious. Since I don’t have a lot of architectural knowledge, I can’t understand the specific material just by the color, and I can only see the specific texture when I look closer, so the scene is still very different from the real world.” (Non-architecture student 16)

“… After using this VR system, I think it might be because the graphics card and the monitor the VR was a bit lacking, causing the virtual environment to still be very different from the real one. Also, despite the use of teleportation method to fix the screen tearing, there are some differences that cause a difference from the real world since teleportation is not the way people move everyday…” (Architect 2)

“… Personally, I feel that it is still related to the equipment, because the equipment is not enough to show the so it leads to the average picture quality. Since our firm is also utilizing VR for the presentation of architectural projects nowadays, we are exposed to VR more often, but it should be due to the lack of performance of the graphics card resulting in a large gap with the real world…” (Architect 14)

In conclusion, in terms of problems related to the picture quality, the respondents generally believe the issue was related to the quality of the hardware used. In the case of the process used for system design, to ensure that the system can run smoothly, real-time rendering was canceled to avoid screen tearing and, at the same time, increase the use of interactive components in the scene. However, due to the performance of the graphics card and HMD, it still caused the respondents to give low scores to the system’s image quality. In future research, the performance and quality of the graphics card, if increased, the HMD device can be replaced and upgraded to a headset device with better resolution and performance.

The second part that scored lower was the question of “method of movement”. Regarding the mode of movement, most of the respondents expressed that the method of operation required time to learn. Few of the respondents conveyed that the mode of transmission was not very adaptable.

“… because of being less familiar with VR equipment it takes longer to adapt to the equipment and system, increasing the difficulty level. At the same time, because it is the first time to use VR, due to lack of familiarity with the operation of VR, the sense of immersion was lower. Immersion can increase later if I am familiar with VR and can use it in a proper way…” (non-architecture student 7)

“… Because of the teleportation type of movement used in the system, it causes the distance of teleportation to not be bit out of control and create a feeling of being in the same position even after changing, so it would be better if it was replaced with a movement method like a person walking…” (Architecture Student 7)

“It was a little difficult to operate the VR equipment because it was the first time I was exposed to it. Also, because the usual exposure to software and equipment is operated mainly with a mouse and keyboard, it was a bit uncomfortable to utilize a joystick in the system…” (Architecture student 10)

“… Since I am young, I can adapt, but there are many clients who are senior citizens, and it may be a little difficult for such individuals to view using a VR device…” (Architect designer 2)

“… Because now the VR equipment is not particularly popular, in this experience the operation was well explained and after the use felt relatively easy. But for non-professionals to get the VR equipment and relying on their individual ability of use, it is still very difficult…” (Architect 9)

“… In this system using the teleportation method to move around as a frequent VR user felt good because the screen would be cut off if it’s like the way a person normally walks, so the teleportation method is very good. But the decision to give a medium score was based on considering the unfamiliarity of the teleportation method for the average person…” (Architect 14)

In the question related to the movement method, most of the respondents expressed that it was not convenient to utilize the teleportation method as it is not similar to the regular movement of human beings. In the preliminary stage of system construction, the experimenter simulated both “walking movement” and “teleportation”, and based on the examination of the results, when using the “walking movement” method, the screen tearing was very strong, resulting in the sense of dizziness. There are two reasons for this phenomenon. One is because the rendering of the environment in the system is completed in real-time, so when “walking and moving” is used, the screen is rendered. Due to hardware problems, a tearing sensation is generated on the screen, and the other reason is due to “motion sickness”.

Considering the respondents’ opinions, in future research, replacement with a high-performance graphics card can ensure that real-time rendering will not result in screen tearing. Also, the use of “VR Walking Platform” equipment can simulate the process of walking and minimize motion sickness.

2.

Responses received on usefulness of the system for other purposes

The second question in the interview was, “Is it useful (in the future) when evaluating building remodeling as a representative of architects”. All of the respondents in the interviews said that the system developed in this study would be useful in the architectural evaluation stage but would not be possible to use it on a large-scale project at this stage. The main reasons for the identified limitation were, firstly, the high cost of investment required for VR in comparison to the benefits of using it for presentations. Also, if such a system were to be used, it would be necessary to purchase high-performance computers and VR equipment in the office and hire VR-operating staff to set up the system. Secondly, if the age of the client is above 50 years, the use of VR equipment may cause greater discomfort and reduce the client’s impression of the design, as stated in the interviews.

“… Yes, it is useful. Professional architects may have a similar experience when they view a building using photos and when they view a building with VR, because they are professionals able to visualize the building through photos, but for the average person, they can visualize easily with VR because due to lack of professional experience. In the system we experienced this time, it is easy and quick to change the elements of the building exterior with the help of the UI, but in the case of a general system, it is necessary to view multiple building models and then evaluate them, which requires a lot of time and leads to a significant increase in the time spent on viewing the building. Now in this system only 5 architectural elements were added, but in the real project there are a large number of architectural elements, in that case, this system can be time saving and useful…” (Architecture student 5)

“… Being an architecture student I can say that the system has somewhat of an advantage but it still doesn’t make a very big difference in visualization. But for people in general, being able to make architecture more realistic in a three-dimensional perspective is very useful for evaluation…” (Architecture student 8)

“… It is true that it will be useful, and I think that it will be useful if used in the evaluation process, especially to make the common people understand the style of the building more quickly. However, at this stage, the effect obtained by using VR is relatively small compared to the proportion of investment, and if it is used in the office, it may be necessary to hire VR-related personnel, which will increase the investment even more…” (Architect 2)

“… It would be useful to improve the performance of the equipment. It is really useful for explaining to the client, and when there are multiple plans to show, it is more convenient and understandable to use this system than the general method. Nowadays, in architectural design, it is more intuitive to use BIM to connect drawings to 3D models as opposed to CAD, but if VR equipment could be added to the mix, it would make the entire architectural design and evaluation phase easier and faster…” (Architect 9)

3.

Responses received on the usefulness of the system for a commoner in evaluating building design

The third question was about “usefulness when evaluating building remodeling asa citizen representative”.

“… I think this system is very useful for a commoner to make architectural evaluations. It can help an average person to save the time of imagining, and show the future appearance of the building in front of their eyes…” (Non-architecture Student 16)

“… It is very useful, using this system, the general public can see in advance what the building will look like when it is completed and can give their opinions, if it is just evaluated using photographs it may be difficult for many people to visualize what the building will look like, which may lead to inappropriate suggestions…” (Non-architecture student 20)

All of the 35 respondents who were not from an architectural background found the proposed VR system to be useful in the evaluation process. Since an average person does not have a good understanding of the building after observing the drawings and renderings due to the lack of spatial cognition of non-architecture professionals, it may result in a misperception of the building design. Out of the 35 respondents, 5 respondents who stated having previous experience using VR expressed that it was easy to operate the VR system proposed in this study as they have experience in using VR but conveyed that it might be difficult for first-time VR users to use this system.

4.

Responses received on the shortcomings of the developed VR systems

The last question was about the “shortcomings of the system conveyed by respondents”. For this question, most of the shortcomings were similar to the responses to the first interview question and related to either the “picture quality of the screen” or “movement method” used in the system based on the perceptions of the respondents. Apart from these two points, few respondents expressed limitations in terms of other aspects, as mentioned in the following interview extract.

A: First, if you add the ability to make changes to the environment around the object building using the UI, you will be able to appreciate the impact on the building style when the surrounding environment or building is changed. (Architecture student 15)

Architecture student 15 felt that architecture could be better understood if the surroundings could be transformed through the UI. A more detailed interview with this one respondent revealed that if transformations were made as suggested, multiple styles of environments could be made using ‘360° panorama photographs using a 360° camera, and multiple “360° panorama photograph” could be added to the environment, which would reduce the amount of real-time rendering in the system, while making it easier for users to switch between surroundings.

Architecture student 17 commented on the VR system in terms of hardware.

A: For architecture the material and color of the building is very important, sometimes the material of the building needs to be touched in person to feel it, so it would be better if you could add the function of touch. (Architecture student 17)

This respondent’s opinion of “feeling the building materials by touch” can be supplemented by “somatosensory gloves” that are connected to additional VR. Since the equipment for this questionnaire survey was primarily based on HMD, purchasing “somatosensory gloves” can be considered in future research to make the VR system more effective.

5. Conclusions

Despite the expanding use of VR in the architectural design process, the use of such technologies in the design evaluation process is yet to be explored. Considering this gap, this study proposes a VR-based evaluation model to examine the effectiveness and practical value of VR in the architectural design process. Based on a literature review on VR system characteristics, immersion theory, and the Technology Acceptance Model (TAM), a research model was designed to examine users’ intention to use the proposed VR system. A questionnaire survey was conducted for three groups of respondents, including architects, architecture students, and non-architecture professionals, to analyze perceived usefulness and perceived ease of use to understand the intent to use the proposed system in the design evaluation process.

Following the tests for reliability and validity of the collected survey data, structural equation modeling was used for examining mediator variables such as perceived ease of use, perceived usefulness, perceived immersion, and antecedent variables, including output quality and system characteristics. Using AMOS 28.0, a measurement model was constructed and was found to be a good fit for the data collected from the questionnaire survey, indicating that the proposed hypothesis model can be used to determine the structural equation model.

The research hypotheses were then tested through path analysis and multi-group analysis for the underlying variables. The results of the analysis between variables showed that, overall, the mediator variables were ranked in descending order of perceived usefulness, perceived ease of use, and perceived immersion in terms of the degree of influence on intention to use. On the other hand, in the case of antecedent variables, output quality has a higher degree of influence on intention to use than system characteristics. According to the results of path analysis and multi-group analysis, in terms of the influence of antecedent variables on mediator variables, output quality has a significant positive influence on VR users’ perceived immersion, and on similar lines, system characteristics also has a significant positive influence on VR users’ perceived immersion and perceived ease of use. Regarding the effects between mediator variables, perceived immersion has a significant positive effect on VR users’ perceived usefulness. Also, perceived ease of use has a significant positive effect on VR users’ perceived usefulness. Regarding the relationship between the mediator variables and the antecedent variables, perceived ease of use has a significant positive effect on VR users’ intention to use. Thus, the study illustrates a new perspective on the practical use of VR-based systems in architectural design by expanding its use in the design evaluation process through users’ intention to use such new technologies. Future research could consider expanding the system functions to meet the needs of the users by considering the analysis outcomes of this study. Also, considering technical changes to provide more intuitive and convenient operation methods can increase users’ ease of use and satisfaction with the system. Meanwhile, by comparing the results obtained from users with different backgrounds, it can be concluded that participants with different backgrounds will have an impact on the evaluation results.

Despite significant results, the current study was subjected to certain limitations due to the lack of technical competence, insufficient time, and budgetary constraints. First, appropriate experimental samples and sufficient sample size were selected for the study to obtain reliable and valid experimental results. However, since the respondents in this study were mainly selected on the basis of their educational background (architectural and non-architectural disciplines), the scope of respondents can be expanded in the future to include various groups such as designers, clients, VR professionals, and ordinary citizens, to obtain more constructive feedback. Secondly, in this study, the evaluation of “using VR systems to evaluate external building elements” mainly focuses on testing VR rather than evaluating the framework of VR systems.