Exploring the Impact of Digital Technologies on Team Collaborative Design

Abstract

This paper presents the results of a protocol study exploring the impact of various digital technologies on team collaborative design processes. Previous studies have suggested that compared to traditional methods such as sketching, digital technologies can provide further benefits for collaborative processes. However, there persists a lack of understanding about the impacts of digital technologies on such processes, particularly in relation to emerging significant digital technologies such as immersive Virtual Reality (VR). Therefore, this study aims to fill that gap by exploring team collaboration behaviours of two groups of professionals working in two digital design environments—desktop 3D modelling with Revit and immersive VR using Hyve-3D—as well as their behaviours during traditional sketching sessions for benchmarking purposes. Utilising protocol analysis method, the think-aloud data of participants was recorded, transcribed and coded using an adapted collaborative practice model. Team collaboration activities are broadly categorised as ‘Content’ or ‘Process’: content referring to design task-based activities, while process refers to activities related to the organising of group processes. The results suggest that during the design collaboration process, designers allocated the majority of their efforts towards process-oriented design activities. Differences between design environments only had a minor impact on the amount of effort expended on process-oriented activities and content-oriented activities. Moreover, traditional sketching design environments were shown to be potentially beneficial for problem-solution and associated negotiation activities. Additionally, immersive environments were associated with a reduction in the designers’ cognitive effort that was expended on exploring the design environment.

1. Introduction

Architectural design is among the most vital components of the building and construction sector, since the design stage determines up to 80% of a building’s operational costs [1]. Research has shown that overall design quality, accuracy and efficiency are influenced by digital modalities and the effectiveness of collaboration [2,3]. Architects typically collaborate utilising traditional sketching work environments, while collaboration via virtual means usually is reserved for rarer situations such as design reviews [4]. The recent decade has witnessed an increasing adoption of digital collaboration methods as the widespread preferred mode for collaboration, including in the architectural design industries [5]. Since 2020’s acceleration of remote virtual working globally, there has been an increase in the adoption of virtual working technologies throughout a variety of professional industries, with the architecture and design fields being among them [6]. Although computer-supported cooperative work in the design field has been extensively studied during the past few decades, during that time, the field has not advanced significantly in actuality [2,4].

Amongst the digital technologies used in the Architecture, Engineering and Construction (AEC) industry, digital technologies such as Building Information Modelling (BIM) have been widely adopted to facilitate effective stakeholder collaboration. However, there remains a lack of critical understanding regarding how such digital technologies influences the design behaviour of architect teams, particularly for emerging technologies such as immersive VR environments, which envelop the user inside a computer-simulated world of which they feel a part [7].

Therefore, this study aims to explore the impact of digital technologies on design collaboration with benchmark comparison to a traditional sketching environment. The digital design environments selected in this study are a 3D modelling environment (whereby designers collaborated across distributed locations via Revit and Zoom) and an immersive VR environment (which is a Hyve-3D environment). Hyve-3D is a type of VR CAVE environment that supports collaborative design with augmented reality (AR), VR and immersive visualisation technologies, particularly in fields like architecture, engineering and design cognition [8]. To achieve the above research aim, a protocol study to explore the team collaboration process of two groups of professional architects and planners in both digital design environments and traditional sketching environments were conducted. To analyse professional designers’ collaborative behaviour, we developed a design collaboration coding scheme by adapting a collaborative practice model developed by London and Pablo [9,10], with two foci of design collaboration action—content and process—as proposed by Stempfle and Badke-Schaub [11].

This paper consists of five sections. Following this Introduction Section, the Background Section provides studies on design collaboration and digital technologies supporting design collaboration. This is followed by the Research Method and Design Section, wherein the protocol analysis method is introduced and the developed coding scheme is described. The Results Section presents the coding results from the protocol data and the paper concludes with the Discussion and Conclusion Section.

2. Background

2.1. Studies on Design Collaboration

The act of striving towards shared design goals via a team-based design activity approach is known as design collaboration. Design collaboration’s benefits include the potential application of a wider number of approaches towards shared problem-solving; such shared activities can also help to effectively re-align differing perspectives of diverse stakeholders closer toward consensus [12]. During design collaboration, designers work in a team to achieve shared design goals. Various factors can affect the effectiveness of design collaboration, such as to what extent team communication is effective to share knowledge/expertise between team members [13], how much shared understanding exists among designers [14], team members’ design collaboration skillsets [15] as well as the level of efficiency of social networking services [16].

Although there are studies that support the use of collaborative design, there is still a gap regarding the nuances of how different fields of study can benefit from collaborative design and how different conditions influence the collaborative approach to design. In architectural design, for instance, it has been found that design collaboration within a team can enhance architects’ reflection upon their actions [17] and help overcome design obstacles such as unclear goals, closed-mindedness, limiting preconceptions, cognitive bias and limitations of design resources [4,18]. However, the nuances of decision-making during the collaborative process and how different conditions may influence such processes have not been studied. This gap in knowledge can potentially be linked to a lack of a theoretical framework or method to study and analyse collaborative design.

In trying to theorise how collaborative design is conducted, Stempfle and Badke-Schaub [11] propose two main foci of actions that are based on content and process. Content refers to design task-based activities, while process refers to activities related to the structuring or organising of group processes. This theorisation is similar to John Lang’s nominal work in architecture (Lang, 1987), where he argues how architectural design is shaped through ‘substantive’ and ‘procedural’ alterations. In this sense, substantive theories/practices focus on the content or substance of architecture. They deal with the physical aspects, such as form, materials, function, aesthetics and the built environment’s impact on society and culture. On the other hand, procedural theories/practices emphasize the processes and methods involved in creating architecture. They study the techniques, methodologies, design processes and decision-making frameworks that architects employ to realise built environments. Procedural theories are concerned with how architects approach design problems, make choices and execute their ideas.

Similar efforts to divide collaborative tasks have been also studied in other fields. For instance, Morgan, Glickman, et al. [19] aimed at understanding team evolution and maturation in operational Navy contexts, with the objective of enhancing Navy team training, performance and maintenance functions. This attempt involved both substantive (task-work) and procedural (teamwork) elements: the substantive aspect focused on gaining insights into team tasks, knowledge acquisition and adaptation to environmental demands, while the procedural aspect involved synthesising existing models and methodologies to develop a working model of team evolution along with prototype procedures for measuring team development. They based their study on task-work and teamwork activities, which further underscored the importance of delineating between the content (task-work) and process (teamwork) aspects of collaborative endeavours.

Since our study aims at understanding how team collaborative design is affected by various types of digital technologies, it draws upon the mentioned works and adopts the two main foci of actions—content and process—as part of the proposed and utilised coding scheme. This framework allows for a detailed analysis of collaborative design activities within digital contexts, considering both the substantive aspects related to design tasks and the procedural elements governing team interactions and decision-making processes.

2.2. Digital Technologies Supporting Design Collaboration

It has previously been established that design’s efficiency, accuracy and final quality are directly affected by digital modalities and the effectiveness of collaboration [2,20]. Effective collaboration is often supported by digital technologies. For example, in BIM environments during the conceptual design stage [21], social networking services can facilitate remote design collaboration effectively [16]. Previous studies have explored shared digital environments for design collaboration [22,23], including the effect of those environments on designers. Three-dimensional virtual worlds were found by Gu and Kim [24] to benefit synchronous design collaborations through the enhanced perception inherent to such virtual worlds. Furthermore, virtual environments offer valuable feedback on the three-dimensional aspects of a design, thereby enhancing engagement with the project, promoting visual thinking and supporting a deeper comprehension of spatial relationships and context [8]. Combrinck and Porter [25] also found higher levels of novelty and appropriateness in multi-user virtual environments compared to online sketching environments, due to the former’s explicit communication cues. Meanwhile, the AEC industry’s needs for collaborative VR in design and construction have been highlighted in various studies [26,27].

Recent advancements in immersive VR environments have enabled more intuitive interactions between designers and their design spaces, enhancing collaboration and engagement [28]. Other benefits include enabling distributed design collaboration, facilitating greater efficiency and cost-effectiveness in design [29], increasing design accuracy [30], increasing levels of engagement with the design, augmenting the perception of shared design representations [24] and facilitating deeper architectural understandings about space and place [31]. Sopher, Milovanovic, et al. [32] suggested that immersive VR can enhance architectural students’ communication better than non-immersive virtual environments. Roupe et al. [33] stated that VR systems together with multitouch tables can facilitate design collaboration and foster better understanding and communication among stakeholders. George et al. [34] emphasised the benefits of utilising VR for design collaboration in a studio context, which could assist students in better understanding design decisions and prototyping their design ideas rapidly and more effectively. Despite existing studies about digital technologies facilitating design collaboration, there remains a lack of understanding about how such technologies affect team designers’ behaviours, both in terms of their collaborative process as well as individual design exploration.

Throughout the AEC industry, digital collaboration tools such as BIM 360, Resolve, Unity Reflect Review, Fuzor and Trezi are already in use, mostly for construction scheduling and design reviews. A few specialist collaboration tools such as Mindesk, Gravity Sketch Wild and Arkio have also recently emerged, targeting the concept design stage of projects and supporting 3D import of models from common architecture design software tools such as SketchUp or Revit. Hyve-3D, a virtual collaboration tool focused on 3D sketching in immersive environments, supports team collaboration design. This tool has been chosen for use as the immersive design environment for the present study.

3. Research Method and Design

3.1. Protocol Analysis

Protocol analysis has been selected for this study as it is a method that can effectively turn qualitative verbal or gestural information into data [35,36] and has been widely utilised in design research as it relates to design cognition [37,38,39]. After protocol data collection, the collected data is categorised via applying coding schemes to the data, facilitating a detailed examination of design processes in a given design environment by applying retrospective and concurrent protocol collection methodologies to design experiments [35,40]. In the aforementioned context, a concurrent protocol refers to experiment participants intentionally verbalising their inner-thoughts while they are working on a task; this is also sometimes referred to as the ’think aloud’ method. Meanwhile a retrospective protocol refers to exploration of designers’ thinking using a process that is applied after the design task is finished. In the present study, we adopted a concurrent protocol collection method and a short interview was also conducted regarding participants’ views as to how digital tools affect their collaborative design process.

3.2. Tasks and Conditions

Participants comprised two expert architects and two urban planners who were divided into two groups, each including an architect and a planner working in teams. All participants had professional work experience as architects or urban planners. They all had good experience with Revit, however they were not familiar with Hyve-3D. Each design session was approximately 30 min. Each group was required to collaboratively conduct three design review tasks, respectively, using three different design environments in the following order: a face-to-face sketching environment, a 3D modelling design environment using Revit software and Zoom and an immersive VR design environment utilising Hyve-3D. The design tasks were to review designs in the context of discussions around pre-formulated housing issues related to planning, design or construction (such issues include consideration of built form, design alternatives and its relation to the public area according to the council regulations and guidelines). All three design review tasks had similar levels of complexity in terms of size, land use, building form and functions, etc. Hyve-3D (Figure 1, right) and the sketching environments were both face-to-face, while the 3D modelling environment was remote collaboration, with the designers located in two sperate rooms and communicating via Zoom (see Figure 1, left). The 3D modelling environment was conducted remotely; this is due to the nature of 3D modelling collaboration in which designers need a separate computer to complete their design task and a remote environment to minimise any voice distractions from one another. We do not expect there to be differences between a remote 3D modelling environment or a face-to-face 3D modelling environment.

3.3. Coding Scheme Development

Table 1. Coding scheme.

	Coding Category	Definition	Example
Content	Expertise	Relevant expert knowledge or personal experiences.	“I interact with the neighbours in other places.” “spaces that tend to be more public versus residential as an architect, we’d start to suggest some more hard-wearing and resilient materials.”
	Problem Solving	Design solutions or suggestions for the identified design problem.	“Yeah, we are thinking to have a green wall on the west facade to protect the building from west sunlight but not the entirety, just one part of the commercial area will be green vertical.” “Yeah, I will need to have a second entrance.” “And also the area of the residential room must be at least 7.5 square meters.”
Process	Organising Mechanisms	Procedures and processes which aim to progress collaboration.	“Back to our previous conversation.” “And what we’d like to do is talk through this ground floor plan that we have here.” “So that’s something for us to review as well.”
	Shared Goals	Segments in which a shared goal or solution relating to a design problem is being worked towards, leading to or progressing towards collaborative agreement about the solution.	“Is it better this way or that?” “No, so not a swimming pool?” “Let’s say it’s transparent.”
	Solution Negotiation or Agreement	Segments related to negotiations about potential solutions or reaching an agreement about solutions.	“Yeah, it’s in the basement.” “So parking is not considering all this active activity space, this event space, it’s not calculating all this area?” “Do you also consider the visitors when it comes to this parking?”
	Shared space	Segments pertaining to digital or physical shared workspaces, including discussing optimised software use, the orientation of the observer within the design representation as well as broad discussions related to the design representation.	“Can you see two buildings?” “What’s over here?”

shows the developed coding scheme for this study. The coding scheme was adapted from the collaborative practice model developed by London and Pablo [9,10] as well as the two main foci of actions during collaborative design—content and process—proposed by Stempfle and Badke-Schaub [11]. The original collaborative practice model focused on multiple stakeholders’ collaboration during the design and construction process and identified the following nine key elements of collaboration: Leadership, Shared Goals, Expertise, Change, Problem Solving, Investment, Shared Space, Organising Mechanisms and Technical Standards [9]. Our present study only focuses on the design stage and some of the elements. Change, Technical Standards, Investment in Relationship and Leadership are not relevant to our study; hence our study only focuses on the other five elements and we also added the category Solution Negotiation or Agreement. This is due to the nature of our design experiment since designers tended to discuss the solutions and sometimes reached agreement about certain design decisions. All six elements were grouped into two main foci of actions—content and process—as shown in Table 1. Since the content-oriented activities are design task-based activities, Expertise and Problem Solving are categorised as Content, while the activities of Organising Mechanisms, Shared Goals, Solution Negotiation or Agreement and Shared Space are related to the structuring or organising of group processes and thus are all categorised as Process. During the coding process, multiple codes for each segment were adopted.

4. Results

Percentages of codes were calculated and analysed after coding. This section presents the results of coding distributions in the two digital design environments—the 3D modelling environment and the immersive VR environment—as well as in a traditional sketching environment for baseline comparison. Figure 2 demonstrates the coding distribution of content-oriented activities and process-oriented activities for all six design experiments. From the figure, we can see that on average, process-oriented activities make up 73.12% of the overall activities, while content-oriented activities make up 26.88%. This suggests that during the design collaborations, designers expended approximately three times more effort in process-oriented activities, such as establishing shared goals, negotiating solutions, organizing the process or establishing shared space, than in content-oriented activities, which focus on individual tasks. We can see that team-related activities appear to be the dominant activities during the design collaboration process.

Figure 3 shows the comparison of coding distributions of content-oriented activities and process-oriented activities, respectively, in the three design environments. The results suggest that the coding percentage of content-oriented activities and process-orientated activities is similar across the three design environments, which indicates that design environments do not have a significant impact on the percentage of content-oriented activities and process-oriented activities and that the designers expended similar amounts of effort on group work process in all three design environments.

Figure 4 illustrates the coding percentage of content-oriented activities and process-oriented activities across the whole design session of all design experiments. The results show that content-oriented activities were relatively consistent across the whole design session, whereas process-oriented activities fluctuated across the design session—during the early design session process-oriented activities decreased, while towards the end of the design session they increased. This suggests that potentially designers started discussing initial ideas together, exploring shared space and setting up shared goals; then, as the design proceeded, their team-based activities decreased as they focused on individual design activities and then increased towards the mid-design session. Towards the end of the design session, the team-based process-related activities increased again, which may be related to agreement upon solutions.

Figure 5 presents the coding percentage of content-oriented activities and process-oriented activities across the whole design session in three different design environments. From the figure, we can see that during the early design stage, the process-oriented activities in the sketching environment increased then decreased, while they both decreased in digital design environments. This may be because during the early design stage, sketching environments stimulate collaboration-based activities such as exploring shared space or setting up shared goals. During the mid-design session, process-oriented activities in the 3D modelling environment were higher than in the other two environments, which may indicate that the distributed location of the 3D modelling environment may cause designers to expend more effort on group process activities, such as exploring shared space and establishing shared goals. During the end design stage, designers’ process-oriented activities decreased and then increased in the two digital design environments, while the traditional sketching environment exhibited the opposite. This potentially suggests that digital design environments stimulate team-based activities towards the end of a design session, such as when reaching agreement.

Figure 6 describes the coding distributions in all three design environments. From the figure we can see that Shared Goals take the highest percentage (47.37%); this is followed by Expertise (23.09%), Shared space (12.53%) and Solution Negotiation or Agreement (9.67%), while the least activities are Problem Solving (3.79%) and Organizing Mechanism (3.55%). During the collaboration process, the setup of shared goals is related to the exploration of the design problem space, including the constant restructuring of the design problem during the design process. The high percentage of established goals is potentially beneficial for design problem space redesign and possibly for the emergence of design creativity [41].

Figure 7 presents the comparison of coding distributions in the three different design environments. Results show that designers exhibited higher coding percentages of Problem Solving (5.61%) and Solution Negotiation or Agreement (17.38%) in the sketching environment than in the other two digital environments; while the Shared Goals coding percentage is lower in the sketching design environment (37.59%) than in digital design environments (51.35% and 53.19%). This indicates that a traditional sketching environment is potentially beneficial for assisting designers in exploring the design solutions space; while digital design environments may assist designers in exploring the design problem space more effectively. Furthermore, the coding percentage of Shared Space is lower in the immersive VR environment (8.17%) than in the other two design environments (15.29% and 14.12%). This may be because the immersive VR environment is intuitive and realistic, which is easier for designers to understand and thus there is less cognitive effort expended on exploring the VR environment.

Figure 8 describes the coding percentage of selected codes across the whole design session in the three design experiments. The four codes of Problem Solving, Shared Goals, Shared Space and Solution Negotiation or Agreement were selected for analysis, since they are the most relevant to designers’ main cognitive activities during collaboration. From the figure we can see that the coding percentage of Shared Goals decreases at the beginning of the design session, then increases and stays relatively consistent during the mid-design session, then decreases towards the end of the design session. This suggests that designers establish shared goals more so at the beginning of a design session and then their cognitive effort is reallocated to other design-solution-related activities as the design proceeds. The results also show that the overall coding percentage of Shared Space has a slightly decreasing trend towards the end of the design session, which may be because designers have developed a greater understanding of the design environment as the designing process progresses, therefore less cognitive effort is required for exploring the shared space. Another observation is that the coding percentage of Solution Negotiation or Agreement increases towards the end of design session, where more design decisions need to be made.

Figure 9 presents the coding percentage of Shared Space across the whole design session in the three different design environments. The results suggest that at the beginning of the design session, the coding percentage of Shared Space decreases faster in the two digital design environments than in the sketching design environment, indicating that designers explore the digital design environments at the beginning and once they get familiar with the environment then less cognitive effort is required for understanding the digital environment. Another observation is that there is a peak of coding percentage for Shared Space towards the mid-design session in the sketching environment, potentially because designers need to read and understand sketches during the mid-design session.

Figure 10 illustrates the coding percentage of Shared Goals across the whole design session in the three different design environments. From the figure, we can see that designers expended more effort on setting up shared goals in the immersive VR environment during the mid-design session than in the other two design environments, which may be because immersive VR environments provide an intuitive and realistic environment that assists designers in exploring the design problem space.

Figure 11 describes the coding percentage of Problem Solving across the whole design session in the three different design environments. The results suggest that designers’ problem-solving activities gradually increase at the beginning of the design session in all design environments, then decrease during the mid-design session. During the mid-design session, more problem-solving activities are witnessed in the sketching environment than in the two digital design environments, which indicates that the sketching environments may be beneficial for problem solving, especially during the mid-design session.

Figure 12 presents the coding percentage of Solution Negotiation or Agreement across the whole design session in the three different design environments. From the figure, we can see that across most of the design session the coding percentage of Problem Negotiation or Agreement is higher in the sketching environment than in the two digital design environments. This indicates that sketching design environments are beneficial for designers to explore design solution spaces together, potentially because sketching may stimulate design solution negotiation. In the two digital design environments, towards the end design session, the designers tended to perform more Solution Negotiation or Agreement than in the early and mid-design sessions. This may be because in digital design environments, designers expend more cognitive effort on exploring a shared space so there is less negotiation of design solutions at the beginning of the design session.

5. Discussion and Conclusions

This paper presents a preliminary study exploring the collaboration behaviour of designers in two different design environments—a digital 3D modelling environment and a digital immersive VR environment—and in a traditional sketching design environment for baseline comparison. The main findings of the research are as follows:

Firstly, designers’ foci of activities can be categorised as process-oriented activities or content-oriented activities. Process-oriented activities refer to the structuring or organising related to group processes, while content-oriented activities refer to activities related to tasks which are mostly individual based. During the design collaboration process, designers allocated most of their effort to process-oriented design activities. Design environments differences had only a minor impact on the amount of effort expended on process-oriented activities and content-oriented activities. Furthermore, during the design collaboration process, process-oriented activities fluctuated across the whole design session; such activities decreased at the beginning of a design session and increased towards the end of a design session. By contrast, designers expended a relatively even level of effort on content-oriented activities throughout the entire duration of the design session.

Secondly, in terms of the impact of design environments on designers’ collaboration behaviour, at the early design stage, their process-oriented activities decreased in digital design environments, but increased in the sketching design environment. During the mid-design stage, there were more process-oriented activities in the 3D modelling design environment than in the other two design environments. We also found that there was more problem-solving and associated negotiation activities in the sketching design environment than in digital design environments; while there was more design problem space exploration (such as establishing shared goals) in the digital design environments, compared to the sketching design environment.

Additionally, designers had less shared-space-related design activities in the immersive VR design environment, which may be due to the character of immersive VR design environments in that they are more intuitive and easier to understand, thus less effort may be needed to comprehend their shared space. This suggests that designers expend less cognitive effort in exploring an immersive VR environment, thus they can focus more on other design activities such as exploring the design solution and problem space. These findings are in accordance with prior studies which have suggested that immersive virtual environments can facilitate more intuitive interactions between designers as well as between designers and their design environment [28], causing improved spatial cognition [42] that may benefit more effective design collaboration.

We are aware of some unavoidable limitations of this study; for example, since this is a preliminary study, its sample size is rather small. Future studies should enlarge the sample size to obtain more statistically sound results and to enable more generalisable results. Potentially, future studies may also include analysis of additional collaboration-related rich data that is generated by advancing immersive VR environments (such as the impact of different levels of immersion), which could deepen our understanding of design collaboration in such increasingly immersive design environments. This study’s developed coding scheme delivers an innovative theoretical advancement in the area of design collaboration. The results of this study further our understanding of the design collaboration process within different digital environments, especially in relation to the impact of newly emerging immersive VR environments on team design behaviour. The enhanced understanding of design collaboration using digital technologies will potentially promote virtual collaboration within the AEC industry. This will ultimately lead to an acceleration of design collaboration and improved design solutions in the building sector, which can potentially increase project productivity, optimise design quality and reduce building costs. The results of this study are potentially beneficial for architects, design educators and software developers.