City Ditty: An Immersive Soundscape Sketchpad for Professionals of the Built Environment

Abstract

Soundscape planning remains a challenge to many urban practitioners due in part to a scarcity of soundscape design tools. While many sound planning tools exist, they are generally geared towards acousticians rather than professionals of the built environment (e.g., urban designers, planners, or landscape architects). This paper walks through the user-centered design process for the development and evaluation of a new soundscape design tool, City Ditty. A User-Centered Design approach was utilized to identify and develop functionalities that would benefit urban practitioners that do not currently specialize in sound. This began with a literature review of existing soundscape tools, followed by a user needs assessment with professionals of the built environment, consisting of a workshop including focus groups, tech demos, and a collaborative soundscape design exercise. These results funneled into the development of City Ditty: an immersive soundscape sketchpad that facilitates rapid audio-visual prototyping of urban soundscapes. To make City Ditty accessible to users with no expertise in sound, we developed a sound awareness session that walks the user through 36 tasks. These hands-on tasks illustrate soundscape design principles while serving as instructions on how to use the many functions of City Ditty, e.g., listen to the city soundscape at different times of the day, pedestrianize the city centre, modify permissible construction times, and add birdfeeders to attract sounds of nature. A usability study was conducted with six participants to evaluate the tool using Desktop Virtual Reality, determine new functionalities, and determine how to best facilitate user engagement in order to encourage adoption by practitioners. The direction of future soundscape tools is discussed.

1. Introduction

Soundscape planning has been shown to be a valued, yet underutilized method for creating and improving urban spaces. For example, when properly planned for, soundscapes can promote public space utilization [1], foster social interactions [1], and promote stress recovery [2]. However, proactive soundscape planning remains underutilized. Professionals tend instead to be reactive to sound-related issues, treat sound as nuisance to be mitigated, and focus primarily on staying below maximum permissible sound levels [3,4]. Efforts to bridge the academia–practice gap include new courses that are being developed [5], workshops for knowledge mobilization held between academics and professionals [6], and design actions that facilitate positive soundscapes formalized [7]. Archives of existing soundscape designs are growing [8,9] and regulations are being reviewed for improvements [10,11]. Furthermore, larger organizations like the United Nations Environment Programme (UNEP) have also recently identified sound planning as an environmental concern that requires addressing [12], which increases visibility and raises hope to elicit this change in culture. The lack of uptake by professionals can also be attributed to the lack of sound-planning tools available for professionals.

To address this lack of tools, we used a user-centered design process to identify the needs of Professionals of the Built Environment (PBEs) (e.g., urban planners, designers, or landscape architects). These professions have a great influence on our cities, yet do not typically consider sound in their work, nor do they have much training in it [4]. While specialized tools exist for acousticians (see [13] for a review), they typically rely on mapping sound levels over broad areas or detailed indoor spaces. While invaluable for some purposes, sound levels do not tell the entire story—nor is this software accessible to non-acousticians. As such, we argue for the need of an intermediary sound-planning tool that is accessible to a greater range of professions. We propose that such a tool should both help raise their sound awareness and facilitate rapid prototyping of different urban soundscapes, focusing on designing towards the human experience. What will attract or detract people from using a public space, beyond sound levels? How can PBEs engage with sound as a resource, while acknowledging and working within a scope of their limited knowledge of acoustics? Such tools remain unavailable to PBEs, but could help pave a pathway to better sounding and healthier cities [14]. Given this need, this paper discusses the process of the research, development, and first evaluation of such a new tool we named City Ditty.

This research was conducted as part of the Sounds in the City partnership, which brings together academic researchers, PBEs, sound artists, and citizen groups to nourish creative solutions to make our cities sound better. In this context, we have organized knowledge mobilization workshops since 2016 to raise PBEs sound awareness by introducing them to different urban sound interventions [6], and to discuss different prototypes of sound-planning tools [6]. These tools have shifted from “as-is reproductions, which directly record soundscapes and play them back, to simulators which are built digitally from the ground up using multiple sound sources and objects in a virtual space” [15]. The first two iterations in 2016 and 2017 were as-is reproductions which demonstrated how soundscapes can either be conducive to human activity or hinder it. While these early iterations have been able to raise sound awareness with PBEs, it cannot yet be integrated into their workflow. Doing so requires that PBEs have a suite of functionalities available to them which facilitates rapid soundscape prototyping and playback of their designs.

The third iteration in 2019 began to approach these needs by allowing the placement and movement of sounds in a 3D space via an audio-only simulator [13]. This allowed for very realistic soundscapes to be built and modified to immediately hear the changes requested by PBEs. While more effective than its predecessors, it was still operated by researchers, rather than from the ground up by PBEs. Furthermore, as an audio-only simulator, it fails to provide the visual context which is a key consideration for soundscapes [16]. Indeed, a need for multisensory soundscape tools has been identified as one of the top priorities and challenges for soundscape research as a field [17,18,19], as the interpretation of sound is inseparable from other environmental factors. As such, these considerations have led us towards a fourth iteration, as described herein.

Following previous experiences with PBEs [6], the user-centered design process continues into this next iteration. However, this time we approach it through the lens of Human–Computer Interaction (HCI) [20,21]. This approach observes the ways that people interact with computers and helps identify and implement novel technologies in a manner that is amenable to their wants and needs. Such an approach is crucial, as this issue requires not only the integration of cross-disciplinary contexts (i.e., social sciences and humanities, natural sciences and technology, and public health), but also requires interaction with those from the public and private sector, requiring a transdisciplinary approach [22]. HCI and user-centered design provides a framework for this integration process. We place emphasis on PBEs and designing to their needs, to introduce and facilitate soundscape planning in our own work. Following the HCI approach, the form of a new tool should be dictated by its required functionalities. As such, we take a ground-up exploratory approach to discovering these, based on the following research questions (RQs):

RQ1: What are the user requirements and functionalities of a tool for PBEs which would facilitate a soundscape approach to sound planning?

RQ2: Do the identified functionalities facilitate sound planning by PBEs?

Current HCI research on soundscapes does not address these needs. A recent review of 235 HCI soundscape articles had a gap in how PBEs need to engage with soundscape design in a manner that helps them adequately address the auditory experience [23]. Given that most PBEs do not currently engage in soundscape work, one cannot simply ask them what they need, as is a central tenet of user-centered design. Rather, we must lead the discussion and interact with PBEs at different intervals to present ideas and gather further feedback. Through this iterative approach, we aim to gather a better understanding of how PBEs would like to use a new sound-planning tool (RQ1) and have them evaluate such a tool and its functionalities (RQ2), rather than the soundscape designs themselves. The proposed tool focuses on (1) raising soundscape awareness, and (2) rapid design and manipulation, rather than soundscape evaluations, which remains beyond the scope of this study.

We address these research questions in several phases. First, we discuss the requirements gathering for City Ditty which includes (1) a literature review, (2) the identification of relevant conceptual frameworks to help guide user adoption and aid the evaluation of City Ditty, and (3) a user needs assessment with PBEs. Second, functionalities are identified from the collected data and are implemented into the first version of a new immersive soundscape sketchpad. This is accompanied by the creation of a sound awareness session, which guides the users through the software. Third, a usability study is conducted to gather feedback on City Ditty and its functionalities. Finally, this early feedback is used to discuss and direct both theoretical and practical implications regarding how new tools can help design soundscapes to enhance human experience, the form they could take, and how to facilitate future adoption and help support a new generation of both soundscape-savvy PBEs and tools.

2. Requirements Gathering, Design, and Development

We report on a literature review of the academic soundscape literature, followed by user need assessments conducted as part of a future workshop held in 2019. This invited both public and private sector PBEs to partake in several soundscape-based events to gauge their level of interest in new tools, how they might use them, what barriers from entry they may have, and what their current level of discourse on the topic is. By utilizing a combination of themes from both literature and workshop, we transform these into a core set of functionalities for a new prototype tool.

2.1. Literature Review

While there is much research on the benefits of soundscape design in general, few studies discuss what tools are available and how they should be used at design time. Yanaky et al. discusses the need for sound-planning tools in more detail, along with how current literature has explored the viability of using immersive tools for soundscapes [15]. The majority of previous studies focused on determining if reproductions are ecologically valid, i.e., can reproductions elicit responses and outcomes comparable to experiences in real-life environments [13,15,24,25] and/or which hardware works best at reproducing this to facilitate the evaluation of soundscapes themselves [26]. For example, Maffei et al. [27] compared VR environments to real in-situ environments for evaluation of public spaces, while Hong et al. [28] studied the quality of different sound reproduction techniques. Other work goes beyond the hardware’s ability to reproduce audio-visuals by applying the concept of ecological validity [29] to soundscape reproductions to try to produce optimal lab reproductions [15,30]. These identify further aspects by including the sample population and activities performed by the user to elicit the most accurate evaluations in a reproduction. This trend continues as a hot topic as others follow with reviews specifically on ecological validity for VR [31], and further techniques in evaluating soundscapes in VR [32].

The reproduction of soundscapes in a lab environment is highly desirable as it provides control over contextual factors such as times of day, weather conditions, and seasons, which have been shown to influence soundscape evaluations; see Tarlao et al. [33] for a review of a wider range of contextual factors. Not only can soundscape reproduction leverage control over contextual factors, but also help identify issues that may not otherwise be apparent at design time, due to the complexity of the urban space [13]. For instance, while individual sound sources can be easily considered in isolation (e.g., traffic is loudest during rush hour), multiple sound sources combine into a larger acoustic environment that can quickly become overwhelming to keep track of. For instance, pubs are louder at night, grocery stores attract loud trucks for early morning deliveries, construction noises occur within their respective start and stop times, etc. These complex, ever-changing acoustic environments will lead to different responses and outcomes from city users, as these soundscapes can either facilitate—or disrupt—how a city user utilizes a public space or not.

To capture this complexity, the Quality of the Urban Public Experience (QUPE) model accounts for the psychological, behavioural, and social responses from city users to soundscape design [1], e.g., did the soundscape foster social interactions? Did people linger to utilize local businesses? Did it promote stress reduction? Such feedback is crucial in understanding how effective a soundscape design is. Unfortunately, it is also very resource expensive to both prototype and collect adequate feedback from designs in city spaces.

On-site soundscape interventions require extensive collaboration between researchers, city officials, and local firms to plan and implement soundscape designs and evaluate them with public space users [1,34]. Simulations could allow for quick digital prototyping and evaluation of different options to narrow down the possibilities before physical prototypes are built.

While it is not unusual for private firms and researchers to create 3D digital prototypes for consulting and soundscape evaluation, these still require very specialized skillsets and remain time consuming to build, which will act as a barrier for entry to most. For instance, Jiang et al. [35] created a model of Piazza Vittoria in Naples, Italy which allowed users to walk around to explore and evaluate the city soundscape in different places. However, users could not use it to make changes to the soundscape. Payne and Bruce [36] went a step beyond this by having the user enter a set of parameters that could be used to construct different soundscapes, such as amount of vegetation, size of water fountain, or amount of traffic. However, in the end these settings still only linked the user to a particular video that had been pre-rendered; nothing could be modified by the user in the end. Further work is required to facilitate user designs.

In summary, existing tools focus predominantly on reproduction techniques to present and evaluate existing sound environments. They do not allow non-sound experts to manipulate contextual factors and sound sources in a way that facilitates rapid prototyping. Such methods of both manipulating and interacting with a digital environment is foundational to making a tool for PBEs, which is the focus of this work.

Identifying Frameworks for Tool Research and Development

This HCI approach relies on two relevant frameworks to help explore new themes and guide the evaluation of a new tool: presence [37] and user engagement [38].

Due to some conceptual overlap with ecological validity, the concept of presence may be a useful bridging gap for applying some concepts of ecological validity to the development of new tools, while remaining less restricted by its stringent nature. Presence suggests that if a suitable degree of immersion is provided within a plausible environment, people can experience an illusion of being in the virtual environment, and thus will respond similarly to how they would in real life. Slater [39] defines presence as “the strong illusion of being in a place in spite of the sure knowledge that you are not there”. This illusion of presence is facilitated by two factors: First, immersion, which is the technological ability to produce audio, visual, and any other relevant information to be interpreted by the senses, and second, plausibility. Plausibility is a higher cognitive belief that what is happening in the digital reproduction is actually (or plausibly) happening. In other words, the world acts and responds to the user as one would expect it should, e.g., small cars do not sound like large trucks, and traffic volume is much lower at night. When a sense of presence is held by the user, a virtual world can elicit similar sensory sensation and interpretations from people, causing responses within the virtual world as it would in the real world [40,41,42].

Second, the user engagement model is used to help evaluate the experience of PBEs and help indicate whether they may adopt such a tool in the future. O’Brien et al. [43] defines this as “User engagement (UE) is a quality of user experience characterized by the depth of an actor’s cognitive, temporal, affective and behavioural investment when interacting with a digital system”. More simply, was the user’s experience engaging and would they use it again? This framework breaks down user engagement into four factors that help evaluate the user’s experience, including focused attention, perceived usability, aesthetic appeal, and reward. UE includes a validated questionnaire that can be used to evaluate this experience [43]. UE was chosen over other models and usability heuristic devices as it provides both a well-rounded set of themes for exploration, as well as a questionnaire towards evaluating a user’s experience towards potential repeated usage.

2.2. User Needs Assessment

The workshop, entitled “Co-designing soundscapes of public spaces: Integrating new technologies and approaches”, was a one-day event with 25 PBEs from both the public and private sector [13,15]. The PBEs were divided into three groups and rotated between three main activities: (1) focus groups, (2) a collaborative soundscape design activity with an in-house built audio-only simulator (the third iteration tool [13]), and (3) a hands-on tech demo area which had numerous existing sound-planning tools and immersive experiences to try.

The focus groups are the primary source of information for this analysis, although it is supported by observational data from the tech demo room and collaborative design exercise. Full description and results of the collaborative design exercise are available [13]. The tech demo rooms were full of different existing tools and devices for participants to try as they pleased, including (1) 360-degree videos on both smartphone-based VR (an iPhone 6) and an HTC Vive Pro VR system with a sample of urban soundscapes recorded with a first-order Ambisonic microphone (Soundfield ST350) and Insta360 Pro camera to demonstrate both high and low-end playback resolutions on different quality devices; (2) 3D-rendered content of a public space focusing on transit created by Arup, viewed through several Oculus Go VR sets; (3) an archeology-based 3D recreation of a Paris soundscape from 17th century titled ‘Bretez’, presented on a laptop [44]; (4) an audio-only demo of noise barrier interventions using acoustic modeling software MithraSON and Micado, on laptop; and (5) a traversable 3D rendering of Piazza Vittoria in Naples, Italy [35] on laptop with two sets of headphones, allowing single or pairs of participants at once. A researcher was assigned to each technology and took notes on how the participants reacted to the demos.

Focus groups were transcribed and coded following a grounded theory approach [45] to support an inductive analysis. Additionally, observations were gathered during the tech demo and collaborative soundscape design activities. Together, these produced the themes from the axial coding process, topped by the need for new tools as the overarching theme. Note that two groups were held in French, and one in English; supporting quotes from French are translated and provided in English in italics.

Seeking Better Sound-Planning Tools

All focus groups acknowledged that they lack adequate tools for considering soundscapes in the early stages of their work, and that relying on the decibel level alone was universally recognized as insufficiently capturing the entire sound experience, e.g., (1) “Our benchmark measure is 55 dB, but that is not necessarily realistic in the sense that everyone believes that as soon as we exceed this level, it becomes a nuisance when it is not necessarily the case”; (2) “a swimming pool pump at night, yes, is perceived as very disturbing, while no one realizes it by day”; (3) “Even at the regulatory level, at the municipal level, we do not have the regulatory tools or the laws that come, which allow us to govern, precisely, adequately, anyway, I would say. So, and even in our constructions, we don’t think about it, acoustics”.

After experiencing the collaborative soundscape design activity (using the audio-only soundscape simulator), some users reported a high degree of interest in such new tools for sound design. “I think in terms of design, that would be very interesting as a tool. I even think in terms of—especially large-scale projects at the level of in town planning or in development—this is something that would be super useful, and in fact, I was impressed”. Even those who had some experience expressed an interest for better tools. “I know that we do exercises with virtual or augmented reality for in terms of consultation. It was not necessarily what we see today, but I think in terms of planning, it would be super useful”. Such responses both acknowledged their current tool-based limitations, as well as express interest in expanding their repertoire with newer tools. This overarching theme is further supported by the following subthemes.

Must Support Development of Soundscape Culture

To further advance the culture of soundscape planning, a new set of PBEs must emerge who are experienced in soundscape designs. These new leaders would contribute to a growing archive of soundscape designs, help lead real world projects for real world impact, and work with others to help shape future sound-based regulations, guidelines, and curriculum. How can tools support the creation of new leaders, though?

Currently, there are insufficient regulations and policies to guide soundscape planning and evaluation sessions, beyond the current status quo of adhering to noise limits. As such, intrinsic motivational methods must be utilized to encourage new PBEs to engage in soundscape designs. When applied to tools, this means it must cater to the usability needs of the user and encourage user engagement through extended and repeated usages from a positive user experience to produce and share more soundscape culture.

Must Raise Sound Awareness

While participants were generally aware of many limitations of relying on just sound levels, they were unsure of the next steps to take to better plan for sound. Rather, they sought guidance in how to apply this to their own work. “As engineers, we are used to applying certain principles which, at an absolute level—noise level, is really easy. You pass or you do not pass… for this, we have no tools, we do not know how to do, in fact. And that’s why we’re here, by the way”. Furthermore, those who had a bit of sound experience sought additional ways to demonstrate to others (e.g., clients) the benefits of soundscape planning.

Concerns for Operating new software and hardware

Many participants expressed concern over their ability to use some of the existing software and hardware—particularly regarding the use of virtual reality equipment. Concerns ranged from setting up and troubleshooting hardware, to operating the software on their own. It was also recognized that the larger audio-only simulator used for the collaborative soundscape design session could only be operated by the researchers/demonstrators. As such, accessibility to those without strong technical backgrounds remains a key issue for future adoption.

Desire to interact with the Virtual Environment

Participants wanted to move around in the virtual environments, even when there were no interactive options possible. For example, when given a head mounted virtual reality device displaying a 360° video or virtual rendering that does not support movement beyond head turning, participants often tried to walk through the virtual environment. Some participants remarked that it was so realistic that they forgot multiple times in a single demo, while others lamented the inability to explore the virtual environment. Even during the audio-only design session, several participants still walked through the listening space as a manner of interaction, despite no tracking available.

Ability to Explore Different Contexts in the Same Space

The literature review suggested a weakness in current research on how to manipulate the context and sound sources with current tools. This was echoed during the workshop as PBEs showed a great understanding that our soundscapes are primarily affected by people. “Sound happens when people start to interact. You need to know what will happen before you install it”.

When considering a past project, one PBE remarked “I think we should have used a 3D design of the [public building we constructed], and we should have tested the- like, what we saw this morning in the [collaborative sound design session]. You know, how the sound is diffused and would be different in different situations, and different times of the day, or different times of the week too. And also, if we- since we had the intention to produce shows around the building, I think it should have been one of the first considerations to take in account.” This alludes to both fairly common and predictable contexts (i.e., time of day/week) which could potentially be automatically modeled by the tool, as well as more unique events like shows that would require special attention by the designer.

Should Facilitate Collaborative Designs Amongst Different Users

Collaboration comes in many forms. In some cases, PBEs related this to the residents and city users themselves (e.g., stakeholders), acknowledging the individual differences in some communities and how urban planning can affect this. E.g., “When there are buses that run all day in an area where are there many retirees—people who are at home [during the day]—, it is perceived differently than in an area where there are many workers leaving for the day.” As such, to obtain feedback from all different types of potential stakeholders, the tool must have an easy way to view and experience the soundscape, with minimal-to-no technical training.

Observationally, participants also seemed to favour activities in the tech demo room where they could participate in pairs or small groups. This allowed for discussion and greater engagement with the soundscapes and technologies, as they were able to exchange ideas and ask questions. During the collaborative design session, this also allowed for more creativity, ideas, and immediate feedback, as the group of PBEs were all able to participate at once.

Planners are Visual

Although the audio-only design exercise was well received, PBEs are used to working with visual information. “We’re very visual—we want to see the design space.” PBEs stated that they typically rely on visuals like photos, videos, 3D models, and other material which help in generating discussion on the history and contexts of a space, which influences future design decisions. As such, when planning sound, they would appreciate more visual cues to help in their design and evaluation process. However, they are not too clear on how this may be done. This could be aided by literature on representing soundscape quality via colours [46], although it must be repacked from a UX perspective. That said, some participants also appreciated not being constrained by any visual aspects, suggesting that there is room for both audio-visual, and audio-only tools.

Integration into their work

Aside from the desire to include early-stage sound planning into their work, one participant inquired on how sound designs could be reported on. Currently, as very few PBEs consider sound, it does not seem as though many have considered how they would like to document and report on sound designs (beyond the decibel).

Provide an Immersive Audio-Visual Experience

Participants acknowledged that soundscapes are inseparable from their visual surroundings. Despite their positive experience with the audio-only collaborative design exercise, they felt the experience was lacking without the immersive visuals to accompany it, e.g., “For me, there was a lot of data missing for that because it was ‘well, where am I located in relation to the sounds I hear? Am I upstairs? Am I downstairs?’ I didn’t understand where I was in this picture.” This theme blends well with those regarding the need for visualization, and providing the ability explore the different contexts of soundscapes. If sound relies heavily on visuals, the inverse should be true in consideration of an urban space.

2.3. Design Recommendations

The design and implementation consist of an analysis of the themes, through to the implementation of a first working audio-visual sound simulator, dubbed City Ditty. First, the themes are used to extract proposed functionalities for City Ditty. These functionalities are discussed in terms of importance and feasibility to determine which are included in its first iteration. Second, the design of the sound awareness training sessions is presented, which introduces a series of informed tasks which leads the user through the software and different elements of soundscapes.

2.3.1. Identifying Functionalities from Themes

A breakdown of the proposed functionalities is included in

Table 1. Proposed functionalities, drawn from themes.

Theme	Proposes Functionalities
Support development of soundscape culture	Ability to create, save, share, and view soundscape designs
Raise sound awareness	Include a sound awareness module with hands-on soundscape design examples to follow which promotes active listening to the environment
Concerns for operating new software and hardware	Integrate user instructions with the sound awareness module Minimalist user interface aimed at novices
Desire to interact with the virtual environment	Ability to navigate a 3D city model Ability to add, modify, and remove sound sources in 3D space
Explore different contexts in the same space	Manipulate time of day, indirectly changing the sound sources and other contexts (e.g., traffic) Be simple to save and compare different designs
Facilitate collaborative designs amongst different users	(Future) support multiple users at once
Planners are visual	Provide visual aids to help locate, manipulate, and evaluate a sound sources’ impact on an area
Integrate into their work	(Future) Allow for integration with existing design software and assets (e.g., user-owned 3D models, sounds, etc.) (Future) create soundscape reports for use outside of the software
Realistic, immersive audio-visual experience	Provide realistic, high-fidelity audio and visuals

. Each functionality was drawn from the identified themes. All of these functionalities are implemented, except where indicated to be future additions.

2.3.2. Sound Awareness Session: Creating Informed Tasks for Training and Testing

Since most PBEs currently have little-to-no experience designing soundscapes, a self-guided sound awareness session is included, which walks the user through 36 tasks. These tasks seamlessly integrate sound awareness lessons with instructions on how to use City Ditty by incorporating hands-on lessons. Through this, the user will walk themselves through each of the different functionalities before opening them up to three free design tasks which lets them apply what they have learned.

To help keep these tasks relevant, many of these tasks are drawn from practical examples from Cerwén et al.’s Soundscape Actions [47]. These include examples such as the introduction of wanted sounds (e.g., add bird feeders to attract birds or water features to incorporate auditory masking), reduction in unwanted sounds (e.g., replace old noisy machinery or install sound barriers), and localization of different sounds (e.g., do not place an HVAC near a storefront). While the set of tasks are by no means all-inclusive of Cerwén et al.’s Soundscape Actions, they provide the user enough background to get started using the software, no matter their level of experience.

Beyond the user, these tasks are also designed to facilitate the usability testing of the software and its functionalities. This allows for tracking of the user’s experience, linked to each functionality, and ensures each relevant part of the user interface is tested.

Table 2. Tracking user goals through themes, functionalities, and relevant user interface elements to produce an informed and testable task for users to complete.

	Broad User Goals	Create a relaxing atmosphere to promote city park usage
	Influencing Theme(s)	Raise sound awareness	Introduce wanted sounds
	Functionality	Must be able to add objects to a city scene
	User-Interface Element(s)	Add Object Menu	Placement Interface
	Individual Task	Place a bird feeder in a tree to attract birds

shows an example of how each task is informed from previous research and implemented into the interface for testing. A full list of tasks and their influencing themes and soundscape actions can be found in Appendix A.

2.4. Implementation: City Ditty, An Immersive Soundscape Sketchpad

In order to facilitate a fully immersive and interactive 3D audio-visual design environment, City Ditty was designed using the Unity game engine. The current version runs in a Desktop Virtual Reality format with monitor, headphones, keyboard, and mouse. A fully navigable 3D model of a city is included, which includes a simple user interface that allows the users to both traverse and interact with the city. An urban city centre was reproduced for this study as it provides a busy environment that can attract a lot of people, traffic on two sides of the city centre, and provides a sizeable open space for design. It was not modeled to emulate any particular location to avoid biases from comparison. The centre is divided into two green spaces with several sample scenes available, including an initial setup that includes a fountain, construction, a café with piano music under a pavilion, people talking, pub music at night, and an HVAC atop one of the buildings. The other green space remains empty to give the user a place to design their own space. The city centre has roads on two sides, with the side streets blocked off from cars to allow an interrupted pedestrianized section (Figure 1).

Audio is rendered through Unity’s built-in audio engine which produces a stereo output that simulates a 5.1 speaker setup through headphones. Due to the nature of ground-up designed soundscapes, an isolated recording of each individual sound source was introduced into the 3D virtual environment in mono. The Unity Audio Renderer relies on panning, distance attenuation based off logarithmic roll-off, and Doppler effect for moving sound sources to reproduce the spatial qualities. The individual audio files came from a variety of sources. Ambient sounds were locally recorded by our team using a first-order Ambisonic microphone (Soundfield ST350). Specific sound sources were close-miked with directional microphones and came from different sound clip archives such as freesound.org and professional sound recording engineers’ libraries. Sounds were mixed (e.g., indoor pub music with voices of patrons) by a professional sound recording engineer to provide realistic sound levels of individual sound sources while ensuring that the sound level of the overall soundscape remained comfortable.

Upon starting City Ditty, the user enters the sound awareness session. This places the user in the center of the city centre between the two green spaces, facing the populated side, and encourages them to explore the space while following the sound awareness tasks. During this exploration, users learn to add and manipulate different sound objects while learning more about soundscapes. Users are also introduced to different functions that help them find and visualize the different sounds together (Figure 1) and are encouraged to interact with the environment at different times of the day and season (Figure 2) to experience different contexts in the same space.

After the user has completed the directed training tasks, they are given three open-ended design tasks which ask the user to create three different soundscapes. These design exercises are based off of the 2019 workshop’s collaborative design session with the audio-only simulator [15,30]. The design exercises first ask the user to create an ideal soundscape. Second, they are asked to spoil this ideal soundscape as their second design. Finally, the user is asked to create a realistic soundscape based off a sample design, mixing the good with the bad. Once the sound awareness session is complete, users are free to use the free design mode to continue with their designs. Currently, only a single city model is utilized that consists of the city centre, which expands out to approximately six more blocks on all sides, although it supports importing other models. A short series of prototype videos are available for view online in the Supplementary Material.

3. Evaluating City Ditty with a Usability Study

The usability study was conducted to identify potential issues in the user interface and the instructions. Participants went through the self-guided sound awareness session, followed by a short questionnaire and exit interview to discuss their experience.

3.1. Methods

3.1.1. Participants

Six participants—as is standard sample size for usability studies [48]—were recruited from a range of backgrounds, including the urban planning, architectural, and other areas with an interest in sound, through university recruitment emails; only two were knowledgeable with soundscape research. Industry-experienced PBEs were excluded, in reservation for a more stable version. Participants were required to be aged 18+ with normal or corrected-to-normal vision and hearing.

3.1.2. Procedure

Participants sat at a computer which used a 34” ultrawide monitor (3440 × 1440 resolution) and were given AKG K240 open-back headphones with the volume set to a consistent, comfortable level. They first went through the self-guided sound awareness session that took them through the entire software experience on their own. Participants were allowed to ask questions regarding the software’s operation if stuck but were otherwise encouraged to go through each task by themselves. Software controls were provided in the self-guided session and were accompanied by a paper reference sheet available in front of their keyboard and mouse.

After the completion of the tasks which presented a new functionality (24 of 36 tasks), they were given an on-screen 1–5-point Likert usability question which asked, “How easy was this task to complete?”. If they were unable to complete the task, an option was given to indicate this. Upon completion of the self-guided session, they filled out a short-form user engagement questionnaire [43] with an additional question on virtual presence, as modified from [49] (Appendix B).

Finally, a short semi-structured exit interview was held to discuss participants’ experience with the virtual environment, the interface, and how they might see themselves using a tool such in the future (Appendix C). Researchers also asked about any extreme ratings (e.g., difficulty completing a task). Upon completion, participants were thanked and given $25 for their time.

3.2. Results

The analysis is broken down into four sections corresponding to task performance, usability issues, user engagement and presence questionnaire data, and a thematic analysis from the exit interview data. These consisted, respectively, of verification of whether users successfully completed the tasks or not, their self-reported difficulty ratings of these tasks, and the results of the user engagement scale which measures a person’s experience with the software based off cognitive, affective, and behavioural factors. The thematic analysis utilized previously identified themes, along with the user engagement and presence frameworks, to help build a richer understanding of the user’s experience.

3.2.1. Task Performance

The completion of the self-guided session took 55 min on average (range: 33 to 73 min). Task performance was measured through click tracking of 30 task-relevant functionalities per user, which were coded as follows: 3—self-discovered and used the functionality before introduced to it; 2—used the functionality correctly exactly when asked to; 1—did not use the functionality when asked to (but used it correctly in a later task); 0—never used it. Intended use was confirmed through observational data.

Overall, 171 tasks (out of a total of 180) were completed scoring a 2 or higher, indicating that users were able to complete the vast majority of tasks without any issue in just a few minutes. The remaining issues revolved mainly around the saving function, as we utilized two save buttons—Save, and Save as New (e.g., one overwrites the existing file, while the other starts a new file). However, multiple users overwrote old files that they should not have, or always created new files. These will be relabeled and given confirmation dialogue boxes.

Finally, there is a function that will allow time of day to progress without the intervention of the user (three virtual seconds for each real second). This start/stop clock button was not used at all by three participants, and rarely if ever touched again by the other participants, suggesting that this might be an unnecessary feature that could be removed.

3.2.2. Usability

Overall, the usability ratings were very high with a mean rating of 4.5/5 across all tasks and six users. A breakdown of the results is provided in Figure 3. Individual ratings of 3 or below were discussed in the exit interview. These lower ratings can be attributed to either unclear instructions (corrected between participants to avoid further issues), or difficulty with controls. Issues with controls primarily revolved around a clunky control scheme for rotating objects. Participants suggested using a control scheme closer to those in PC computer games, which will be implemented in the future.

3.2.3. User Engagement and Presence

User engagement measures different aspects of the cognitive, affective, and behavioural aspect of interaction [50]. When the UES is analyzed per O’Brien et al. [43], user engagement is broken down into four aspects of user engagement—focused attention, perceived usability, aesthetic appeal, and reward. Additionally, a single scale was used to measure Presence—the feeling of ’being there’ in there in the virtual environment. Ratings were generally positive, with perceived usability and reward receiving the highest ratings (Figure 4).

3.2.4. Themes

The exit interview data were transcribed and coded in the same manner as the 2019 workshop. However, rather than seeking new themes, supporting evidence for these existing themes were gathered through the study data. Additionally, the user engagement and presence frameworks were also utilized as themes to further understand the participants’ experience. These themes are presented in ways which may identify barriers, improve usability, identify new functionalities, and support user engagement.

Raising Sound Awareness

Participants reported a positive learning experience which changed the way they think about sound as it applies to their own work. E.g., [P03]: “It was interesting. I guess I haven’t thought too much about an ideal soundscape, and as I was making it, I realized it’s kind of subjective and impossible sometimes [to please everyone], you know. But yeah, I enjoyed that part”; [P04]: “So I feel like we are very careful to consider everything else when we’re designing a space, but you’re right. Noise has been something that’s like ‘oh it’s loud’ or ‘it’s not loud’ and that’s about the full extent of the thought that goes into it, so it was neat to actually experience the differences and be able to kind of compare that easily.”

Participants also commented on how such as tool could be used to raise awareness or support planning interventions. E.g., [P02]: “That’s what it would sound like with no cars and it’s all like, nice and peaceful. I feel like that would be helpful to get people on board with pedestrianizing more streets. Like, here is traffic noise with people screaming versus nice quiet streets without cars.”

Locating and Visualizing Sound Sources

Several functions allowed participants to visualize and locate sound sources. Participants appreciated the color-coded highlighting of different sound sources (Figure 1) to help visualize which objects produced sounds, along with the different colors to help differentiate between different categories of sounds (i.e., machinery, nature, and people). They also commented on the ease of locating sound sources through the Find button (which produced a line connecting the user to the sound) but suggested using a more visually appealing line.

Immersive Audio-Visual Experiences

This theme was split and coded using the presence and user engagement frameworks. First, the presence model is applied to help (1) model the virtual built environment (immersion) and (2) interpret the virtual built environment (plausibility). Next, the user must (3) interact—or engage—with the virtual built environment (user engagement).

Presence

According to the presence model, immersion—as facilitated through realistic audio and visuals—must be present to elicit a plausible interpretation of the world, which in turn, facilitates a realistic response from the user. All participants indicated that visuals were “good enough” but were quite impressed with the realism of the audio (including distance attenuation), which even compensated for the lower fidelity visuals. e.g., [P03]: “[The visuals] obviously weren’t hyper realistic, but I like it. It did what it needed to do. Basically, I was never limited by how it looked…It still felt pretty realistic… The sounds were really good. That kind of made up for the fact that it wasn’t like you’re looking at like an actual tree.” Participants further commented on the role of sounds in presence, creating the illusion of “being there”. [P01]: “You could really feel most of the sounds you hear in the city.” [P02]: I think the construction noise was spot on. [Laughing] I was like, oh, God, there’s construction. That’s [my city]!”.

The presence framework next suggests that if a suitable degree of immersion is provided within a plausible environment, people will respond similar to how they would in real life. Different aspects of this were identified when discussing the audio and visual aspects with participants. Participants were at times pleasantly surprised at the realism, noting how the environment responds to their actions in different ways. [P04:] “I moved the time of day [slider] before [the tasks] told me to and I was when I was like, ‘oh, I wonder if this will change the traffic or different things like that’, then sure, enough, yeah, that changes different things. Every kind of intuitive thought that I had was something that was covered later.”

However, this high contextual realism also set high expectations that made anything that seemed ‘off’ extra noticeable and distracting to the user. [P05]: “I mean, people don’t move, so it sometimes feels like it’s not standard… the cars move so I kept asking myself why doesn’t the person move? But you can hear them talk, so that’s a good start.”

To participants, the missing or slightly off aspects really stood out and were commented on with some disappointment but understanding that it was an early prototype. Beyond people and traffic, some participants pointed out that several pubs and cafes played music, but there were a lot of similar but silent businesses. Three participants listened for sounds from trees, with one even manipulating the wind intensity to see if there was a difference. Interestingly, P06 noted that although the instructions were in English, the people in the city centre spoke French, reminding them that they were in a virtual environment. These breaks in presence might distract the users from their tasks.

Two participants also enquired about some issues relating to the ecological validity of the simulations. P01 pointed out from personal experience that the sounds of traffic should be much more intense at various times of the day than the software provided. As such, while City Ditty can currently help PBEs with general sound awareness and general prototypes, it could later benefit from using real traffic and sound level data to facilitate fully fledged traffic studies and help facilitate more accurate design decisions.

Participants were also asked if it sounded like anything was missing from the city that would help make it feel more realistic. All six participants specified that occasional car honks and a greater variety of vehicles would be more realistic, although the absence of these did not bother people as much as the breaks in presence. Other moving objects such as bicyclists, animated pedestrians, and small animals would also be beneficial.

User Engagement

Once a suitable virtual built environment is available, what would encourage PBEs to regularly engage in soundscape design with it? Through the lens of presence, many examples were already identified which may hinder engagement. For example, breaks in plausibility disrupt their workflow, while blurry spatial audio may cause discomfort. However, when the environment responded to the user’s actions in realistic ways, this may generate interest and promote further exploration of other sound implications from both contextual and intentional design decisions. Such actions lay at the heart of this tool—to promote an interest in sound awareness, and to let PBEs explore and experience the acoustic consequences of their actions in a fleshed out urban context.

To help analyze this further, the user engagement model [38,43] is used to explore the different factors of user engagement. This includes aesthetic appeal (AE), focused attention (FA), perceived usability (PU), and reward (RW). This also incorporates and replaces the ‘Desire to interact with the virtual environment’ and ‘Concerns for operating new software and hardware’ themes, as participants have now had a chance to experience these firsthand.

Aesthetic appeal (AE)

Aesthetic appeal heavily overlaps with presence’s immersion. This qualitative data suggest a bidirectional relationship where greater aesthetic appeal can contribute to greater sensory immersion (and therefore presence), while higher immersion can also lead to greater user engagement.

Perceived Usability (PU)

Perceived Usability varied according to a user’s skill level. Participants P02 through P05 all rated this highly, as although they experienced several usability issues, they were quickly able to adjust and continue relatively unimpeded. However, P01 and P06 rated this lower for different reasons which gave valuable feedback.

P01 was already experienced in both soundscape and video gaming (including city building games), and as such, was looking for a more polished, less guided experience. Indeed, half-way through the sound awareness session, P01 stopped following the instructions entirely and started a free design of their own using a lot of the functions that had yet to be introduced. [P01]: “I don’t like tutorials [laughter]. I just wanted to say like, OK, OK, let me just play the game. I know the text is very important because you need to share some ideas about the soundscape, but I think some people will behave like me.”

After a 6-minute detour of rapid exploration and free design, placing dozens of sound sources and trees, P01 quickly clicked through most of the remaining tasks (having already completed similar actions on their own), completing the entire session in 33 minutes. [P01]: “after half an hour I became a bit like I didn’t want to do stuff anymore because it was a bit tiring, and I know that.” P01 reported that it was taxing because it took too long to navigate through the menus for rapid prototyping. While it was very easy for them to place multiple objects of the same type in rapid succession, it was slow to switch between objects. As such, several interface designs need to be modified to facilitate future advanced users.

On the opposite end of the spectrum was P06. They had zero history with navigating digital environments, playing games, nor did they have any experience with urban planning; they were a complete novice in every manner. They required a lot of guidance from the attending researcher and took a total of 73 minutes to complete the session—even skipping the final free design exercise. They also reported this as a very taxing experience, but for different reasons than P01.

P06 reported difficulty navigating the 3D space, got lost within the space, and often second-guessed themselves and asked for support and reassurance. Despite the very slow start, P06 was able to navigate themselves through the free design exercise at the end with very little intervention from the researcher, which indicates clear evidence of learning. [P06]: “[I’m] totally new [to virtual environments] and I think that was the most frustrating aspect of it… navigating the environment was really challenging”.

On the other hand, P06 shared that the menu system and the instructions were very clear—even a complete novice could follow the instructions, navigate the interface, and complete the required tasks. E.g., [P06]: “I found your menus simple... I guess everything related to the sound I found easy and enjoyable, so adding objects, modifying their volume, and the time frame when they’re active. That part was very simple to me”.

Overall, this feedback shows that City Ditty is on track; it can be followed by complete novices and operated by anyone with a single short training session. However, it still requires some modifications to support rapid prototyping after the initial learning phase to avoid becoming taxing to use.

Focused Attention (FA)

Focused attention refers to when users feel absorbed in the interaction and lose track of time. Users were generally observed to be highly involved in the tasks at hand, were interested in the activity, and rated this positively on the UES (Figure 4). E.g., [P04]: “It was pretty enjoyable [experience] actually. I definitely did get like lost in it. Like, the hour slipped by pretty fast, and it was also just kind of neat to experience those different sounds.”

That said, the UES data only gauge the overall impression at the end of the experience and is limited in that regard. However, FA can fluctuate during the course of the experience, which was not directly measured. Low periods of FA can be suggested when the users experienced breaks in presence or experienced usability issues which needed researcher attention. While P02-P05 had little problem operating the software on their own, P01 and P06 provided different insights which may help them focus better. As an advanced user, P01 expressed an interest in shorter visual vignettes with less text as a way to keep themselves moving forward with less reading, as they did not need the soundscape info—just a quick introduction to the functionalities. P06 on the other hand was often stumped by the very existence of some of the objects available for placement during the free design tasks, as they were not explicitly introduced during the sound awareness session. E.g., [P06]: “There are moments when I ask myself “what’s the point”, you know? Is it for sound? Is it for design? The “so what” aspect of it, which I think whoever is using it for their job will know that, but for me it was kind of annoying.”

Currently, placeable objects are listed in a menu with just their name and a ‘preview sound’ button available. As such, when presented with objects like a drinking fountain, P06 was confused as to how these are useful in soundscape planning. To help combat this, a short, inobtrusive description of each sound sources design benefits—and cautionary tales—could be provided. This could both inspire new design ideas while reminding designers of other drawbacks or regulations, without requiring further sound awareness sessions.

Felt Rewarding (RW)

A rewarding experience is crucial to developing user engagement and potential future adoption; if the experience is not rewarding, there is no reason to use it again. That said, despite some issues, all users found this to be a rewarding experience (Figure 4). The novelty of the tool played a major role in this, as it not only raised sound awareness, but it helped users realize that they COULD do more sound planning, in a very short period of time, with very little effort. However, to further explore repeated use by PBEs, it must be helpful in their professional work.

Integrate into Their Workload

What would encourage users to re-engage with the software for their own projects? When asked if they would consider using such a tool in the future, participants gave positive responses which related heavily to the Ability to explore different contexts in the same space theme. E.g., [P03]: “Yeah, I’m just thinking, if they’re planning to add a new commuter rail stop or something like a bus stop, what [sound] implications that might have?”

Participants appreciated the cause-and-effect aspect of designs and wished to see more functionalities related to that. E.g., [P06]: “Could you add a bike lane which would limit the width of your road? … also, potentially lower car traffic by having more public transit?” Traffic was a major point of discussion for some people, with a desire to manipulate the speed and density of the traffic, beyond the pre-set values.

Integration with other software was also brought up, as some participants wanted to import their own assets (e.g., city models, 3D objects, and audio recordings) that they are already familiar with. [P04]: “I’m picturing the SketchUp library of objects because I know those. There’s obviously a lot that are premade, but you can also upload your own.” More advanced functions were also requested that relate more to urban planning than just sound. E.g., [P05]: “if we’re thinking urban planning and investment in general, we could add a couple more objects and tools. You know, maybe paths. Yeah, people will use their paths more often than not. Yeah, so how do you plan your public space? That would be useful”.

3.2.5. Implications for Future Versions

Currently, City Ditty shows great promise for raising sound awareness and teaching the basics of soundscape design. It provides a simple interface that helps users visualize sound and quickly identify and experience cause-and-effects of sound designs. In its current state, it remains accessible to all users for learning and prototyping their own soundscapes. However, once the software permits outside files and software, the required user knowledge will jump considerably.

Allowing the use of outside assets (e.g., 3D models and audio) presents challenges in using standardized file formats and ensuring that the assets are optimized for real-time rendering of a virtual environment that provides the required levels of virtual presence. This is possible. However, it adds a layer of expertise back into the equation which most PBEs do not have a background in. For instance, audio clips should be close-miked to minimize leakage from nearby sound sources and reverberation from the recorded environment. Likewise, 3D models should have simple geometries to place less computational burden on the software; architectural models are often too large to run effectively in real-time rendered environments and must first undergo transformations for this. As such, various degrees of specialized expertise will always be needed to create an accurate reproduction of a specific project. In absence of this expertise, a large built-in library of sound sources can give the users more design choices with pre-set sound settings (e.g., playback times of the day, sound levels, and other acoustic properties), which can allow the user to jump straight into their sound designs without additional time, knowledge, or resource barriers.

Software interoperability is also important, as no one software is designed to do everything. For example, the addition of landscape/pathways will affect where people (and their respective sounds) will be produced. However, if users are importing their own architectural model, which software should the pathways be created or modified in, and what are the consequences for the other software? Important data created in one software should be easily transferable to the other. Otherwise, if these changes need to be done manually between software (e.g., add the path manually to both), this may lead to inconsistencies, wasted time, and frustration. On the other hand, if this data integration between the two software is smooth, the users could use what is most convenient for them, which may even facilitate new ways of collaboration. E.g., One person could change the architectural model using their existing skills and software (e.g., Autodesk’s Revit), while a second person sees the changes in real time remotely, or in virtual reality. Interoperability between multiple software inevitably requires future user-centered investigation to enhance the user experience, rather than create barriers.

4. Discussion

Through utilizing a user-centered design approach, this identified many functionalities that were suggested to serve the practical needs of PBEs by giving them intuitive and meaningful ways to interact with a virtual built environment. The usability study provided many insights regarding the evaluation of these functionalities, their interfaces, and the sound awareness session. Beyond these practical contributions, we continue to discuss (1) theoretical contributions via the interplay of user engagement and presence, including how it should influence the development and use of immersive soundscape tools. This is followed by (2) a discussion of the future applications of soundscape tools.

4.1. The Interplace between User Engagement qnd Presence

The results of the usability study suggest extending the user engagement to include the different factors from presence. Figure 5 describes how the different components interact.

First, Immersion and Aesthetic Appeal are identified to heavily overlap with, and feed upon, each other. Greater aesthetic appeal can lead to greater immersion, while greater immersion can lead to a greater aesthetic appeal. Although conceptually very similar, aesthetic appeal implies human judgement, whereas immersion is only defined as the technological ability to produce the audio, visual, and any other relevant information to be interpreted by the senses. Further interactions with immersion and user engagement are possible, but not yet explored.

Plausibility is seen to directly affect focused attention, as when the virtual built environment changes in a plausible way as a result of the user’s actions, this caught the user’s attention, often prompting undirected investigation into the cause-and-effect relationship(s) further. Similarly, when something did not respond as they expected, this also caught their attention, but detracted them from their tasks with some disappointment.

Plausibility is also seen to directly affect perceived usability. When the virtual built environment changes in a plausible way as a result of the user’s actions, this allows users to experience the cause-and-effect of their actions, and thus, further encourage exploration of their designs and increase their sound awareness. Furthermore, participants requested more functionalities based around this cause-and-effect, such as how would traffic volume change if they were to add a bike lane or public transit. Similarly, when the virtual built environment does not respond as the users know it should, this may call into question the perceived usefulness of the software.

Finally, plausibility also affects reward. When the environment changes in a plausible way as a result of the user’s actions, this signals to the user that they now understand the implications of a particular design or soundscape intervention technique. Through the software, they could both utilize this new knowledge in their own work, and become better equipped to demonstrate and explain these difference to their stakeholders.

4.2. The Future Application of Soundscape Tools

We use the term sketchpad for City Ditty as this implies a rough drawing or general outline of a soundscape, as a sketch can be very realistic and give a strong idea of what the finished product may look and sound like. By using this medium quality of prototyping, this encourages the user to design with sound in mind, rather than to be constrained by it, as existing acoustic modeling software may influence. Following this, we suggest that sketchpads should be built to facilitate a plausible urban ecology at design time. Such an approach would serve to both remind the designer of the human experience, while guiding user engagement through its different applications.

Provide a Plausible Urban Ecology

This new term is proposed to help capture the crucial human experience within the virtual built environment (Figure 6). This combines different aspects of the provide an immersive audio-visual experience theme, with aspects of urban ecology.

Verma et al. [51] defines urban ecology as “the study of the relationship between living organisms and their environment, their distribution and abundance, the interactions between the organisms, and transformation and flux of energy and matter, in an urban area.” When applied to soundscapes, this can include the study and relationship between living organisms and their sound environment. Since soundscapes (A) play a vital role in our health and everyday lives, and (B) our interpretations of them are inseparable from the built environment, we suggest highlighting its relevance within the broader urban ecology field. Second, Plausible was chosen due to its prominent role in the provide an immersive audio-visual experience theme, as well as its high influence on facilitating user engagement (Figure 5). As such, rather than simply prototyping a built environment, we hope that prototyping a plausible urban ecology will highlight the importance of the auditory experience, its need for a highly plausible virtual built environment for better understanding of cause-and-effect under different contexts, and how sound is inseparable from the other aspects of urban planning and urban ecology.

After a plausible urban ecology is provided, users must engage with it in meaningful ways. Figure 7 illustrates three main categories of applications for use, along with their relevant considerations based off this study.

Educational

Through exploring different urban contexts and how individual sound sources are affected by their environments, City Ditty has demonstrated that anyone can learn and apply the basics of soundscapes in a very short period of time. That said, only a sample of sound awareness tasks have been provided to explore such a tool, leaving much room for growth for training future PBEs and other interested parties. Currently, there are many soundscape actions [47] remaining which could be implemented into modular lesson plans. However, some soundscape actions will require further improvements to the plausible urban ecology to create a proper impact on learners. For example, City Ditty does not currently model sound occlusions (i.e., sounds being blocked/mitigated by walls). As such, users will not be able to fully appreciate what some sound interventions would sound like (e.g., moving an HVAC from the front of a building’s roof to the back, away from the entrance, or installing a sound barrier). Similarly, City Ditty is not designed to make large landscape changes. As such, some lessons would require further functionalities and possibly integration with other software to support them all.

Support Rapid Prototyping

A sketchpad should be easily accessible to facilitate an early wave of first-time adopters. For some purposes, these sketches may be enough. However, often these sketches will need to be passed off to further consultants to complete the work. Such higher skilled users will require more advanced functionalities to better integrate sketches into their existing projects, and at times, produce the required high level of accuracy and precision for insightful positive changes to our urban soundscapes. Such high-skilled PBEs may be slow to train and grow in numbers but will be invaluable future leaders of this field.

Share with Others

City spaces should be designed with the needs of the city users in mind. They can be used for early public consultations and getting feedback from a wide range of stakeholders, including citizens. Sketchpads are also suggested to be effective collaborative design tools for both highly skilled PBEs, as well as less skilled collaborators and stakeholders to help determine how urban spaces will be used and should sound. Finally, these sketches can be used to help make marketing material and persuasive sales pitches by providing immersive experiences.

4.3. Divergence from Existing Software

A major diverging aspect of City Ditty is that it places the designer in situ to experience their designs. While many reproductions have done this to collect soundscape evaluations, they have lacked a way to quickly modify sound sources and listen to the result under different contexts. Similarly, by providing a real-time audio-visual environment, this begins to address the need for multisensory soundscape tools, which was identified as one of the top priorities and challenges for soundscape research as a field [17,18,19].

By focusing on designing with a focus on sensory experience without being limited by sound levels, this produces a low learning curve to do simple soundscape sketches, with no experience necessary. We believe such a tool is crucial for getting PBEs comfortable with the thought of working with sound and may prime them for further learning.

Importantly, sketchpads are not a final design. PBEs are encouraged use their sketches to help generate discussion with acousticians, who can then work with these other PBEs to help refine a design and meet regulations based on physically accurate acoustic measurements and models. This sketchpad does not attempt to replace acoustic expertise, but rather complement it for early prototyping.

4.4. Limitations

First, this study has focused on design time, rather than soundscape evaluation time. An inevitable future step would be to implement a method of evaluating a prototype soundscape within City Ditty itself (e.g., data collection through a public consultation, either locally or remotely) before selecting a final design. Previous research suggests that there is a positive future for evaluating reproductions in a laboratory setting (see Tarlao et al. [52] for a review). However, at such an early stage, this would distract from improving the design experience for PBEs. Second, the proposed extended user engagement model (Figure 5) is derived from the qualitative exploration, and as such, remains to be tested quantitatively in future research on larger group of participants. That said, this is a typical sample size for usability studies, which was our main objective for this study. Follow-up studies are still required from a wider range of participants.

5. Conclusions and Future Work

The evaluation of City Ditty demonstrated its ability to (1) simulate realistic soundscape environments; (2) provide basic training to PBEs by raising their sound awareness and teaching them some fundamental soundscape principles; and (3) facilitate rapid prototyping of their own soundscape designs. Users also reported that (4) they could use this software to help sell their sound-based design ideas to stakeholders, while (5) we believe this software will help lower costs by providing an immersive but inexpensive software experience for facilitating all of the above points.

The evaluation of this software was heavily conducted through the lens of facilitating user engagement. Through a rich qualitative approach, a proposed extension of the user engagement framework was presented which includes the different factors from presence as facilitating factors of user engagement with immersive tools.

Three main areas of application were identified for future work, including education, supporting rapid prototyping by PBEs, and manners of sharing with others: each warranting their own user-centered processes for extending its usage. Current plans focus on (1) increasing the audio/visual capabilities by implementing binaural audio and extending to head-mounted VR; (2) further testing with more experienced professionals in both individual and group settings to obtain a more focused direction for identifying and prioritizing improvements towards adoption; and (3) facilitate adoption by importing BIM architectural files, focusing on software interoperability to help PBEs use City Ditty for their own projects.