Wireless Channel Assessment of Auditoriums for the Deployment of Augmented Reality Systems for Enhanced Show Experience of Impaired Persons ^†

Abstract

Auditoriums and theaters are buildings in which concerts, shows, and conferences are held, offering a diverse and dynamic cultural program to citizens. Unfortunately, people with impairments usually have difficulties in fully experiencing all the provided cultural activities, since such environments are not totally adapted to their necessities. For example, in an auditorium, visually impaired users have to be accompanied to their seats by staff, as well as when the person wants to leave the event in the middle of the show (e.g., to go to the toilet), or when he/she wants to move around during breaks. This work is aimed at improving the autonomy of disabled people within the mentioned kinds of environments, as well as enhancing their show experiences by deploying wireless sensor networks and wireless body area networks connected to an augmented reality device (Microsoft HoloLens smart glasses). For that purpose, intensive measurements have been taken in a real scenario (the Baluarte Congress Center and Auditorium of Navarre) located in the city of Pamplona. The results show that this kind of environment presents high wireless interference at different frequency bands, due to the existing wireless systems deployed within them, such as multiple WiFi access points, wireless microphones, or wireless communication systems used by the show staff. Therefore, radio channel simulations have been also performed with the aim of assessing the potential deployment of the proposed solution. The presented work can lead to the deployment of augmented reality systems within auditoriums and theaters, boosting the development of new applications.

1. Introduction

According to the latest World Health Organization (WHO) reports [1], 2.2 billion people suffer from some type of visual impairment. Diseases that can cause severe visual impairment or even blindness are glaucoma, diabetic retinopathy, or macular degeneration.

In the literature, numerous studies have presented assistive devices for people with visual impairments; specifically, smart glasses for the recognition of obstacles that send information to the person with disabilities through audio [2]. In this work [3], a detection intelligent stick is added to the smart glasses for objects that are outside the user’s viewing area. This stick provides fall detection, sending the GPS coordinates of the user to a predefined contact. In indoor environments, the solution is complicated, as it requires the deployment of ZigBee or BLE (Bluetooth Low Energy) sensors to determine the paths that people with disabilities have to follow [4,5].

In environments such as auditoriums or theaters, there are usually devices for people with hearing impairments that may be mild (hearing devices) or severe (modulated frequency systems transmitters for users wearing cochlear implants), but no guidance devices are available for people with visual disabilities. This work wants to overcome this barrier, and for this, a network of wireless sensors and body area sensors connected to an augmented reality (AR) device like Microsoft HoloLens smart glasses [6] is proposed to improve the autonomy of people with these disabilities as well as their experience in this type of environment. The deployment will be carried out in the Baluarte Palace of Congresses and Auditorium of Navarre, located in the city of Pamplona, in the main auditorium that is inside. The objective is therefore that people with visual disabilities can move freely throughout the auditorium, without the help of hostess services.

The radio characterization has been carried out through simulations based on 3D-ray-launching software (3D RL), where the dimensions of the auditorium have been considered together with the dielectric properties of the obstacle materials, such as the dielectric constant and the conductivity. The results show that optimized radio planning could be a key issue, due mainly to the interference levels present within the auditorium.

2. AR Solution for Visually Impaired People

AR has evolved significantly since the 1960s from the technological and business standpoint. Recently, it has been pointed out as a key technology enabler for Industry 4.0 [7] due to its potential applications. Nevertheless, its use to enhance leisure activities is still very incipient. In the case of people with visual impairments, AR has the potential to enable a wide range of user-friendly and real-time applications, like providing information and assistance (e.g., vibrations for haptic feedback, sign-reading assistance [8]), locating items, guidance and navigation, interacting with internet of things (IoT) devices, or augmented collaboration.

Among the different commercial AR devices, Microsoft HoloLens smart glasses were chosen since they are one the most promising AR headsets on the market [7]. HoloLens contains a custom-built holographic processing unit (HPU 1.0), 2 GB of RAM and 64 GB of Flash, embedded sensors, see-through holographic lenses, automatic pupillary distance calibration, and human-machine interfaces (voice support, gaze tracking, spatial sound, and gesture recognition). In addition, the smart glasses can be connected to other devices through WiFi (IEEE 802.11 ac) or Bluetooth 4.1, and its battery lasts approximately 2 to 3 h with active use and up to two weeks in stand-by mode.

3. Radio Characterization of the Auditorium

In order to characterize the radio channel within the proposed auditorium scenario, intensive measurement campaigns, both to characterize the radio channel and to assess the present interference levels have been carried out. In addition, simulations by means of the 3D RL in-house developed code have been performed. The algorithm is based on geometrical optics (GO) and geometrical theory of diffraction (GTD). The code has been programmed in Matlab, and an exhaustive description of its operation procedure is presented in [9].

3.1. Auditorium under Analysis

The scenario object of the radio-planning studio is the Baluarte Congress Center and Auditorium of Navarre, which is located in the city of Pamplona. Baluarte is composed of different rooms, among which the main auditorium stands out for its size, as can be seen in Figure 1. It has a total capacity of 1568 seats, which are divided into 1036 in the room and 532 in the auditorium box. It has dimensions of 65.26 m × 30.58 m × 16 m (length × width × height). The room is paneled in beech wood and the seats are made of leather.

Table 1. Material properties for ray-launching simulations (at 2.4 GHz).

Material	εr	Conductivity (S/m)
Beech wood	2.0473	0.0527
Seat leather	1.2	0.24
Concrete	8.5	0.02
Metal	4.5	37.8×106

shows the parameters of the materials that the auditorium is composed of and which have been considered in the simulations.

During the measurement campaign, several spectrograms were measured to analyze potential interfering systems in the auditorium and thus be able to plan the placement of the wireless sensor network. Such spectrograms were obtained with an Agilent’s FieldFox N9912A spectrum analyzer. In Figure 2 and Figure 3, the spectrograms are shown in the 2.4 GHz and 5 GHz bands, where the amount of interfering systems in Baluarte mainly shows WiFi channels in both bands. This is important to keep in mind, since Microsoft HoloLens glasses operate both at 2.4 GHz and 5 GHz using IEEE 802.11ac and Bluetooth 4.1.

3.2. Simulation Versus Measured Results

In this section, the simulation results obtained with the 3D RL tool are presented in order to compare them with real power measurements obtained with the spectrum analyzer for the frequencies of 2.4 GHz and 5.5 GHz, at different points of the auditorium to fully characterize it. In Figure 3, the Baluarte auditorium model created with the 3D RL is presented, where the transmitter (represented by a green square) is located on the audiovisual cabin shelf, 1.55 m above the ground, and the measurement points (represented by blue circles) are at the height of the seats, 1 m above the ground.

The transmitter is a VCO (voltage controlled oscillator) that operates with a different transmission power according to the operating frequency. At 2.4 GHz, the ZX95-2500 model was used, where the maximum VCO power is 8.38 dBm, while for 5.5 GHz the ZX95-5400 model was employed, where the maximum VCO power is −4.3 dBm. For both cases, a dual-band antenna was used, which works in the ranges 2.4–2.5 GHz and 5.2–5.8 GHz.

Table 2. Ray-launching simulation parameters.

Parameters	Values for 2.4 GHz	Values for 5.5 GHz
Transmitted power	8.38 dBm	−4.3 dBm
Antenna type/gain	Monopole/0.3 dB	Monopole/3.74 dB
Launched rays resolution	1 degree	1 degree
Permitted maximum rebounds	6	6
Cuboids size (mesh resolution)	50 cm × 50 cm × 50 cm	50 cm × 50 cm × 50 cm

shows a summary of the parameters configured in the simulations.

Figure 4 shows compares the measurement and simulation results for 2.4 GHz and 5.5 GHz. As shown in the figure, for both working frequencies the 3D RL method is validated due to the similarity between measurements and simulations. For the higher operating frequency, lower power reception levels are obtained, mainly due to the increase in propagation losses and the lower transmission power. Specifically, at 2.4 GHz a 0.78 dB mean error and a 0.6 dB standard deviation are obtained, while for 5.5 GHz a 2.38 dB mean error and a 2.7 dB standard deviation are obtained. These results indicate that the 3D RL tool is valid for planning the deployment of the wireless sensor network.

4. Discussion

This paper presents a first assessment of the wireless channel for both 2.4 GHz and 5 GHz operation frequencies. Measurements within the auditorium under analysis show that the environment presents high interference levels at both frequency bands, due mainly to the WiFi networks of the auditorium. In order to gain insight into the radio channel behavior, simulations by means of the in-house 3D RL algorithm as well as RF (Radio Frequency) signal propagation measurements have been performed with the aim of studying the viability of deploying WSN/WBAN connected to an AR device (Microsoft HoloLens smart glasses). Future works will consist of the development of applications for the improvement of the autonomy and the show experience of visually impaired people.