Semi-Immersive Virtual Reality as a Tool to Improve Cognitive and Social Abilities in Preschool Children

Featured Application

The system and the activities presented in this manuscript can be successfully employed for the empowerment of social abilities in pre-school children, promoting inclusion and preventing isolation. Potential applications are also the improvement of weak functions in the at-risk population (e.g., on children with neurodevelopmental disorders and children with language and communication disorders).

Abstract

Virtual reality (VR) creates computer-generated virtual environments where users can experience and interact in a similar way as they would do in real life. VR systems are increasingly being used for rehabilitation goals, mainly with adults, but also with children, extending their application to the educational field. This report concerns a study of the impact of a semi-immersive VR system in a group of 25 children in a kindergarten context. The children were involved in several different games and activity types, specifically developed with the aim of learning specific skills and foster team collaboration. Their reactions and behaviors were recorded by their teachers and by trained psychologists through observation grids addressing task comprehension, participation and enjoyment, interaction and cooperation, conflict, strategic behaviors, and adult-directed questions concerning the activity, the device or general help requests. The grids were compiled at the initial, intermediate and final timepoint during each session. The results show that the activities are easy to understand, enjoyable, and stimulate strategic behaviors, interaction and cooperation, while they do not elicit the need for many explanations. These results are discussed within a neuroconstructivist educational framework and the suitability of semi-immersive, virtual-reality-based activities for cognitive empowerment and rehabilitation purposes is discussed.

1. Introduction

Virtual reality (VR) has been defined as the “use of interactive simulations created with computer hardware and software to present users with opportunities to engage in environments that appear and feel similar to real-world objects and events,” [1]. In computer science, definitions of VR emphasize the possibility and ability to combine software with hardware to create a fully immersive experience [2]. In health care, the term is used in a slightly different way; to describe both non-immersive and immersive experiences that create an alternative reality [3]. Its main applications seem to be in the field of motor learning, whereas only few randomized controlled studies address its effects on other cognitive functions [4]. Its advantages rest on the possibility to fine-control, personalize and hierarchize tasks. Furthermore, it gives the possibility of providing multimodal feedback in real time; indeed, the stimulation of multiple perceptual channels, implemented by the use of auditory and visual feedback, increases the patient’s awareness of his performance and allows a sense of global wellbeing [5].

Indeed, VR represents a very promising technology for neurorehabilitation. Computers elaborate a simulation of the real world using real-time graphics, through which the subject can interact with the environment [6]. There are different types of VR, including (a) immersive virtual reality; (b) desktop virtual reality; (c) projection virtual reality (or semi-immersive virtual reality); and (d) simulation virtual reality [7]. VR ranges from non-immersive to fully immersive, according to the degree to which the user is isolated from the physical surroundings while interacting with the virtual environment [8]. Usually, immersive systems involve computer interface devices, such as head-mounted displays (HMD) or projection screens surrounding the subject (cave systems), fiber-optic wired gloves, position tracking devices, and audio systems providing 3D sound. Immersive virtual reality in particular provides an immediate, first-person experience and a deep “sense of presence” [9], i.e. the perception to be immersed in a different world created by the components of software and hardware [10]. The level of presence is a subjective feeling and depends on user experience [11]. Compared to immersive VR systems, semi-immersive devices do not provide the constant update of the visual information according to the participant’s head movements. On the other hand, they are more immersive than typical 3D monitors, particularly in terms of the range of sensory modalities accommodated [12,13].

The use of VR in neurorehabilitation has grown considerably, and experimental evidence suggests that this technology could favor functional recovery in neuropsychological disorders [8,14]. With respect to other tools in neurorehabilitation, VR has a number of peculiarities. Among others, the possibility of creating tailor-made training programs, so that the rehabilitation process can be individualized and adapted to each patient’s specific needs. Moreover, VR can foster active involvement, thanks to the possibility of creating new and appealing environments [14]. Very often, VR gives therapists the possibility to individualize treatment needs, as well as the opportunity for repeated learning trials, while gradually increasing task complexity and/or decreasing therapist support and feedback [1].

Immersive and semi-immersive VR systems provide the opportunity to practice cognitive and motor activities that cannot be practiced within the clinical or the educational environment, performing simulations of real-life scenarios and activities [5,15]. In many cases, clinicians need special software development tools for the design and coding of interactive simulated environments to achieve specific rehabilitation goals [1].

VR also offers the possibility to test the patient’s progress within controlled, ecological, and secure testing environments, that reproduce the crucial characteristics of the real world and are selected ad-hoc for each patient and situation (e.g., [16,17]). In experimental settings, VR systems allow researchers to design dynamic and realistic environments (virtual environments or VE), while monitoring behavioral and physiological responses [18]. The high degree of control allowed during the investigation of the cognitive and behavioral components of a certain skill is an additional advantage.

It has been demonstrated [19] that the patients are able to transfer what they have learned from VE to real life. Spatial navigation skills have been the object of several studies involving VR (e.g., [20,21]. The results of such studies suggest that the mental representations of space in VE resemble those implicated in the navigation of the real world. Montana and colleagues’ systematic review [14] has shown improvements in spatial memory after practicing with navigational tasks in VR. Most importantly, it has shown a transfer of the improvements to more general aspects of spatial cognition. Both immersive and non-immersive VR systems have been shown to improve navigation and orientation abilities. It has even been proposed that the improvement observed in visual-constructive abilities, attention and upper limb motricity could be due to the so-called shadow effect (i.e., the patient’s shadow on the screen while performing VR training) of the immersive system [22]. At the neurophysiological level, the mechanisms through which VR works are only partially understood. It is hypothesized that VR entrains the same neural pathways that are involved in motor learning and motion-related cognitive processes [23]. VR training thus seems to promote brain plasticity through mechanisms related to the reactivation of brain neurotransmitters, and its results can be even better than those obtained by conventional treatment [24,25].

As to the applications of VR with children, most have addressed motor problems [26], for instance in cerebral palsy [27,28], or cognitive, social and emotional problems, especially in autism spectrum disorders [29,30,31]. However, VR applications are not restricted to neurorehabilitation and more and more experiences involving its use are found also in educational contexts [32].

The general psycho-educational framework is that of neuroconstructivism, where cognitive development occurs through the pro-activity of the child in exploring, manipulating, and interacting with his/her environment [33]. According to the theory of social constructivism [34], moreover, the learning environment should encourage the pupils to collaborate and participate actively, experiment, share and develop ideas, use language to reason, plan and reflect on one’s actions.

In Richard et al. [35], it has been suggested that immersive virtual reality technology provides an alternative educational process by providing a knowledge-building experience. It is crucial that learning goals and solutions are established in collaboration with teachers [36]. The authors underscore that individual factors like age, gender, computer experience, psychological factors, cognitive and learning styles are likely to strongly affect learning outcomes, as well as technology-related factors such as immersivity, so that empirical studies are needed to determine which characteristics of virtual environments can really be pedagogically exploited. Less recent, but seminal studies, such as Winn’s [37], suggest that among the main features contributing to learning, there are free navigation and first-person point of view, the manipulation of the relative size of objects in virtual worlds, the transduction of otherwise imperceptible sources of information, and the reification of abstract ideas.

The conceptual framework known as TEL—Technology Enhanced Learning—suggests that technology can help the construction of new rules through the interaction with data in the learning environment. Indeed, games encourage exploring, the exchange of ideas, communication and decision making [38]. Games offer an invaluable opportunity for learning. The players of a game have to interpret images, sounds and actions [39]. They need to understand and learn what Gee calls the “internal design grammar” [40] of the game, forming hypotheses, adjusting their behaviors according to those hypotheses and, based on feedback from the virtual world, accepting or revising the hypotheses. Thus, games, serious games, or edugames, represent a unique opportunity to pursue rehabilitative and educational goals, without necessarily involving the children in long and effortful therapy sessions, but rather exploiting their own interests.

Although it has been stated that motivation is a key factor in the success or failure of education [41] and that fun and passion are key ingredients of the learning process [42], there are only a few studies that show that game technology brings substantial benefits [43,44,45]. Various peripheral devices have been used in such projects, including head-mounted display gear, data gloves, or body suits, and employing different techniques from specially designed glass cubicles to wall projection. However, practical concerns and limitations, first of all related to the high costs of such devices, but also to teachers’ and educators’ difficulties in using them, restricted dissemination of this technology in K-12 and higher education settings [45]. In more recent years, however, costs have begun to decrease and technology has become more user-friendly, so that more studies are being conducted in educational settings also; yet, most studies involved older children or university students, and studies on preschoolers are very rare. Nonetheless, some studies using VR to stimulate collaborative behaviors [46], support art education [47] or vocabulary learning [48] showed positive results on both learning and motivation (the most frequently reported concern, usually from parents and teachers, is about side-effects or addiction in the use of digital devices).

The project in which the present study was included envisaged the creation of an intelligent space in which preschool children and caregivers could experiment with a wide range of activities (see [49,50]). The setting and the playing sessions were organized so as to stimulate particular functions and skills, and foster team collaboration, as well as group integration. The goal was playing together to learn while having fun. Through the direct observation of caregivers and psychologists, we also wanted to characterize the children’s behaviors during the playing sessions, in order to describe the impact that the system had on their emotional states and on their interactions with the environment and with the people involved in the educational activity.

The educational principles that lead the choice and the design of the activities and games (see [32]) were:

• Define activities that can cover the whole range of neuropsychological functions, and can adapt to different children and different development profiles
• Choose activities that stimulate children’s curiosity, motivation, creativity by promoting inclusion through collaboration (cooperative learning) and communication
• Define VR-based activities that can be seen as an empowerment rather than as a transposition of traditional activities (see [51]), emphasizing the aspects of active construction of knowledge and competence.

Given the requirements, Nirvana 1 system (BTS Bioengineering, Italy) was selected as a playing environment. Nirvana 1 is indeed a semi-immersive virtual reality device, where one or more subjects can interact with virtual activities projected on a wall or on the floor, without the use of markers or other sensors placed on the body. The system is also equipped with a platform for the design of activities, in terms of graphic effects and feedback. The system has already been used in some rehabilitation studies [5,15,30], mainly addressing adult patients and aimed at motor rehabilitation.

During the playing sessions, the children were led and helped by their teachers and by two trained psychologists, who filled an ad-hoc-constructed observation grid for each of three timepoints: the beginning, midpoint and the end of each session. This grid was used to record the children’s reactions and behaviors during activity, and it was constructed so as to be able to highlight play and social behaviors as well as problem-solving attitudes and skills (in a similar way as other instruments created to describe preschoolers’ behavior during play with toys [52]), and their changes through session time.

The aims of the study were multi-faceted:

The main aim was to observe the overall impact of the activities on both cognitive and social processes, including interaction, collaboration, participation, conflict management and strategic behavior, but also to give some insights on accessibility/usability of the system, as emerging from the children’s observed behaviors (showing understanding of the functional principles of the games, asking questions about the proposed activities and requesting help from the adult observers).

Secondly, we wanted to describe the impact of the different kinds of activities and the different processes that could be stimulated and initiated by them. Indeed, the various activity types and the different games within each typology were expected to induce different reactions and different social dynamics, due to the varying degree of communication, turn taking and shared strategies required by the activities.

Finally, we wanted to describe how this impact is modified, by increasing familiarity with the games and activities from the beginning to the end of the session. Again, this aspect was investigated both at a general level, relating to the system as a whole, and at the level of single activity types. In general, it was expected that the comprehension of the game structure and function, as well as the strategic approach, could increase through time, and that also the ability to cooperate and positively interact would increase as a consequence of growing confidence and understanding. These changes, however, were expected to follow different trends for the easier and the more difficult activities, due to the different amount of time and effort needed for their comprehension and progressive mastering. Therefore, we expected that the interest elicited by the most difficult and complex games could decrease over time, at least for subgroups of children, and that intra-peer conflict could arise as a consequence of greater mastery of the activity and increased self-confidence in at least some of the children.

The present contribution describes observed effects at the group level as a first step, and as a pre-requisite in the validation of the instrument and preceding the investigation of its effects at the individual level. No attempt was made, at this point, to describe the effects of the single activities on the targeted functions and abilities. All these further steps are envisaged for successive studies in both clinical and educational settings.

2. Materials and Methods

2.1. The NIRVANA System

NIRVANA 1 (BTS Bioengineering, Italy) is a markerless system that allows the total immersion in a virtual environment, without limiting or altering the freedom of interaction and the human motion. It projects virtual environments on a wall or on the floor and one or more subjects can interact with these environments through simple movements. The activity is supported by a high sensorial, visual and auditory stimulation, that engages the user.

The system is equipped with a workstation, two optoelectronic infrared cameras and two projectors 4000 ANSI lumens (one for the wall and one for the floor projection), a webcam and a Dolby surround system. The configuration of the system used in this work is shown in Figure 1. A1 and B1, the optoelectronic camera and the projector, respectively, for the floor activities, were installed on the ceiling in the middle of the active area; A2 and B2, the optoelectronic camera and the projector for the wall activities, were placed 2.5 m high. The required total electrical power was about 3500 W.

The interaction with NIRVANA is straightforward: whenever infrared rays emitted by the optoelectronic infrared camera intercepts a body or a part of it, or even an object, an event occurs. This means that multi-touch and multi-subject approaches are possible.

The NIRVANA system is supplied with a graphic environment for the design of exercises; the software section is named Contents and it allows one to define new exercises according to six different typologies, with the following features:

Sprites: in this kind of exercise, it is possible to define a landscape and a variable number of objects, placed in user-selected locations (the user can specify X and Y coordinates with reference to the background). For each element, it is possible to define a size. These objects have two states and switch from one state to the other (another image or an animation) when touched. The user can define the timeout between different statuses of the element.

Particles: in this exergame, body movement removes the elements that cover the underlying background image. When the movement stops, elements cover the background again.

Reveal: in this kind of exercise there is a landscape with four states (images or animations with or without sounds). Changes in these graphic layers are allowed. The transition from one state to the following is defined by the adjustable percentage of area covered with the movement. If the movement stops, the sequences go back to the previous state. The default effect is water movement.

Move to: in this exercise it is possible to set a landscape and user-defined elements (e.g., dots, flowers, leafs). When an area is touched, elements gather in that area. A default activity of the system with these features is named “Dog” that can be customized by changing wallpapers.

Move away: this kind of exercise is similar to the previous one, but elements escape from the touched area (this typology has not been used for the present study).

Follow me: moving items should be tracked and an effect occurs when you reach the target. A default activity with these features is “Whack-a-mole”.

2.2. Design of Activities

A series of activities have been defined, either developing completely new activities and games, by adding images, videos and audio-files to the pre-existing activities and defining new tasks and goals, or (in the few cases where customization options were limited) using the activities and games already provided by the system, but re-organized and structured according to a precise educational and cognitive framework, so as to stimulate the targeted functions in the most effective way. Adjustable parameters have been set so as to redefine the tasks and model the activity according to very specific cognitive goals. In many cases, structural or functional adaptation was aimed at extending individual activities to group activities, where each individual has a specific task or needs to take turns and/or monitor/interpret the other individuals’ actions and intentions to modulate his/her own activity.

The creation of an immersive environment with wall and floor projection offers the possibility to carry out activities that can involve a medium-large group (5–10 children), with a variable need of mediation on the caregiver’s side, according to the planned activities. Access to the activities is not determined by the age of the children, but it is rather tied to the contents proposed by the caregivers and to the organization they decide to impose on the activity itself.

From the cognitive point of view, the most stimulated abilities were gross motor-praxic and fine motor coordination skills, perceptual, attentional and memory functions, along with problem-solving skills. Both wall projection and floor projection were used in the design of the activities, trying to exploit their specific characteristics and potentials, in order to have a greater impact on both motor demands and collaboration/strategic planning requirements at the group level. Indeed, floor projection allows for more complex motor tasks and for more structured group arrangements and coordinated motor activities. However, wall projection allows for the greater stimulation of fine motor coordination of movements performed with arms and hands.

Table 1. Exercise Typologies, implemented activities, type of system projection and neuropsychological functions involved in each activity.

Exercise Typology	Activity	Projection	Neuropsychological Function
Sprites	Eggs	floor	Visual discrimination Motor coordination
Sprites	Sound environment	floor	Auditory discrimination Auditory-visual-matching
Sprites	Old MacDonald Had A Farm	floor	Auditory discrimination Auditory sustained attention Executive functions (Planning and Inhibition) Motor coordination
Sprites	Musical canon	wall	Auditory working memory Auditory discrimination Auditory sustained attention Auditory divided attention
Sprites	Musical puzzle	wall	Auditory discrimination Auditory working memory Executive functions (Planning)
Sprites	Jam Session	wall	Auditory discrimination Auditory sustained attention Auditory divided attention Executive functions (Planning) Auditory working memory
Reveal	Water Lily Pond	floor	Motor coordination Proprioceptive awareness Executive functions (Inhibition)
Particles	What’s hiding?	floor/wall	Visuospatial attention visual-motor integration Lexical access
Follow Me	“Whack-a-mole”	floor	Visuospatial attention visual-motor integration
Move to	Dog	floor	Motor coordination Proprioceptive awareness Executive functions (Planning)

provides a list of the different task typologies that are provided by the system, along with a description of the specific tasks that were developed for the project.

2.2.1. Typology “Sprites”

The “Eggs” activity was designed by choosing different egg types and different arrangements (Figure 2A). Moreover, a full educational framework was defined, in order to stimulate visual-motor and planning functions in a systematic and meaningful way. In this framework, children play according to defined rules and times, within an educational framework. The goal is to follow increasingly complex tracks with the steps. A starting level is provided to familiarize them with the task. Subsequent presentations include different egg tracks, distinguished by color and sound feedback (

Table 2. “Eggs” activity arrangement. The sound effects “Broken shell 1” and “Broken shell 2” represent two different sound effects.

Name	Number of Eggs	Arrangement	Visual Effect	Sound Effect	Background
Eggs 1	10	a row in a straight line	Bullseye	Broken shell 1	Green lawn
Eggs 2	20	two parallel zigzag rows two types of eggs, different in color	Bullseye	Broken shell 1 Broken shell 2	Green lawn
Eggs 3	20	two irregular rows two types of eggs, different in color	Bullseye	Broken shell 1 Broken shell 2	Green lawn
Eggs 4	20	random arrangement two types of eggs, different in color	Bullseye	Broken shell 1 Broken shell 2	Green lawn
Eggs—Find it 1	20	The eggs are scattered on the lawn. They camouflage with the background	Bullseye	Broken shell 1	Pebbles floor
Eggs—Find it 2	10	Smaller and better hidden eggs are scattered on the lawn. They blend in the background.	Bullseye	Broken shell 1	Pebbles floor

). The presentation of the tracks on the floor creates the condition for the involvement of two teams or, alternatively, the instruction to alternate the sequence of eggs to break by stepping on them.

The newly designed “Sound environment” activity is structured to facilitate recognition and association between a sound environment presented in acoustic mode with the corresponding photographic representation, located in the central area of the wall or floor. Around the image, there are four different colored trumpet-shaped buttons, which, once pressed, emit four different ambient sounds. The activity requires one to correctly match the auditory stimulus to the sound environment represented at the center of the layout. The location of the target-sound and the other sounds that act as distractors vary for each game set, to prevent a learning effect (

Table 3. “Sound environment” activity: stimuli and matching.

Game	Central Figure (Target)	Sound Effect (Target)	Sound Effect 2	Sound Effect 3	Sound Effect 4
Sound environment 1	Asian megalopolis	Traffic noise	Kids playing voices	Jungle with animal noises	Symphonic orchestra
Sound environment 2	Animal farm	Animal farm noises	Birds	Symphonic orchestra	Traffic noise
Sound environment 3	Jungle	Jungle with animal noises	Animal farm noises	Traffic noise	Thunder
Sound environment 4	Mexican Mariachi	Music from Mariachi band	Rock concert	Kids playing voices	Flute solo
Sound environment 5	Thunderstorm with lightning	Thunder	River water	Traffic noise	Murmur of the sea
Sound environment 6	Circus tent	Circus theme	Music from Mariachi band	Symphonic orchestra	Kids playing voices
Sound environment 7	Stadium	Supporters choir at the stadium	Kids playing voices	Traffic noise	Rock concert
Sound environment 8	Symphonic orchestra	Symphonic orchestra	Rock concert	Flute solo	Music from Mariachi band

).

Among the proposals of the “Sound environments”, the original activity “Old MacDonald Had A Farm” was designed. In this variant, the different characters in the well-known song replace the colored buttons to be pressed (Figure 2B). The presentation can be used both on the wall and on the floor. In the layout, there are drawings of the farmer and farm animals. The motor interaction with the character “Farmer” is coupled with the musical base of the song “Old MacDonald Had A Farm”. To make the activation of the song accessible only to the educator, the farmer’s character was positioned high and away from the other animals, near where the children were positioned. The track has been modified by adding silent pauses, in correspondence with the noises of each named animal. Five farm animals were identified in the song and for each one, a button with the animal’s drawing was inserted, which was associated with an audio file of the noise. The activation of each character can be viewed, thanks to the colored borders that remain lit as long as the associated sound is produced. Children are required to activate their character at the exact moment when the song names their animal, producing the associated voice. The activity stimulates auditory attention functions, auditory work memory, executive functions of planning and inhibition; it also promotes the enhancement of shift management skills and collaboration, offering correctness feedback that can be easily recognized by children without adult mediation.

In the ad-hoc conceived “Musical canon” activity, the goal is the creation of a canon composition, starting from four different guitar strings presented on the wall. Each string of the virtual guitar is associated with a complete verse of a song, which lasts about 15”–20”. Children are required to select the strings in sequence, with progressive timed insertions, creating the canon effect (Figure 2C).

Additionally, the “Musical Puzzle” activity has been designed by the authors as a task requiring one to reconstruct a song from musical fragments associated with four different trumpets presented on the wall. Each audio track has a duration of 5”–10”. The trumpets must be activated in the correct order to obtain the complete song.

In the “Jam Session” activity, the goal is the composition of creative music sessions. Ad-hoc selected audio tracks of the distinct musical instruments that make up a song are presented. Each of the four strings of the virtual guitar projected on the wall is associated with an audio file with an isolated partition of an instrument. The tracks each last 10”–15”. The strings can be played sequentially or simultaneously to render the effect of the entire song.

Table 4. Contents of musical activities.

Activity	Song or Music Track	Procedures	Projection
Musical Canon	“Fra’ Martino” (“Brother John”, traditional song)	Mediation and instructions required	wall
	“Capra Capretta” (nursery rhyme)	Mediation and instructions required
	“Stella Stellina” (nursery rhyme)	Mediation and instructions required
	“La canzone del cuculo” (nursery rhyme)	Mediation and instructions required
Musical Puzzle	“La casa” (nursery rhyme)	Whole audiofile to be presented first	wall
	“Il leone si è addormentato”—Italian version of “The Lion Sleeps Tonight” (children’s song)	Whole audiofile to be presented first
	“Ci vuole un fiore” (children’s song)	Whole audiofile to be presented first
Jam Session	“House of the rising sun” (traditional folk song)	Free access or mediated by educator	wall
	“La Bamba” (Mexican folk song)	Free access or mediated by educator
	Reggae melody (instrumental)	Free access or mediated by educator

lists the content of musical activities.

2.2.2. Typology “Reveal”

In the “Water Lily Pond” activity, motor responses such as inhibition and movement and body control are requested. The goal of the ad-hoc designed tasks is to remain as still as possible on some lotus leaves, assuming the positions assigned by the educator (for example, standing still on one foot, the position of the frog, two children on the same leaf, etc.) Visual feedback is provided by the water surface of the pond, sensitive to movement. A motor variant of the task is proposed, requiring children to jump from one leaf to another. In this version, the goal is to be very accurate in positioning itself on the leaf, minimizing water movements.

2.2.3. Typology “Particles”

In the “What’s hiding?” activity, children are invited to organize themselves to find out what (object or short sequence) is hidden under a layer of elements (“particles”), that move due to the interaction of the body on the floor (

Table 5. “What’s hiding?” activity arrangement.

Name of the Set	Content Elements	Arrangement
In the grass	Ladybug; Mushrooms; Apple; Shoe; Ball; Can; Watch	Small red elements spread in the grass
What fruit is it?	Watermelon; Strawberry; Kiwi; Melon; Pineapple; Orange; Apple; Chestnut	Enlarged significant details of each element are shown on the screen
What animal is it?	Tiger face (very close); Cat face (very close); Starfish (small); Seashell (small); Crab (small); Frog (camouflaged); Butterfly (camouflaged)	Mixed enlarged, small and camouflaged elements are shown on the screen
What object is it?	Wall clock; Corkscrew; Sharpener; Scissors; Nutcracker; Dominoes; Pen; Key; Brush; Clew	Enlarged significant details of each element are shown on the screen
What game is it?	Football; Tennis; Ping-pong; Basketball; Table football; Golf	Enlarged significant details of each element are shown on the screen
Video 1	A butterfly comes out of the cocoon; Lightning pierces the night sky; Cookies baking in the oven; Birds fly in the sunset sky; A pigeon walks on an urban background; A monkey crosses a road	Short movie clips are shown in succession on the screen
Video 2	An airplane flies in the sunset sky; A car drives on a road; A man paddles on the sea while a shooting star appears in the sky; A pedestrian walks in an urban background; A flower blooms; Movements of the sponges on the coral reef	Short movie clips are shown in succession on the screen
Video 3	A man rowing; A squirrel peeks out; A car crosses an urban background; Some elephants cross a road; A seagull flies in the sunset sky; A big monkey crosses the road	Short movie clips are shown in succession on the screen

). The particles become rare with the movement of the body and re-thicken in a short time (Figure 2D,E). This activity was largely manipulated so as to address several cognitive functions. The images to be unveiled were selected following the general criterion of being difficult to either spot or recognize, thus involving visuospatial attention, visual discrimination, visual representation/integration and, on the other hand, the need for cooperation between many several children, to achieve the objective of identifying the target. Three types of visual combinations were envisaged:

(1) Very small target-element with slow-moving particles

(2) Very large target-element with fast-moving particles

(3) Chronological sequences (with several elements in a sequence) with fast-moving particles

2.2.4. Typology “Follow Me”

The “Whack-a-mole!” activity was used as provided by the system. Selective and diffused visual attention functions are involved and the game requires a gross-motor response while performing the activity on the floor. The goal of the game is to tap the moles as soon as they emerge from the ground. The speed of presentation of the stimuli is variable and the players receive visual and sound feedback when they “catch” the mole.

2.2.5. Typology “Move to”

The “Dog” activity was customized by adding special floor backgrounds, on which different paths are traced through the use of images of footprints or path signs (circles, start and finish lines, platforms, etc.) (Figure 2F). Children were invited to divide themselves into small groups (2–3 children), according to different rules given by the educator (hand in hand, back to back, etc.). The aim of the activity is to coordinate with each other to follow the paths on the floor, without being joined by the dog that chases them. The system reports how many times the dog “touches” a child on the floor projection.

2.3. The Game Sessions

A subgroup of designed activities was devoted to the stimulation of visual-perceptual and visual-attentional functions combined with motor coordination skills, especially supported by floor projection. Children were requested to walk and reach objects or targets disposed according to configurations of varying complexity and to move within predefined spaces maintaining fixed configurations with other children, while the outer context may be varying, or to organize group strategies to reach a given goal.

In another set of activities, auditory discrimination and auditory attention were targeted, as well as the abilities to form visual-auditory associations or to train auditory working memory, executive functions and creative expression. At the same time, children were offered an opportunity to refine their musical sensitivity and their perception of rhythm and melody. Various types of auditory-based activities were proposed to the children subdivided into small-medium groups (3–5 participants). The activities stimulated the ability to recognize and associate certain sounds typical of a natural or human environment, or they included musical tracks to be rearranged or to be combined logically or creatively. Most of the proposals were designed for wall projection, to decrease the motor activation of the body and to promote cooperative interaction between children. Cooperation was stimulated both in terms of sharing information and in making decisions.

The psychologists who took care of conducting the activities with the children built a narrative frame, on the theme of the “Journey” as a metaphor for a journey through various areas of expertise. A fictitious character named “Auntie Simalù” was invented and presented through drawings and animations, playing the role of a guide who leads the journey and acting as a glue and connecting element between the various proposals.

2.4. Participants

The games were proposed to 25 children (12 males), aged 4–5 years, attending a kindergarten in Lecco, Italy, at the beginning of the new school year. All the children whose parents gave informed consent to participation were included in the study. All children had been invited to participate by the team of school teachers, provided they were at least 4 years of age and there were no organizational factors that would make participation discontinuous (e.g., children who could not be at school in the first observing session due to late morning arrival were not asked to join the project). Children from different sections (four classes) were mixed into different groups, so as to obtain comparable groups with respect to age, level of mutual knowledge (which was generally low since the school year had just started) and multicultural features. There were 3 children of non-native origin in the group. There were no known disabled children, except for a child with mild mobility difficulties.

All of the children had had previous experience with electronic devices, and at least a PC or a tablet was present in each of the families, as reported by the parents in a questionnaire. The social background of the families was that of a small city, and the school was located in a part of the town where no particular social problems (crime or deviant behaviors) are usually reported.

The children were divided into three different groups of 8–9 children each and were guided through the activities by the psychologists and class teachers. The choice to divide the children into three groups allowed one to plan game sessions of one hour each, which was deemed to be a reasonable time, considering the attention times and play skills of children of this age group. Moreover, groups of such size allow rich and dynamic group interactions to be observed, and at the same time are easily managed by adult observers. All sessions were video recorded.

The children’s parents signed informed consent forms, as well as permission and release forms for images, videos and sound recordings, in accordance with the Declaration of Helsinki.

2.5. Observation Procedures

A structured observation grid about the observed behaviors and interactions was filled by the adult participants (teachers and psychologists) at the end of each session. A copy of the grid can be found in the Supplementary Materials.

In the observation form, 10 descriptive variables have been provided, assessed independently by each observer involved, for each activity proposed in the different days of experimentation in the school context. The grid required evaluation on a Likert 5-point scale (from 1 = “not at all” to 5 = “very much”) of a series of variables concerning: “Game” (understanding of the game; strategic behaviors; participation; enjoyment), “Interactions with peers” (interaction; conflict; cooperation) and “Clarification requests” (questions about the device = Q-device; questions about the activities = Q-activity; active help requests = Q-active help). Each variable was rated at three different moments of the playing session: initial (T1), intermediate (T2) and final (T3). Since the emphasis was on the groups and not on the single individuals, and since variables such as interaction and cooperation are most meaningful if evaluated at the group level, the raters were asked to fill a rating form for each of the observed groups.

2.6. Data Analysis and Statistics

Data from the ratings collected on the groups performing the same kind of activity were collapsed by averaging within each rater, since there was no hypothesis of any difference between groups. Then, a reliability analysis was performed among the raters, to assess their agreement. Specifically, intraclass correlation coefficients (ICC) were computed for each activity, considering a two-way random model and looking for the absolute agreement. When the agreement was considered optimal, data from different observers were averaged. Subsequently, the scores of all the different activities were analyzed together, in order to evaluate the system as a whole. First of all, a non-parametric paired analysis was performed to check for differences among timepoints for each variable: Friedman test was run among T1, T2 and T3 and, when statistically significant, a post hoc Wilcoxon test was performed between couples of timepoints. The level of significance was set at 0.05.

Finally, a non-parametric correlation analysis was run between all the variables; Bonferroni correction was applied to account for multiple comparisons. Due to the high level of similarity of two variable pairs (Participation and Enjoyment on one hand, Interaction and Cooperation on the other hand), as reported by the raters and as confirmed by pairwise correlations (rho = 0.86 and 0.91, respectively), it was decided to consider such pairs as expressions of a same underlying construct and therefore, to have each pair count as 1 for the Bonferroni correction (thus, correction was applied for a total of 7 variables instead of 9, with alpha set at 0.05/28 = 0.002).

Analyses were performed in SPSS 21.

3. Results

The test in the kindergarten setting allowed evaluation of the app functionality.

The activities were highly appreciated by the children, who experimented with the device with curiosity and enthusiasm. A video illustrating various moments of the study, with the children performing different activities, is provided in the Supplementary Materials.

Figure 3 depicts specific moments during activities, with virtual environments projected on the floor (Figure 3A,B) or on the wall (Figure 3C,D).

A total of 58 filled grids were collected on a set of 31 total game sessions (each game session was devoted to one activity). The forms were filled by the different observers who participated in the activities (educators and psychologists).

The number of observations for the various activities (9 different activities) ranged from 12 (What’s hiding) to 2 (musical canon), with a mean of 6.44. The different numbers depended on the teachers’ time schedules that allowed for participation in the activities, only at certain times of the day. At least one of the two psychologists was always present.

First of all, ICC values computed among the raters for each activity were larger than 0.9, but for Musical Puzzle (ICC = 0.5). Therefore, the data from different observers, except for Musical Puzzle (for which ICC was deemed to low) and Canon (for which too few observations were collected to be considered reliable), were collapsed by averaging.

Figure 4 shows the results about the variables rated at the different timepoints, T1 (Initial), T2 (Intermediate) and T3 (Final), considering the system as a whole (for this analysis, Musical Puzzle and Canon were not considered). As supported by the statistical analysis (

Table 6. Comparison among timepoints for each variable. Data in T1, T2 and T3 are reported as mean ranks. Statistically significant correlations (p < 0.05) are shown in bold. When the Friedman test was statistically significant, p-values of post hoc analysis (Wilcoxon test) are reported. Legend: COMP—Comprehension; CONF—Conflict; STRAT—Strategy; ENJ—enjoyment; PART—Participation; INT—Interaction; COOP—Cooperation; Q-DEV—Q-device; Q-ACT—Q-activity; Q-ADU—Q-adults.

	Friedman Test					Post Hoc—Wilcoxon Test
	T1	T2	T3	Chi-Squared	p-Value	T1 vs. T2	T1 vs. T3	T2 vs. T3
COMP	1.00	2.13	2.88	15.20	0.001	0.012	0.012	0.028
CONF	2.19	2.38	1.44	5.25	0.072
STRAT	1.00	2.38	2.63	13.07	0.001	0.012	0.012	0.674
ENJ	1.31	2.19	2.50	6.69	0.035	0.043	0.035	0.173
PART	1.25	2.25	2.50	8.00	0.018	0.021	0.017	0.144
INT	1.19	2.13	2.69	10.14	0.006	0.017	0.018	0.092
COOP	1.13	2.31	2.56	9.74	0.008	0.017	0.012	0.612
Q-DEV	1.94	1.88	2.19	1.40	0.497
Q-ACT	2.44	1.63	1.94	3.91	0.142
Q-ADU	2.13	1.88	2.00	0.50	0.779

), a significant increase in comprehension, strategy, enjoyment, participation, interaction and cooperation along timepoints was highlighted. A post hoc analysis stated that significant differences were mainly between T1–T2 and T1–T3, while they stabilized at T3, but for comprehension, that significantly increased also at T3.

Figure 5, Figure 6 and Figure 7 show the impact of activities on each variable; for this analysis, we decided to take all of the activities into consideration (including Musical Puzzle and Canon), for the sake of completeness.

As can be seen from Figure 5, the understanding of the activity improves from the beginning of the session to the end, and achieves good absolute ratings in general. The exceptions are some of the music-based activities, Musical Puzzle and Canon, which remain below 2 at the end.

Strategic behaviors in the groups of children tend to increase over time, with both increasing practice and familiarity with the task, especially from T1 to T2, reaching high ratings in “Whack-a-mole” and Eggs activities.

Participation is generally high in all of the activities; it increases from T1 to T2 and remains stable over T3. Enjoyment is high in all the activities and increases over time, with the exception of music-based activities: in Musical Puzzle, Canon and Jam session, enjoyment at T3 is lower than at T2.

Variables related to social interaction are represented in Figure 6. Interaction among peers is good, especially in “Whack-a-mole” and Eggs, and increases over time; the lowest value is observed in the Canon activity. Conflict is very low overall, and tends to decrease after the first evaluation. Cooperation is high, especially in Sound Environment, “Whack-a-mole”, Farm and Eggs.

Figure 7 represents the requests directed to adults and care-givers: the requests of explanations and clarification are infrequent (between 1 = “not at all” and 2 = “rarely”) concerning both the device and the activity, and direct requests of help are quite uncommon, except for Musical Puzzle, where requests increase during the session.

Finally, results of the correlation analysis are reported in

Table 7. Correlations among variables. Data are reported as Spearman’s rho (p-value). Statistically significant correlations (p ≤ 0.002 to correct for multiple testing) are shown in bold. Legend: COMP—Comprehension; CONF—Conflict; STRAT—Strategy; ENJ—enjoyment; PART—Participation; INT—Interaction; COOP—Cooperation; Q-DEV—Q-device; Q-ACT—Q-activity; Q-ADU—Q-adults.

	CONF	STRAT	ENJ	PART	INT	COOP	Q-DEV	Q-ACT	Q-ADU
COMP	−0.15 (0.477)	0.74 (<0.001)	0.46 (0.024)	0.73 (<0.001)	0.60 (0.002)	0.44 (0.030)	−0.05 (0.819)	−0.19 (0.375)	−0.40 (0.055)
CONF		0.35 (0.098)	0.26 (0.213)	0.06 (0.800)	0.01 (0.958)	0.08 (0.714)	−0.41 (0.049)	0.19 (0.368)	0.43 (0.035)
STRAT			0.50 (0.014)	0.53 (0.008)	0.69 (<0.001)	0.51 (0.010)	−0.03 (0.892)	−0.03 (0.885)	−0.16 (0.447)
ENJ				0.86 (<0.001)	0.65 (0.001)	0.63 (0.001)	−0.23 (0.287)	−0.12 (0.166)	−0.03 (0.892)
PART					0.60 (0.002)	0.58 (0.003)	−0.30 (0.161)	−0.37 (0.076)	−0.16 (0.449)
INT						0.91 (<0.001)	0.19 (0.379)	−0.46 (0.024)	−0.17 (0.419)
COOP							0.18 (0.390)	−0.52 (0.009)	−0.13 (0.537)
Q-DEV								0.20 (0.340)	−0.34 (0.110)
Q-ACT									−0.11 (0.595)

. The understanding of the activity strongly correlates (Spearman’s rho > 0.7) with strategic behaviors and participation, and has a moderate correlation with interaction. Furthermore, strategy, enjoyment, and participation moderately correlate with interaction. Enjoyment has a very strong correlation with participation and a moderate correlation with cooperation. Finally, a very strong correlation was found between cooperation and interaction.

4. Discussion

The present study describes a system intended to stimulate motor skills, perceptual functions, executive functions, and social skills.

The system was tested with three groups of normally developing children during kindergarten activities. The number of different observations and the varying composition of the observed groups does not allow one to statistically compare initial, intermediate, and final observations for the single activities. Therefore, the general trends concerning changes over time in the observed behavioral variables have been analyzed based on the averages of the various activities. Nonetheless, a general evaluation of the graphs representing the mean levels of the observed variables for the single activities sometimes show very clear patterns and trends that can be considered significant at a qualitative level and will be discussed in the next paragraphs.

Overall, the results of structured observation reveal that the system is easy to understand, enjoyable, elicits high levels of participation and little conflict, moderately favors strategic behaviors, cooperation and social interaction and can be used by the children with a limited need of instruction or support from an adult. Notably, the children’s engagement and their cooperation during the activities improve with the familiarization and use of the system. More precisely, there is a clear improvement in these aspects from the beginning to the midpoint of the activity, and a general stabilization from the latter to the end of the activity. Conflict, by contrast, remains stable (and low) across the whole session, and so do questions about the activity and help requests directed to adults.

Comprehension of the tasks generally improves during the session, and achieves a very good level in the end. An exception is represented by two of the music-based activities, Musical puzzle and Canon, which start with rather low levels of comprehension and improve only minimally across the session. This probably depends on the nature of these activities, which require a high level of coordination among several members of the team, and very rigid turn-taking, based on the accurate reconstruction of song texts in one case, and the ability to sing out-of-phase in the other case. These are highly complex skills that are probably developed later on. The easiest activities are the motor-based activities, for which both initial comprehension and progressive increase in comprehension reach higher levels.

Additionally, strategic behaviors tend to increase with time. This is particularly true for motor games, such as Dog, “Whack-a-mole” and Eggs, where organization and coordination between participants in the group make their action much more effective. This advantage has probably been noticed and exploited by the children, and it is also reflected in the significant, positive correlation between strategic behaviors and both task comprehension and interaction (the correlations with participation and cooperation, even if non-significant after Bonferroni’s correction, also represent moderate associations between variables). In other activities where the need for structured, organized action of the groups is even more explicit, such as the music-based activities, a clear increase in strategic behaviors is also observed, and it is present even for the most difficult activities, such as the Musical Puzzle (where an improvement is reached at the end of the session only) and Canon. These results suggest that accessibility and usability of the proposed activities are adequate and satisfactory.

Participation is generally high, with an initial increase and then stabilization. The only exceptions are the two most difficult activities, Musical puzzle and Canon, for which there is an initial increase in participation, followed by a decrease towards the initial levels. A similar trend is observed for both Interaction and Cooperation among peers, which tend to increase over time (also from midpoint to end of the session) for all activities but Canon, which remains at rather low levels for both parameters. This result is probably to be interpreted with regard to the difficulty in understanding the game, even after some time has passed and some experience has been gained. Indeed, there are high and significant correlations among comprehension, participation and interaction scores, supporting this interpretation. Increases in interaction and cooperation are particularly evident for activities like Sound Environments and “Whack-a-Mole”, which have a clear group structure but are easier to understand.

Conflict is generally very low, and tends to be stable, or even decrease for some activities such as Sound environment, “Whack-a-mole” and Canon, i.e. the most challenging activities, where the advantages experienced through strategic interaction and cooperation may have encouraged group cohesion, rather than competitive behaviors.

Enjoyment of the activities is high and shows an increase from the first to the second time measure. This is especially true for motor activities requiring structured, organized group action, such as Waterlilies and Farm. The most difficult activities (Musical puzzle and Canon) even show a first increase followed by a decrease in enjoyment, suggesting that such activities could be particularly tiring and possibly boring for children as young as preschoolers. Correlations with enjoyment confirm that this variable is strictly linked to participation, which can be seen as conceptually related to it (both express the capacity of the activity to attract and involve the child), but also to interaction and cooperation, which suggests that sharing the activity with peers increases the pleasantness of the activity itself.

As to the questions directed to the adults, including requests of explanation, clarification (concerning both the device and the activity) and help, these are very infrequent and confirm the good levels of accessibility and usability of the system. The absence of correlations with other variables, however, may indicate that all types of questions were rather an expression of participants’ needs, rather than reflecting the effects of other factors.

Such results suggest that VR and digital applications have the potential to become important instruments in promoting children’s cognitive and social development, and in improving routine educational work in kindergarten settings. The system turned out to be flexible and able to adapt to the various goals of the educators and psychologists: even starting from the same typology of activity, it was possible to address very different skills and functions, with a moderate amount of programming work. The possibility to vary both images and sounds and combine them in various ways offered many different pathways to stimulation of a certain capacity, so that even a single, very specific rehabilitation goal could be pursued without having the child performing repetitive and boring activities. Another valuable characteristic of the activities is the ability to keep high levels of attention and motivation during the whole session, even increasing them from the beginning to more advanced phases. In other terms, the “novelty effect”, which is often reported in studies on the use of VR in educational contexts [32,51], was not observed here. Last but not least, the system (which has mainly been developed for individual rehabilitation) revealed to be very suitable for group activities, provided that educators or trainers have good mastery of the programming part and can exploit the various options in changing, creative ways.

Floor projection turned out to be very suitable to motor games, not only in standing position (jumping, running, balancing etc.), but also in lying position, where the children could roll and crawl, and use their legs, hands and arms to produce effects on the underlying images. This was very entertaining and motivating for the children, and also evoked many different ways to interact with peers and organize various motor strategies to improve effectiveness. Wall projection, by contrast, was used mainly for music-based activities (“pulling” the strings to produce notes or melodies or song parts). This was performed with hands and fingers, but it was less immediate for the children to perceive the direct link between finger/hand movements and their effects, due to the characteristics of the projection/detection system. The feedback that the system gives is indeed driven not only by the direct contact of hands and fingers with the wall, but also by the movement in the space between the optoelectronic camera emitting infrared rays and the wall. Another application of wall projection was for the “what’s hiding” activity, where the children had to reveal, through their movement, objects and object parts hiding under floating elements like balls or clouds. In this case, the effect of hands and arm movement was amplified by using colored feather dusters, which the children greatly enjoyed, and which produced a greater feeling of directly acting on the wall surface.

The single activities appear to have some unique characteristics that should be kept in mind when planning an intervention for cognitive empowerment or rehabilitation. In particular, motor games and, more generally, activities involving gross body movement, such as following paths, jumping, chasing, appear more motivating and yet, are able to stimulate strategic behaviors. Music-based activities can be difficult in this age-range and should be kept as simple as possible, allowing the free expression of creativity (like in the Jam session activities or in the Sound environments tasks). They appear to stimulate cooperation and interaction more than strategic behaviors. More complex activities may suit older children.

The system used, based on the principles of virtual reality, was demonstrated to be suitable for young pre-school children, as it was easy to understand and to interact with. The multiplayer feature of the system is a strength in this playful context, with respect to other systems, used by a single player in a rehabilitation session [53,54]. However, the main disadvantage of the system, at least in the wall configuration, is the lessened realism of interaction, due to the method of capturing movements with the optoelectronic system.

On the whole, a positive impact of the activities at both the cognitive and the social level is observed, which suggests that the system can be effectively used in a kindergarten setting to empower motor planning, strategic behavior, and cooperative skills. The system seems to be suitable also for rehabilitation applications in children with cognitive or motor disorders. Most of the principles described by Gee [40,55,56] are implemented in the activities. Among others, the “Active, Critical Learning Principle”: the learning environments encourage active and critical learning; the “Multiple Routes Principle”: there are multiple ways to move ahead, and this allows learners to make choices, exploit their own strengths and styles of learning, explore alternative styles; “Skills as Strategies”: the learner isn’t simply practicing a skill for its own sake, but with the goal of solving a problem; the “Multimodal Principle”: meaning and knowledge are built up through various modalities (images, sounds, words, symbols, interactions, etc.); “System Thinking”: games make players think in a bigger picture, helping them see how the pieces can fit together; and the “Intuitive Knowledge Principle”: intuitive knowledge is built up in repeated practice and experience, and this occurs in association with an affinity group.

A limitation of the present study is the lack of direct indexes of the children’s performance during the activities. Future studies will be needed to collect objective data on the effectiveness of the system and activities in improving the cognitive and neuropsychological functions targeted in the various games and observing the impact on social attitudes and behaviors at the individual and group level in a more systematic way and over longer periods of time. In order to do this, it would be very useful if the system could offer more possibilities to record data about accuracy and qualitative information during the activities (e.g., number of different children actively participating in a certain activity at a certain moment, number of steps taken to reach a certain goal, etc.) Generally speaking, it would be desirable that this type of semi-immersive system, essentially designed and used in rehabilitation settings, could be adapted (in terms of design, user-friendliness and cost) for use in educational settings. In this perspective, technological solutions (not conceived as mere transpositions of traditional activities, but exploiting all the unique possibilities of virtual environments) could, at the same time, extend the range and breadth of situations to be experienced and skills to be learned, and also constitute an introduction to the more and more pervasive ICT reality.

A last note should be devoted to the multidisciplinary teamwork that led the programming of the activities, the games, and the organization of the game sessions, involving psychologists, educators, teachers, engineers, and technicians. This allowed for pedagogical and cognitive, as well as functional, and usability-related issues to be implemented and harmonized in the final programs: the results suggest that this co-designing work was advantageous for the success of the experience and the positive reactions of the children.

5. Conclusions

The present study focuses on the impact of the proposed activities at the group level, before extending investigation to the effects produced on typically and atypically developing children at the individual level. The results suggest that the system can be successfully employed for empowerment of social abilities in group activities. Further studies are envisaged, testing the impact of the activities on children with neurodevelopmental disorders, especially those involving deficits of motor-praxic organization, perceptual and social skills. Children with language and communication disorders are a very interesting target, as they could train some of the perceptual prerequisites for language learning, such as auditory discrimination and rhythmic abilities [57,58]. The activities could have a twofold application: improving weak functions in the at-risk population, either in a mainstream or special educational context or in a rehabilitation context, and empowering developing functions in typical populations in schools or other aggregation/education centers, whilst at the same time promoting greater inclusion and preventing isolation, aggression or bullying behaviors.