Augmented Interaction Systems for Supporting Autistic Children. Evolution of a Multichannel Expressive Tool: The SEMI Project Feasibility Study

Abstract

Background: Over the past ten years, the authors have been designing, developing, and testing pervasive technology to support children with autism (ASD). Methods: In the present study, an integrated system based on multimedia and augmented interaction technologies have been tested on young subjects with ASD and dyspraxia in the age range of 6–10 years, in charge for rehabilitation treatments; a team of clinical psychologists has analyzed the results of the experimentation. The ten children involved in the project underwent an initial assessment of praxis skills and motor coordination. Subsequently, the subjects were subdivided into two subgroups: five children participated in the experimentation and five were evaluated as the control group (treatment as usual). Results: The evaluation showed an increased score in the several aspects considered, and particularly those related to motor coordination. An improvement in balancing tests and in hands-movement testing was found. Conclusion: The children involved in the sessions showed greater ability to self-control the movement as well as to select specific motor areas. The methods used seem to be promising to improve emotional and social skills too in a motivating and enjoyable climate. A high level of acceptance by professionals was observed and parents’ feedback was also positive.

1. Introduction

1.1. Background

Smart environments have a positive impact on therapeutic practices aimed at children with autism [1,2,3,4,5,6,7,8,9]. In recent years, the authors have designed different tools, showing how interactive environments can have a positive impact when used in synergy with traditional therapeutic practices [10,11,12,13]. Autism is characterized by deficits in social interaction, communication, and behavioral flexibility. Cognitive functioning in persons with autism, related to behavioral symptomatology, has been reported largely within the domains of theory of mind [14], executive functioning [15] and central [16]. So, there are several core underlying capabilities/deficits that coexist in ASD and that contribute to the pathogenesis of autism [17].

Clinicians have recognized the potential of information technology in research and treatment. The use of new technologies is able to strengthen the motivation and interest of children, getting closer to their language and their cognitive functioning, is able to facilitate the processes of attention and concentration, providing constant and stable feedback, strengthening a positive self-awareness. Pervasive games are gaining importance in the last years: these games are usually played in an interactive space, in which several user interfaces following a multimodal approach work together. Pervasive games have proven to be useful in several fields, including the educational field and also in the field of treatment and rehabilitation for children with special needs, such as Attention Deficit Hyperactivity Disorder (ADHD) [18] or Autism. Children with special needs often experience a deficit in executive functioning (EF) and assistive technology can facilitate organization, time management and planning [19]. This work originates from a collaboration between the Signals and Images Laboratory of the Institute of Information Sciences and Technologies (ISTI) and the Institute of Clinical Physiology, both of the National Research Council of Italy, CNR.

The present article describes, in particular, the design, implementation and evolution of an augmented interaction-based system that helps children with ASD to improve their communication and cognitive skills. Rather than superimposing digital information (images, audio, text) on the real world, as in “classic” AR (Augmented Reality) systems, in our system the technology is used to increase the sensorial relation with the environment. The systems are based on a computer connected to various sensors, so that it can reacts to movement with multimedia stimuli [8,9,10,11,12,13].

The learning content is conveyed by activities of production and perception of visual and auditory phenomena, aimed at increasing their communicative competence. The system is based on gestural interfaces and custom applications that extract features from the human gestures, mapping them to audio and visual stimuli under predefined schemes. The augmented interaction with the environment may improve the proprioception in the subjects. SEMI, the interactive multichannel expressive system [20], is a project that inherits the experience of previous implemented asset with SiREMI and SiDOREMI projects (auditory feedback-based system and its evolution in a home version prototype), enriching it with new interaction modes that also include the visual channel [10,11,12,13].

The setting of the present feasibility study is The Maria Assunta in Cielo Onlus Foundation (fMAiC Onlus), a not-for-profit organization set up in Pistoia in 1989, on the initiative of a group of volunteers already engaged in associations and activities for people with disabilities [20].

These multimedia interactive devices have been experimented on a small group of children with autism [10].

1.2. Children with ASD and Intervention with Interactive Multimedia: The Previous Projects

Advances in Information Communication Technology may help children with Autism by creating an intervention system involving the also the participation of parents and therapists [21].

Autism is defined as a pervasive developmental disorder [22,23,24,25,26]. The complexity of the clinical presentation requires early and appropriate intervention to promote positive outcomes for children and families [27,28]. There is a growing body of evidence reporting promising results of the efficacy of early and intensive behavioral intervention [27] as well as of developmental or relationship-based interventions delivered by therapists or teachers [9] and parents [28].

The development of new and affordable technology (mobile phones, tablets etc.), gestural interfaces and virtual reality (VR/AR systems give the possibility of developing new approaches to (early) autism treatment [29,30,31,32,33,34,35,36]. Autistic children understand better, becoming more collaborative, if the work space is well defined, and it is always the same with few distracting stimuli. The structuring of time, on the other hand, clarifies with what sequence and for how long a specific activity will have to be performed. When the user understands what is expected of him, the anxiety due to a new activity or the transition from one action to another is reduced and it is easier to independently perform tasks that require planning skills. The computer can also be used for affective education through multimedia presentations on emotions and rules of behavior, with images and videos of role playing or personal life situations. The didactic and rehabilitative use of the computer offers the possibility of improving and developing communication skills, both verbal and non-verbal, if thought within a coherent and adequate programming. Greater vocal interactions, greater eye contact, significant improvements in spontaneous communication, shared attention and learning are the possible results of stimulations that new technologies can provide to children with this type of specific difficulty. Since the computer activity is structured, organized and clear, the computer is a simple tool that allows to achieve remarkable competences in autonomous use and has positive effects on self-esteem, psychological well-being and integration. Measurement and treatment for people on the autism spectrum can be improved with new and forthcoming technologies, if researchers shape them for such improvements [37]. The language of information technology responds to the need for communicative precision because it is clear, structured and predictable, without emotional inferences or typical implications of communication between two or more people. The computer appears to be an advantageous tool within this range of difficulties due to its characteristic: mechanical and anaffective voice, rigidity, structuring and predictability. It is easier to understand a context and to imagine the consequences of a behavior if variables such as desires, opinions, beliefs have not to be controlled and taken into consideration.

SIREMI is an interactive audio system proposed as a game in which the main actors are the autistic child and an adult operator. Its main goal is to develop social and problem-solving skills. In the intervention we considered the children sensory profile that undergoes interactive sound stimulation [10,11,12,13].

During the sessions, the children can freely move inside an interactive space. The algorithms detect several parameters of the human posture and gesture, such as position in space, arms and legs angles, etc. These parameters can be linked to several controls of the synthesized sounds using a GUI (Figure 1). Each synthesizer works in an independent way, and, for each of them, it is possible to select the instrument from different banks.

This augmented interaction system helps to improve the relationship with the environment.

The experimentation [10] was performed in the Institute Massarosa 1 (Lucca, Italy), primary school. The subjects were male, 5–7 years old, diagnosed with (low-functioning) ASD. This experimentation was coordinated by Prof. Elisa Rossi of the Istituto Comprensivo Massarosa 1 [10,11,12,13,38,39,40,41].

For maintaining the SIREMI positive effects we developed SIDOREMI, a home version of the system. The working principles have been enriched and modified in a ludic way: The child, moving in front of the screen driven by the parent, is stimulated to make a series of moves thanks to several interactive modes. The audio feedback was also accompanied by the visual one: a stylized avatar of the child’s figure is displayed on the screen, which can channel more effectively the attention of the child. With SIDOREMI, the families of subjects could continue, at home, the re-education pathway started in the school environment (or anyway in the specialized center) remaining connected via the Internet, in real time or deferred, to the central station [10,11,12].

The connection to the central station also allows for remote configuration of home stations and the possibility of remote audiovisual consultancy by specialized personnel. For example, you can view the movements of single parts of the body acquired during treatment sessions. In order to detect attempts to imitate the parent-operator movements (useful at the end of the evaluation of effectiveness), it is possible, by means of a special algorithm, to visualize the instantaneous trend of an “imitation index”.

The experimentation, lasting 6 weeks, started in November 2015 and ended at the end of December 2015. The 4 families involved were identified by Dr. Silvano Solari, Psychologist-Psychotherapist, Professor at the University of Genoa. The sessions were at least twice a week and lasted about 30 min. Interactive scene configurations have evolved throughout the trial, thanks to the constant connection to ISTI-CNR cloud.

The various ways of using the system (Presets) were created in collaboration with Dr. Silvia Michetti and were structured in 4 different types (Exercises):

1.3. Free Body Toning Movements

These presets are useful for getting familiar with the system’s operating mechanism: the child moves freely in front of the sensor and his/her movements are translated into closely related sounds. For example, raising an arm produces a sound from rising height while striking a foot will produce a percussive sound etc.

1.4. Simulation of Musical Instruments (Piano, Drum, Accordion, Trumpet)

In this case, exactly the gesture needed to play real musical instruments is simulated. For example, the child must imagine a piano keyboard in front of him: pressing the invisible keys will hear the corresponding piano notes. Other simulated instruments in this way are the accordion, the drum, the trumpet.

1.5. Basic Motor Coordination Exercises

In these presets the child must perform sequential movements: at each of them, the system activates an audio with a voice that pronounces a number. The purpose of the preset is to make the right sequence of movements by putting in sequence the numbers. Each time you choose this mode, the sequence is different, forcing the child to discover it each time. There are 3 different ways, with different degrees of difficulty: a sequence of four, five or six movements (the latter is not easy even for an adult without interference).

1.6. Co-Ordination Exercises with the Aid of Cognitive Abilities

This exercise is similar to the previous one, but sentence fragments are produced instead of numbers, by sequencing. Exercise therefore also requires a cognitive effort to create a meaningful sentence. The system has 10 different phrases to complete. The child cannot move to the next sentence unless he/she has completed at least one time the previous sentence. The set of sentences is not random but goes to build a story, a kind of overall adventure freely inspired by Odyssey.

The psychological evaluation of the results was carried out by Prof. Silvano Solari’s team. Psychologists agreed to report that children involved in the project participated with great interest in the mentioned experiences and have had a great benefit from sensory perception and interaction levels (see Supplementary Materials, Video S1: SIDOREMI Project overview).

2. Methods: The SEMI Project

Individuals with autism, depending on age and clinical characteristics, can benefit differently from programs that are mostly inspired by one or the other approach and it is always and only on the basis of a careful evaluation of the characteristics of the individual subject and of the available resources that it is possible to define which program and which intervention strategies are most suitable [42]. Recent research underscores the preventive value of early and targeted interventions adapted to each individual child [43].

In general, the therapy should include the integration of different educational, enabling and care tools, therefore a tailored treatment, associated with integrated interventions at the level of the family and of the school, through the presence of a multidisciplinary professional team.

The objective of the SEMI, interactive multichannel expressive system, project was to create a prototype to be included in a rehabilitation therapy for children with ASD and dyspraxia. Children with ASD have basic motor impairments in balance, gait, and coordination, impairments in praxis/motor planning and interpersonal synchrony, so it is fundamental to include motor evaluations and interventions in the standard-of-care of ASD [44,45].

In order to intervene on these aspects, we thought that adding visual feedback could make the control of motor skills more effective, in a way to exercise those coordination skills that are also useful in everyday life.

SEMI is a project that inherits the experience of the previous asset with SiREMI/SiDOREMI, enriching it with new interaction modes that also include the visual channel. This interactive Multichannel Expressive System has been developed by the Laboratory of Signals and Images of the ISTI CNR of Pisa in collaboration with the MAiC Foundation of Pistoia. The project’ goal was to give assistance to rehabilitation, working on the execution of different patterns of motions in a funny and stimulating way. For this purpose, SEMI was composed of a set of interactive games organized for levels of difficulty, based on the sensory profile of the single child. Beneficiaries are children with ASD and dyspraxia predominantly in the age range of 6–10 years, in charge for weekly rehabilitation treatments at the MAiC foundation (see Supplementary Materials, Video S2: SEMI Project overview). The study was approved by the local ethics committees of each institution and was in accordance with the declaration of Helsinki. Each parent gave informed written consent before inclusion for participation and for publication of the clinical data.

2.1. Hardware and Software Architecture

The technology used in SEMI is similar to that of previous systems, but extends auditive with visual interaction: the subject, in this case, interacts mainly with the visual channel, integrated with the auditory.

In this system we used the Kinect SDK, which provides the absolute spatial coordinates of the joints. In order to make to extrapolate features invariant to spatial position (angles, rotations etc.) we included a module for performing geometrical transformations on them. These features are then linked to graphics and sound with various modalities (Figure 2).

The team has developed four distinct applications, each one devoted to intervention on a particular aspect of the disorder:

• Repeat the movements
• Guess the movements
• Connect the dots
• Guess the card

All of this four applications are based on a common software framework, dedicated to the extraction of gestural features. As we can observe in Figure 2, the most important part of this framework is the geometrical processor, which is responsible of the translation of the Kinect data (joints coordinates) into gesture parameters. The basic algorithm, related to gestures tracking techniques, can be described as follows:

• Sensor-relative to body-relative coordinate system transformation: the incoming spatial coordinates, relative to the Kinect position, are transformed into body relative coordinates, so that the gesture extraction is not dependent from the subject’s position in the room.

We chose the spine-shoulder as the origin of the subject’s coordinates system, and the line between the shoulder as the reference angle.

• Normalization: the relative body joints coordinates are normalized respect to the subjects’ height. In this way gestures speed is independent from the biometric measures.
• Smoothing: all the parameters computed in the previous steps are then filtered using a moving average filter, in order to make them more stable. Moreover, all real-time detected outliers measures (due to sensor related limitations and errors) were discarded and replaced with the last valid value. The threshold of this outlier detection has been found empirically during the development, testing it in various conditions and with different test subjects. The threshold is normalized to the subject’s height, using the same methodology descripted above.

After these steps, the detected parameters are ready to be mapped (by the Mapper module) into application specific commands and functions (e.g., move the hand cursor in “repeat the movements” or “connect the dots”, see Figure 3 and Figure 4). Beside this real-time application, all detected parameters (speed, angles etc.) can be saved locally, for a further processing and detailed movement analysis, which can be useful for clinical evaluation based on human behavior sensing methodologies, or healthcare applications in general, also for different kind of pathologies or conditions.

2.2. Repeat the Movements

Lack of imitation skills is typical in children with ASD. Imitation is also connected to other areas of development, so helping it may result in improvements in overall children abilities. This application reproduces two stylized human figures (Figure 3): an avatar showing motor coordination exercises (in loops) and another similar figure that, thanks to Kinect sensor, replicates exactly the movements of the subject which faces it. The purpose of the exercise is to replicate exactly as possible the avatar’s movements displayed on the screen. There are 8 different exercises that can be selected through an icon menu.

2.3. Guess the Movements

This application combines motor exercises together with cognitive effort. As in the previous exercise, the avatar replicating the subject’ posture appears on the screen. The subject can perform a number of predefined movements such as raising arms or legs. Each of these movements, thanks to the gesture sensor, activates the reproduction of a different sound on the computer. By hearing these sounds, the subject must perform a sequence of movements in order to make the corresponding sounds in the correct order. The application provided two modes: in the first one the sounds consist in a recorded voice that pronounces numbers (“One!”, “Two!”, “Three!” etc.). The subject must then make the movements so that the numbers in ascending order are pronounced to the computer. If the child executes the movements in the correct order, the computer will say the word “Bravo!” and reports the elapsed time and the number of errors. The second mode was a bit more complex: instead of sequencing the numbers, the subject must sequence fragments of a sentence so that it forms a complete phrase. When the sentence was completed correctly, a sound is output that is somewhat related to the scene described in the sentence. The application included 10 different phrases, to be addressed consecutively. The whole sequence of sentences composes a complete “story”; the complete “story” is a short summary of Homer’s Odyssey. When completing each sequence, the number of errors and the time taken is reported.

2.4. Connecting the Dots

As stated before, dyspraxia is very common in children with ASD. This exergame requires motor coordination skills, and tries to make a simple intervention in the motor coordination area using a playful approach. This is the classic game of connecting numbered dots to draw a hidden figure (Figure 4): the child chooses the points to connect with the movement of the hand. The aim of the exercise is to exercise cognitive aspects together with fine-motor coordination. The sessions are organized on different levels of increasing difficulty.

2.5. Guess the Tile

Abstract reasoning may be problematic in subjects with ASD. Even a simple relationship between two symbolic representation of the same object/entity is not trivial for them. This exergame tries to exercise the subject to find a relation between a sound (related to an object or action) and its schematic graphical representation (a simple drawing). This tool proposes an exercise where video is displayed as a set of “tiles” selected from the picture exchange communication system (PECS) set, depicting things or actions while simultaneously a sound is played by the computer (Figure 5): the subject should move the hand in the space moving a cursor on the screen, placing it as soon as possible on the tile correspondent to the played sound. As the exercise progresses, the number of screen tiles increases, making the execution more difficult. The game is complicated by the fact that the tiles progressively disappear.

3. Experimentation

The ten children involved in the project were 6–10 years old, all in charge of weekly rehabilitation treatments at the fMaiC. All children had a diagnosis of Autistic Speech Disorder with language presence, four of them had standard cognitive, while the other six had mild degree cognitive disabilities (all exhibited a DSM-V first level severity) [24].

Children also had some gross or fine motorial difficulty. The dyspraxia associated with ASD was found in two subjects.

All children underwent an initial assessment of praxis skills and motor coordination according to the APCM-2 praxis and motor coordination protocol [46]. APCM-2 praxis and motor coordination protocol is aimed at the early identification of problems related to motor-praxic coordination: through the APCM-2 the strictly motor skills are distinguished from the praxic abilities, allowing, from the lower age groups, to distinguish a diagnosis of delayed motor coordination from the more specification of dyspraxia in the developmental age.

Throw this protocol it is possible the identification of the areas of strength and weakness of the child: the scores on the APCM-2 allow the visualization of the functional profile of each individual case, obtained by calculating the distance from the average norm, and therefore it is possible, for each scale to identify, which specific functions are more complete or more inadequate. So it is possible to implement specific therapy projects and then check and compare the improvements achieved over time [46]. Subsequently, the 10 subjects were subdivided into two subgroups: five subjects participated in the control group and five actively participated in the experimentation. Of the five children in the experimental group, two had a cognitive level in the standard, while three had mild cognitive impairment.

The experimental group underwent 10 sessions, each lasting 45 min, during which the various the activities were performed.

The sessions were individual and the subject was assisted in the exercises by an operator of fMAiC. The time allocated for each exercise was about 10 min, however, these times were adjusted to meet the individual’s baby’s preferences, primarily to avoid stress when there was no desire to perform a particular exercise. In each session, the operator recorded the results (times and scores), then analyzed to identify any positive trends.

Preliminary Results

All five of the children have faced experimentation with enthusiasm and during the sessions, all showed a reduction in response time. The use of this instrumentation has allowed individual children to perform exercises in a motivating and enjoyable climate. Demands, albeit complex from the motor point of view, were not perceived by children as a task subject to evaluation and therefore have been very collaborative.

In the “story” exercise, the two subjects with a normal cognitive level showed more interest in the content of the story rather than the correctness of the movements to be performed, still showing the correct results. In the same way, the three subjects with a mild intellectual disability instead paid attention to visual aids and not to the story itself, bringing better results.

Only in two cases a little frustration tolerance in failing to properly perform the exercise has been reported. Anyway, it was possible to quickly recover their emotional state and keep them in a serene way. In addition, all the children in the sessions showed greater ability to self-control the movement as well as the ability to select specific motor areas. Parents’ feedback was also positive.

At the end of the experimentation, all 10 children were re-evaluated by administering the APCM-2 protocol [46] to assess whether the group that actively participated in the experiment reported improvements in motor aspects over the control group. Figure 6 provides an overview of the evaluation of the control subjects, while Figure 7 shows the same framework for who has actively participated in the experiment. As you can see, the evaluation has increased the score in almost all aspects considered, and particularly those related to motor coordination.

In summary, in the children who participated in the experiment, compared to the control group, it was found an improvement in balancing tests and in hands-movement testing.

Extrapolated overall trends reported progressive improvement in the skills of the children’ skills.

Figure 8 show the execution times (average of all subjects) of the “connect the dots” exercise. A substantial reduction in the time at the progression of sessions and a temporary increase in the number of times at session number 6 was noted, in which small variations of the game have been introduced to increase the level of difficulty (e.g., progressive numbers points have been hidden). From session number 6 to the end of the project the execution time remained constant.

Figure 9 shows an analogous graph about exercise “guess the movements,” in its variant in which the goal is to sequencing the movements so as to build a story. Here, too, we note a gradual decrease in execution times. Also in this case, from session number 6 to the end of the project the execution time remained constant.

Finally, Figure 10 shows the reduction of the total number of errors, over the sessions, of the five subjects (indicated by A, B, C, D, E). Even in this case, we note a progressive skills improvement. The temporary worsening of performance around trial number 6 coincides with the introduction of a small variation made to complicate the game. The SEMI project was aimed to experiment new methods of treating autism spectrum disorders to be included in already existing rehabilitation programs. These methodologies, making use of special computer technologies, are not to be considered alternatives to other already existing and well-established ones, but rather an enrichment of the same, creating an integrated approach to the disorder that over time has shown to be of greater effectiveness. Despite its experimental nature, the prototype systems developed have shown very interesting potential. These systems will continue to be used by the MAiC foundation, for further testing campaigns in which new data will be collected. Based on the data collected, it will therefore be possible to lay the foundations for designing and implementing a new, more advanced and robust system version, integrated with AR and VR systems.

4. Discussion

We described the implementation and the evolution over the years of an interactive, computer-based system based on real-time image processing, which reacts to movements of a human body with graphics and sound. Looking at the three systems presented it is clear how we are trying to move towards the realization of a complete system, that on the one hand is user friendly to the children, giving him/her the impression of participating in a game, on the other hand, it is accurate enough to useful for a psychological assessment as accurate as possible.

The third version, SEMI project, developed a set of interactive games organized by levels of difficulty, taking account of the sensory profile of every single child involved. In particular, the prototype has been thought to improve gross and fine motor skills and the enhancement of imitative aspects to foster social interaction and personal autonomy of the child. The children involved in the project were all in charge of weekly rehabilitation treatments in a specialized rehabilitation center, followed by several specialists. All the results have been encouraging, all the children have faced the experimentation with enthusiasm and, during the sessions, all showed improvements in attention and response time. The evaluation has increased the score in almost all aspects considered, and particularly those related to motor coordination. In the children who participated in the experiment, compared to the control group, an improvement in balancing tests and in hands-movement testing was found. Extrapolated overall trends reported progressive improvement in the children’ skills. The use of this instrumentation has allowed children to perform exercises in a motivating and enjoyable climate. A high level of acceptance by professionals was observed and parents’ feedback was also extremely positive. The models used seem to be promising tools to improve emotional and social skills. In addition, all the children in the sessions showed greater ability to self-control the movement as well as the ability to select specific motor areas.

Analyzing the results, we need to consider some limitations of the study, for example the small sample size. The study was conceived substantially as a feasibility study so only a limited number of patients were recruited. The results of the study should be replicated in a larger sample to demonstrate possible application in a clinical scenario. A more complete sample will also allow to evaluate how different the response can be with different types of children with ASD. Finally, the effects of external factors (e.g., family involvement and health conditions) that may have influenced the improvements found were not considered.

Further studies must be carry on by our research group to improve the system. Disadvantages of the proposed system, raised during the experimentation were the following:

• given the broad spectrum of ASD there is a need to design a person specific intervention protocol;
• the need for trained therapists and the economic cost of such treatments;
• parent training and regular monitoring to check whether the parents properly implement the intervention protocol;
• For the exploratory nature of the design this was a feasibility study aimed to assess the usefulness and acceptability of the platform, further elaboration has to be carry on for a clinical validation;
• the proposed system has to prove its ability of generalization to everyday life.

We are continuing our experiments using Microsoft Kinect v.2 and other gestural interfaces like the Leap Motion, more suitable for fine motor praxis dysfunction. We also started to experiment with Virtual Reality based systems. VR has become a popular platform used in ASD interventions and research has delivered successful results in areas including social and cognitive training exercises [46]. Virtual reality enabled training for social adaptation for school-aged children with ASD [47]. VR/AR based systems, multisensory experiences and improved immersion (including audio-visual installations, haptics/tactile) etc. can be used to allow exploration and practice of social activities [48]. We have developed a virtual environment in which some Kandinsky paintings from the abstract period (1920–1930) are displayed. We gave an interpretation of the tridimensional position of the single graphical elements of the paintings, placing them in the virtual space.

The user, wearing an Oculus Go virtual reality helmet, can enter the representation and interact with the individual elements, moving them in space (Figure 11). At the end of the manipulation it is possible to return to the initial vision of the painting, observing the effects on the original painting of the three-dimensional transformations.

This pseudo-game does not ask the subject to execute a precise task, but it lets him to freely express himself with the arrangement of the elements, in some way following the DIR paradigm [9]. At the same time, thinking the effect of 3D manipulation on the final 2D view requires an abstraction skill, not trivial for subjects with ASD. Moreover, the graphical object manipulation requires a precise motor coordination so, considering that dyspraxia associated with ASD is very common, this kind of exercise can be very useful.

We are currently finalizing this system, and its experimentation on young subjects with ASD will start soon.

Multidisciplinary working groups composed of engineers, programmers and doctors are essential for the realization of therapeutic tools. Interventions based on augmented reality technologies or multichannel augmented interaction are a powerful tool because they can reduce the main symptoms of ASD with engaging approaches to multimedia interaction and creativity.