A Systematic Mapping Study in Cloud for Educational Innovation

Abstract

For some years now there has been a growing use of cloud computing technologies in education. This paper reports a systematic mapping study (SMS) about the interest in the innovation of methods, techniques, and tools applied to teaching activities based on the use of cloud computing. The SMS presented here was designed using a consolidated, reliable, rigorous methodology and implemented in a replicable and verifiable manner. The process adopted is based on the selection of papers in accredited online digital libraries. This research made it possible to identify and analyze the empirical evidence in relation to the use of cloud computing in education with the aim of identifying both the main topics currently shared by researchers and the less explored areas to be integrated into a research agenda. The results show that only 17% of the 940 works examined report empirical research about the innovation introduced in educational environments of all levels and degrees by the cloud. The results also show that the type of method most widely used was that of feedback. The areas that have attracted the most interest from the research community are those of the new learning environment, collaboration platforms, and virtual laboratories. However, since the same areas are less supported by empirical research than others, we believe that these should be more closely monitored by the scientific community.

1. Introduction

Teaching today also means designing non-linear, sinuous, often iterative learning–teaching processes with ongoing reviews.

Training processes are key factors in building the knowledge and skills necessary for inclusion and reintegration into the labor market; therefore, they cannot be separated from a social context increasingly permeated by the use of modern technologies, among which the cloud is of particular interest.

Both research and the market have identified in the cloud-based technologies strategic factors for development in the field of education, which, inter alia, enable access to online services from everywhere and promise scalability, extended availability and cost reduction. These features allow cloud computing to favor the use, in comparison to traditional training, of computing infrastructures where hardware and/or software change skin to become accessible services online.

The aim of this work was to realize a systematic mapping study (SMS), aimed at collecting and analyzing the most significant studies on didactic innovation introduced by the use of cloud computing published in journals or presented at specialized international conferences in recent years.

This study was carried out to outline the features of the solutions proposed in the papers taken into consideration, in order to offer to the scientific community a useful tool to promote critical awareness about the areas of interest and to focus on the topics that are most in need of research.

As reported in Fernandez et al. [1]: “A systematic mapping study provides an objective procedure for identifying the nature and extent of the research that is available to answer a particular research question. These kinds of studies also help to identify gaps in current research in order to suggest areas for further investigation” (see also Appendix A).

Although this research sector is investigated from many aspects, as highlighted above, there is however no analogous availability of secondary studies and structured collections that summarize the data relating to the actual didactic innovation produced by the use of this technology.

In fact, due to both the heterogeneity of the fields of use of the cloud and the nature and complexity of the field of education, it is not easy to trace a homogeneous collection of studies in the literature concerning the effectiveness of the cloud in teaching innovation. Based on the above considerations, our first goal was therefore to identify and analyze the empirical evidence on the use of the cloud in didactic activities. This was done in order to lay the foundations for a new declination of the concept of cloud-learning associated with the ubiquitous learning paradigm, thanks to integrated technologies that allow access to a series of services. These services were aimed at the dematerialization of space and time by means of a content production infrastructure also powered from the bottom in a WebStore logic.

The paper is organized as follows: Section 2 illustrates the background; Section 3 describes the research method; Section 4 explains the research questions used in the work; Section 5 shows the experimental process; Section 6 points out the results; Section 7 displays the discussions; Section 8 sets out the conclusions.

2. Background

The education processes are key factors in the process of building knowledge and skills necessary for inclusion, and re-inclusion, in the labor market; therefore, they must take into consideration the social context where the use of modern technologies is massive. In particular, both research and the market have identified in the cloud-based technologies strategic factors for development in the field of education, enabling access to universal online services and promising scalability, extended availability, and cost reduction. These features allow cloud computing to favor the use, in comparison to traditional education, of computing infrastructures, where hardware and/or software change to online services.

However, while the academy converges, considering the use of these technological advancements in education, some advantages include cost, elasticity, greater availability of resources, and user satisfaction. On the other hand, the risks are evident in respect to security, data security, and robustness of the features offered.

The main problems identified in this sector concern the provision of services in cloud architecture specifically designed for education and experimentation in real teaching.

Cloud computing is an innovative way of using Information and Communication Technology (ICT) in which flexible and scalable capabilities can be delivered on-demand as a service. In this way, data [2], software, but also platforms and hardware infrastructures are available to everyone through an Internet connection. It is called cloud because data and applications reside at remote servers connected via the Internet, even thousands of kilometers away. According to NIST [3] the National Institute of Standards and Technology of the US Department of Commerce, the cloud is characterized by:

• Five essential characteristics related to the mode of service delivery: on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service.
• Three models of services that can be offered: SaaS, PaaS, and IaaS.
• Four distribution models: private, public, community, and hybrid.

Due to characteristics of cloud computing (flexibility, computing power, scalability, and economy, as the cost is based on use), it is an excellent solution for educational institutions, that often have reduced budgets for computers and network devices [4] that are suitable for the most complex experiments.

The diffusion of the new Information and Communication Technologies put the school and the educational system in front of the need for adaptation to the new digital technologies. In addition, the problem of “digital competence” in schools has become crucial in the current debate, and many institutions (OECD, EU, UNESCO) are promoting initiatives aimed at encouraging the updating of education systems to match an increasingly digitized society.

Therefore, the formulation of new educational paradigms, that take into consideration the changes taking place and the availability of new tools and technologies, appears as an essential element for the implementation of the student-centered model, introduced by constructivism.

Although in many educational institutions such technologies are already massively used, the processes that integrate these tools in the various educational activities are consolidated and the potential of these systems still remains partially unexpressed. Specifically, the use of cloud technologies and all that concerns the cloud-learning.

Since the concept of the cloud is still in the evolutionary definition phase, many research groups, either academic or industrial, are trying to decline cloud computing in concrete application contexts. Therefore, this is now seen as a series of application services available on the network with pay-per-use logic. At the same time, cloud-learning is considered as a set of services aimed at supporting educational processes. An example of this vision is the fact that “Educational institutions are [...] beginning to use lower level cloud services for such data storage”, as Sclater writes [5]. This practice becomes striking when audio/video documents are provided as open educational resources and then data security is of lower concern.

However, while on the one hand some of the benefits of cloud computing for academic institutions are economics, elasticity, enhanced availability, lower environmental impact, concentration on core basics and end-user satisfaction [6,7,8,9,10]. On the other hand, similar to other areas of science or medicine, it has some risks such as security [11,12,13,14,15], data security, un-solicited advertisements, lock-in, functionality, platform, and technical issues [5].

This considerable interest in the use of cloud technology has led over time to the production of numerous primary studies aimed at increasing the understanding of the problems related to its use, and also outlining the link between this technology and the multiple aspects of teaching.

Therefore, in addition to having the availability of mapping studies that generally investigate the state of the art of the use of the cloud computer [16], or that explore specific problems such as that of security [17,18], it is important to be able to make use of secondary studies and mapping studies aimed at exploring this area of research which provide a structured map of the aspects inherent to the use of cloud technology in the educational field.

Among the studies developed for this purpose, we find that the study conducted by Martínez et al. [19] outlines and examines the state of the art of research on the use of cloud computing in teaching and also the advantages and disadvantages of using this technology, which are examined and highlighted.

Another contribution that provides a structured map of cloud research and teaching is that of Baldassarre et al. [20], which examines the literature on cloud computing for education and analyzes whether research is developing with adequate empirical validation.

A particular aspect investigated among secondary studies is that of "mobile cloud computing". In this context, an overview of mobile cloud computing for teaching and learning [21] is available, which outlines the main research categories, the main methods, and the current state of the sector in terms of research opportunities.

The aspects related to the adoption of cloud technology in higher educations or universities are also investigated in numerous studies, including for example [22,23,24,25,26], in which opportunities provided by the use of the Cloud for these institutions are identified and highlighted.

3. Research Method

The method followed takes its cue from what has been consolidated in the literature [27] and is defined in the following phases schematized in Figure 1.

The steps in Figure 1 are detailed in the next sections.

4. Research Questions

In order to collect and analyze data relating to the state of research on the topic of cloud computing and didactics, this SMS has focused on the following two areas of research:

• Topics and types of innovation dealt with;
• Research characteristics.

In each of these areas, the research questions were identified and defined, as now detailed.

4.1. Topics and Types of Innovation Dealt With

This area proposes to perform the qualitative analysis of the papers and has been articulated on the basis of the research questions RQ1 and RQ2.

4.1.1. RQ1—What Topic is Covered in the Paper?

This question aims to highlight the possible existence of one or more topics shared by researchers in the sector or to verify the existence of a common research agenda. To this end, the taxonomy envisaged by Baldassarre et al. [20] was adopted, as follows:

• Cost Saving: cost reduction, economic aspects of the introduction of cloud computing in training institutions.
• Collaboration Platforms: virtual collaborative environments that can increase the effectiveness of distance education processes by exploiting cloud technologies as well.
• New Learning Environments: proposals, solutions, architectures, platforms, etc., based on cloud technology, to integrate, improve, share resources, educational activities, etc., in training processes.
• m-Learning platforms: aspects of mobile learning and its integration with cloud computing.
• Virtual Laboratories: systems for creating and managing virtual remote laboratories aimed at improving the way in which experiments and practical exercises are conducted.
• Content Innovation: cloud-based solutions to more effectively structure content oriented to students with different characteristics.
• Content Sharing: specific issues related to content federation between different courses, different universities, etc.
• Security: improvement of end-to-end security in order to preserve user privacy and ensure the integrity of stored and/or exchanged information.
• Campus Management: integration and/or migration of IT-infrastructure or non-IT components into IaaS systems and solutions to make global university resources more accessible to a wider audience.
• Digital Divide Reduction: reduction of educational and training gaps in specific geographical areas also with the aim of reducing the digital divide.
• High Performance: execution of complex applications that require large calculation capabilities but for short times.
• Storage Sharing: specific proposals related to the large amounts of storage that are required for storing training modules and support materials.
• Student Response System and Evaluation: interaction between students to improve their learning experience.
• Games: systems to create and manage the learning process in which learners are called to move in-game dynamics, following precise rules, and covering defined roles.

The question provides that each paper can be assigned more than one topic.

4.1.2. RQ2—What Kind of Innovation Is Analyzed in the Paper?

Considering each of the topics in which the papers were taxonomized (see research question RQ1), this question highlights the type of innovation indicated by the researchers and their contributions.

To this aim, three innovation classes have been defined:

• Methods, with reference to the process that uses techniques and tools;
• Techniques, with reference to the way in which a single activity carried out in the process is performed;
• Tools, with reference to the instruments used in carrying out the activities.

4.2. Research Characteristics

This area, divided into the queries RQ3, RQ4, and RQ5, aims to identify the essential characteristics of the primary studies analyzed and detect empirical studies on which to continue the work of research.

4.2.1. RQ3—Has an Empirical Evaluation Been Carried out?

The question aims to quantify how much empirical research is carried out in the reference area. The question also aims to select the papers on which the investigation should continue. All papers on non-empirical research were no longer analyzed in this SMS.

4.2.2. RQ4—What Kind of Experiment Was Conducted?

This research question examines the papers selected through RQ3 and aims to describe the type of experiment conducted according to the following classification:

• Case Study: this typology occurs when the authors focus the empirical investigation in a single case study, generally performed “in vivo” in the business context, or by carrying out a retrospective survey on one or a few projects which are reported by the statistical data but without carrying out a real analysis of validity.
• Experiment in the field: it consists in making one or more variations of one or more independent variables that contribute to determine a process.
• Controlled Experiment: this type includes experiments carried out in the laboratory for which all the variables are controlled by the experimenter.
• Search Action: it aims to introduce experimentation in concrete/real situations; the validity of the theories or hypotheses that it generates depends not so much on scientific verification of the truth as on their usefulness in helping people to act in a more intelligent and skillful way. In Dickens’s view, “action research in the process of researchers participating in studies both as subjects and objects with the explicit intention of bringing about change through the research process” [28].
• Critical Analysis: this model applies to primary studies and original research using the quantitative methodology in order to evaluate its validity, reliability, and applicability.
• Simulation and Test: it consists in a model of reality that allows to collect experimental data starting from the dynamic unfolding of a series of events or processes based on particular conditions defined a priori by the researcher.
• Survey: it consists of a quantitative social research technique, which is a detection procedure, which allows the collection of data through the administration of questionnaires.

4.2.3. RQ5—Does the Analyzed Paper Contain a Comparison with Other Similar Experiences?

In order to assess the amount of research characteristics of extemporaneousness that are compared to what is done for the possible validation of similar analogous research, the question, with a Boolean answer, aims to quantify the number of papers that present a comparison with similar experiments.

5. Conduct Search

The methodology used in this SMS first identifies the authoritative data sources, thus allowing the search for papers. This is done according to a search strategy that allows the evaluation of the completeness of the research that does not support any hypotheses preferred by the research group, but rather identify and report objectively the state of the research. The next phases were those of definition and validation of the search string, of individuation of the relevant literature, and finally that of quality assessment study and data synthesis.

5.1. Data Source and Search Strategy

The completeness of an SMS depends, obviously, on the choice of the repository used for the identification of the papers to be analyzed. In this SMS, the four digital library specified below were used:

• IEEExplore (https://ieeexplore.ieee.org)
• ACM (https://www.acm.org/)
• Science Direct (https://www.sciencedirect.com)
• Springer (https://www.springer.com)

The accuracy of the research process is a factor that distinguishes the SMS from traditional reviews, this rigor is strictly related to the quality of the search strategy [16,17,18,20].

The research strategy adopted here is of an iterative nature and has benefited from preliminary research aimed both at identifying any pre-existing mapping studies and at assessing the volume of potentially relevant studies.

The various combinations of search terms derived from the initial research questions have allowed us to identify a list of synonyms, abbreviations, and alternative spellings; all these details allowed the definition of the search string.

5.2. Definition and Validation of the Search String

In an SMS, the process that allows us to identify and define the search string to be used for the selection of the papers to be analyzed is fundamental. To build a research string that automatically ensures the extraction of a pertinent literature study sample for the purpose of this paper, one must know the keywords most used in the literature. These words have been derived from the procedure described in [20], also referring to string validation.

Then, to formalize the search query it was decided to use this: (Computer Aided Instruction OR Educational Institutions OR Electronic Learning OR Elearning OR Teaching OR Education OR Educational Courses OR Distance Learning OR E-Learning OR Learning Management System OR Distance Education) AND Cloud Computing.

5.3. Extraction of the Relevant Literature

The research was carried out by extracting the papers, published in the years 2012 to 2017, from the four repositories considered, as defined in the paragraph “Data Source and Search Strategy” and respecting the research strategy reported in [20]. A further selection was made on the criteria of quality assessment and data synthesis, in particular selecting only the papers written in English and fully available. The population and distribution of the papers covered by the SMS turned out to be

Table 1. Number of papers extracted according to the quality assessment criteria.

Repository	Number of Papers
IEEE	3826
ACM	1568
Science Direct	456
Springer	4230
Total	10,080

.

5.4. Included and Excluded Study

This process is divided into three steps:

• First phase: based on the title and adherence;
• Second phase: based on the abstract, if referred to educational innovation;
• Third phase: based on the papers related to experimental activities.

Each selected publication was evaluated, for inclusion and exclusion, by two authors who worked independently. The algorithm adopted monitored the equal attribution of the papers to the evaluators. At the end of the evaluation, several board meetings defined the definitive table of paper inclusion/exclusion. When the opinion about an article was not consistent, a discussion was opened, with a maximum time of 15 minutes, between the auditors with the aim of reaching the unanimous agreement. In cases where the agreement has not been reached, the issue has been resolved by the whole group by majority decision.

5.4.1. First Phase of Inclusion/Exclusion

In the first phase of inclusion/exclusion, all the papers have been viewed and selected basing on the title and evaluating their adherence to the subject matter of the SMS. The selection produced a number of papers shown in

Table 2. Number of papers extracted according to the quality assessment criteria.

Repository	Number of Papers Extracted	Number of Papers Selected
IEEE	3826	547
ACM	1568	40
Science Direct	456	88
Springer	4230	273
Total	10,080	940

.

5.4.2. Second Phase of Inclusion/Exclusion

The second inclusion/exclusion operation was carried out on the basis of the abstract, evaluating if referred to educational innovation. This operation produced the results shown in

Table 3. Papers selected in phase 2.

Select Phase 1	Excluded Phase 2	Included Phase 2
940	616	324

.

5.4.3. Third Phase of Inclusion/Exclusion

The third operation of Inclusion/Exclusion was carried out on the basis of the papers relating to experimental activities. As indicated above, the research question RQ3 is crucial to exclude all non-empirical research work from the SMS. The Boolean answer to the question allowed the selection of the results highlighted in

Table 4. Papers concerning empirical research.

Select	Excluded	Included
324	164	160

.

6. Results

The systematic mapping study permitted the collection and analysis of data related to the presence, in the literature examined, of papers containing empirical evaluations about the use of cloud technology in education and, specifically, the type of empirical investigations carried out.

Figure 2 demonstrates that the editorial placement of the articles highlights a difficult placement of research in international high-level journals (just the 36.31%), while the dissemination in international and sectoral conferences and workshops is good (63.69%).

Editorial placement is important to understand how the academic community—while recognizing the importance of using cloud computing as an innovative tool to innovate learning teaching processes—considers these experiments to be exclusively applicative and not up to the level of a scientific research assessment.

Taking into consideration that each paper can be assigned to more than one topic, the result obtained through the research question RQ1 shows that the topic is most diffused in the research work considering the cloud as a new learning environment (49.54%), see Figure 3. Instead, as presented in Figure 4, this result increases to 59.38% by limiting the analysis to the papers with an empirical evaluation. The argument is, obviously and rightly, suitable for experimental analyzes whose data can tell a lot about the goodness of the initiative addressed.

The problem concerning the platforms for collaboration is also very much discussed, in fact, 22.15% of the papers analyzed reported, for various reasons, the experience of the use of various LMS, both proprietary and open-source, and totally remote or blended. The use of cloud technology for virtual labs has involved many researchers who produced 19.69% of the papers. This result remains almost unchanged if we take into consideration only the research works that report an experimentation.

Furthermore, taking into consideration only the set of papers dealing with virtual laboratories, the results show that 44.00% propose an empirical evaluation while 56.00% simply describe the experience in general terms.

The papers analyzed showed that the cost analysis of an e-learning initiative is a very sensitive topic (16.62%). Many authors have highlighted the advantages from an economic point of view of the adoption of the cloud: some of them underline that specific activities, experimentation, etc., cannot be realized due to high costs, while the use of cloud computing makes it possible to develop them.

Interesting themes are considered also the management of content sharing (13.54%) and their innovation (14.46%). Many papers report comments, opinions, and enthusiastic evaluations of the students who consider these initiatives a useful and effective way of fruition, as well as closer to their daily life.

Growing, over the last years, is the interest in m-learning (13.85%). Surely in this case the spread of smartphones, tablets, and mobile devices in general and of BYOD policies (bring your own device) promoted also in educational institutions, contributes significantly.

The topic related to security appears disregarded. Most likely many authors were not technically skilled and perhaps less sensitive to the improvement of the end-to-end security and the user’s privacy. The revealing introduction of data protection regulations worldwide did not produce a change in the trend of the data collected, and therefore it is believed that this can be an issue of immediate investment for research in this sector.

Other topics identified in the taxonomy have also been neglected, as shown in Figure 3. They are research sectors where it is probably better to invest in the next future in order to consolidate the cloud technology in education. In this sense, think about the use of games in education and the impact they can have on the training of learners, especially K12.

Figure 5 presents the results of the research question RQ2 in which the analysis of the papers showed that the most indicated innovation is related to the tools.

In fact, out of a total of 324 papers, as many as 153 papers consider the cloud as an innovation related to the tools, while only 26 papers deal with innovation in the technique and 91 with innovation in the methodology. It should also be noted that in 54 papers the type of innovation introduced with the use of this technology is not explicitly mentioned.

In this context, the data collected show that the researchers consider the cloud as simply a new tool for carrying out teaching activities rather than an element capable of modifying the same mode of delivery and use, and certainly not as a tool able to modify the process. In this regard, it would be interesting to investigate this issue from a pedagogical and didactic point of view.

The research question RQ3 was a reason for further inclusion/exclusion in the SMS (see Table 4). 49.23% are experimental papers while the remaining 50.77% do not present an empirical evaluation of the experiment conducted. This question allowed us to select only 160 papers for the remaining part of the SMS. Although the theme is well suited to the research of an experimental nature, the result highlights the difficulty of most of the researchers in the sector of making a quantitative analysis of the advantages produced through the use of cloud technologies in education.

The analysis of the research question RQ4 highlighted that surveys (28.75%), experiments in the field (24.38%), and case studies (21.88%) are very frequent, as shown in Figure 6. In particular, surveys show the positive character of the experience even if, due to their intrinsic quality, they do not make it possible to taxonomy the benefits induced by the use of the cloud in education.

The results related to the type of experiment adopted highlight the fact that the interest of researchers is in most cases aimed at investigating user satisfaction in the use of cloud technologies for educational activities and/or his propensity to use it.

Lastly, based on the research question RQ5 only 3.13% of the papers made comparisons between the experiment conducted with others similar. Researchers should make more efforts in this direction in order to advance research, starting from similar experiences to compare with their own for a better evaluation of the results produced.

7. Discussion

Consistent with the two main research areas indicated in Section 3, the discussion of the results focused on the “topics and types of innovation dealt with” and on the “research characteristics”.

The qualitative analysis carried out showed that the topic most shared by researchers in the field is that which considers the cloud as a flywheel for the creation of new learning environments. It is a widespread idea that the way of training must be updated, as the performance of schools of all levels can be improved with the introduction of ICT tools suitable for digital natives, who already use them in familiar contexts and leisure.

In describing the new learning environments supported by the cloud, many papers focus on the description of the e-learning architecture model, on the performance of web services, and on the possible solutions offered by the cloud. Furthermore, it is of a shared opinion that the main factors influencing the adoption of a new cloud-based learning environment are substantially related to costs, which in the 50% of cases are assessed in correlation with the new learning environments and platforms for collaboration, and to the usability of the e-learning environment, essentially referred to the effective interaction and user-friendly interface.

Among the advantages of introducing the cloud into the training process, many papers point to the possibility of supporting the creation of a new generation of multiplatform e-learning systems able to provide a storage commensurate with the needs of data storage.

Since these tools require investments that are often not available to many educational institutions, it is widely believed in the papers considered that the cloud can be the key to lowering costs and allowing experimentation with environments that are not otherwise accessible due to economic reasons. In general, it is found that thanks to the cloud, schools do not need to equip themselves with very expensive ICT equipment, and do not need to dedicate human resources with system skills, since it is naturally easier to activate the services necessary for experimentation quickly enough. The advantages of cloud solutions are also evident when it comes to scalability. In fact, many papers highlight the substantial difference compared to outsourced solutions in terms of processing and storage scalability.

In a world that integrates mobile systems more and more into everyone’s daily life, the cloud applied to mobile learning appears, in the analyzed articles, as an indispensable commodity for collaborative learning activities. In the papers classified in the topic M-learning Platforms, the main factors influencing user satisfaction and their propensity to use mobile technology in learning contexts are studied and evaluated.

In general, a further consideration that can be deduced from the analysis of papers and the evidence reported in the answers given to the second question “What kind of innovation is analyzed in the paper?” is that the cloud/teaching topic is often dealt with in general terms, without focusing on the assessment of specific and possible benefits in terms of techniques, methods, and tools.

The papers often report experiences conducted in the laboratory, aimed more at evaluating the impact on learners for using the tool used in the experimentation than the peculiarities and specific benefits of the cloud in terms of teaching methods and techniques.

This consideration is strengthened by the results of the research question RQ1, where it is clearly highlighted that issues concerning the use of the cloud as an instrumental facility, such as the New Learning Environment, Collaboration Platform, and Virtual Laboratories, take precedence over the topics related to the methodologies didactic.

In the authors’ opinion, on the other hand, themes such as Content Innovation and Content Sharing should find more space in discussing the common agenda as strategic elements for innovation in the education and training process.

For example, we think that the availability of resources in the cloud should favor innovative teaching methods such as that of the flipped classroom [29,30].

In fact, the ability to access a multitude of resources, available on dedicated PaaS platforms, facilitates the experimentation of the inversion of traditional teaching moments, allowing to use the classroom time not to expose the contents of the various topics to the students, but to work in a team to transform the knowledge acquired through different cloud channels into skills.

In this regard, it would, therefore, be desirable to overcome the exclusively technical vision of the use of ICT in teaching towards a methodology of innovation of educational planning itself.

As for the characteristics of the research, it emerges that about half of the papers do not make an empirical evaluation, while empirical research allows the production of data on which to base evaluations on the growth opportunities of the sector of interest, thus allowing the transformation of basic research into industrial research. The potential impact on the industrial sector of the transformation of cloud-based e-learning environments is widely perceived, but perplexities remain due essentially to the difficulty of assessing the return on investment. The availability of quantitative and qualitative data on the use of a given technology can provide investors with a system on which to base the decision-making process aimed at evaluating the investment opportunity and maximizing its economic return.

Unfortunately, what is missing among the papers analyzed is precisely the assessment of the return on investment in cloud technologies in terms of increased productivity. Likewise, quantitative evaluation in terms of teaching effectiveness of the experimented action is lacking.

Regarding the orientation of the type of research found, it is interesting to note that a considerable number of papers present the results of surveys aimed at collecting data on the liking or propensity to use cloud technologies in education.

This, in our opinion, shows how this phase is still a phase of consolidation of empirical research in the sector in question. In fact, most researchers seem to consider this as a ground still to be explored, and remain limited to investigate and evaluate the eventual liking, by users, of the cloud technologies used rather than gathering empirical data about the teaching effectiveness resulting from the use of this technology.

The evidence that part of the research in this sector is still in an embryonic phase is also due to the fact that researchers highlight the positive results of their investigations, without even indicating the most critical aspects of the use of this technology in education.

In our opinion, this hypothesis is further strengthened by the lack of comparisons with similar experiences that, on the other hand, would allow a more solid scientific progress.

8. Conclusions

The high number of papers extracted and analyzed highlights the great interest of the researchers about these themes.

The areas that have attracted the most interest from the research community are those of the new learning environment, collaboration platforms, and virtual laboratories. However, since the same areas are less supported by empirical research than others, we believe that these should be more closely monitored by the scientific community. In absolute terms, the topics of security and digital divide reduction have been found to be less investigated, which consequently requires greater research by the scientific community.

The results obtained perfectly capture the situation currently created in the world of education where cloud technology is widely used in every order and degree but it is critical from the point of view of security. The analysis of the papers presented since 2012 show more frequently that cloud computing is no longer seen as a technology that innovates learning–teaching processes. More papers show that the cloud facilitates the iterative nature of the same educational pathways and facilitates on-going assessments, content sharing, and information circulating among students, thus favoring the learning process of the main actors.