Constructionist Learning Tool for Acquiring Skills in Understanding Standardised Engineering Drawings of Mechanical Assemblies in Mobile Devices

Abstract

The purpose of Graphic Design is to transfer information about design into reality and concerns the analysis, design and representation of mechanical components and assemblies. For the correct rendering of mechanical components, this discipline requires a command of, and the ability to, manage techniques and systems for graphical representation and standardisation; 3D models in a virtual environment enable engineering students to develop graphical skills and spatial awareness. The objective of the present study is the development of an application for smart devices (mobile phones and tablets), based on the constructionist theory of learning, which will enable first year engineering degree students to acquire the technical drawing knowledge and skills necessary to render mechanical assemblies. The mobile application tested and designed in this work is called ARPAID. It is a learning tool aimed at teaching students about the representation of mechanical assemblies as part of an engineering Graphic Design course. Teaching material and a process for evaluation have been designed. A detailed description is given of a classroom activity accompanied by a tabulation and analysis of the results obtained. This mobile application, when used in a Graphic Design course, promotes a more rapid understanding of spatial relationships and problems, fosters students’ learning and motivation, and develops higher order skills. Results from before and after the use of the application will be presented and do indeed show significant improvements in student performance.

1. Introduction

Over the last few years many Engineering Schools have updated their plans of study. These new plans have begun, little by little, to incorporate new technologies, reflecting how the university education system is adapting to the world of industry 4.0.

The world of today is a network of instantaneous communication systems where information can be accessed or sent on the spot. The society of the 21st century is interactive, information is portable, and connectivity almost total [1]. One of the principal academic strategies set out for education in the EU’s document, Horizon 2020 [2], is the incorporation of Virtual Learning Environments and Augmented Reality (AR) into higher education.

The subject of Graphic Design is transversal and technological in nature. It embraces the techniques of graphic communication to express ideas and concepts and it could be defined as a technology that coordinates intellectual and instrumental skills [3]. It is an area of knowledge that focuses on the analysis, design, and representation of mechanical pieces, mechanisms and assemblies, components, installations, plots, etc. To ensure the correct rendering of mechanical assemblies, this discipline requires a command of, and ability to, manage techniques and systems for standardised graphical representation.

Virtual Learning Environments and Augmented reality (AR) are technologies with far reaching potential in many disciplines, including education. AR is a hybrid reality that combines virtual and physical information in real time; that is, through the use of technology, virtual information is added to physical data about a real object or environment to create an artificial reality [4]. Alan B. Craig (2013), in “Understanding Augmented Reality” [5], defines AR as “a medium in which information is added to the physical world in registration with the world”. In his article entitled “A Survey of Augmented Reality” [6], Ronald T. Azuma (1997) defines three key characteristics of AR: the combination of the real and the virtual, real-time interaction, and 3D registration.

There is an ever-growing body of literature focusing on teaching and learning experiences with VR and AR. The majority of the applications have been within the compulsory schooling system (ESO: the Spanish Secondary Education system), and to a lesser extent in the university and higher education systems [4,7,8,9,10,11]. AR has been trailed principally in science subjects including: Biology [12,13,14], Mathematics [15,16], Physics [7,17], Chemistry [18], and Earth sciences [19,20], with some studies taking place in the Humanities and Arts [21,22,23]; the least explored field is that of teacher education [24]. All these studies agree that the use of AR in educational environments leads to a series of educational benefits amongst which we find: increased motivation and interest from students about the subject material; the development of creative skills; positive learning; increased attention, commitment and satisfaction; improved understanding of subject knowledge; improved educational outcomes; increased recall and self-sufficiency. The principal limitations described include: access to technology; technical problems with the App used; the small number of students participating in studies; problems with App usability; cognitive overload; poor development of higher order skills; limitations on the ability of educators to adapt the App; short duration of studies; emphasis on qualitative data.

Many research studies have demonstrated that technological applications increase academic achievement and student motivation; these effects are more pronounced for applications involving the greatest levels of student interaction with the environment. According to Bower et al. [24] the use of AR in education provides particular benefits for the acquisition of knowledge and learning content that may be abstract, difficult to understand or difficult to observe. It is an innovative technology that enables the visualisation of data and 3D modelling, and for this reason it is an ideal tool for the development of graphical skills and spatial awareness in engineering students.

In many Engineering Schools across our country greatest emphasis is placed on the acquisition of deep theoretical learning with the development of practical skills essential for future professional development being relegated to second place. This tends to demotivate students from learning subject content. In master and theory classes we observe high levels of student passivity, a lack of participation and a lack of active engagement behaviours.

The problem lies in the educational model (which is sometimes far too teacher centred) and the teaching body, which does not practice methods that promote active engagement. Student motivation and enabling students to see, at every opportunity, the potential, real world uses for the material they are working on has been one of the fundamental driving forces behind this investigation into teaching innovation.

The question we would like to pose is the following: is there a realistic way of solving the difficulties encountered in presential learning environments that is efficient, cheap, and accessible for all our students while at the same time provides the opportunity of improving understanding, spatial awareness, attention span, assessment, and tuition? Our answer is yes. We believe that the use of mobile applications based on Virtual Environments and Augmented Reality (AR) in the classroom, within a constructionist paradigm of learning, can provide us with the means to achieve, or perhaps it would be better to say “catalyse”, the achievement, to a greater or lesser degree, of the objectives laid out in our initial question.

In order to reverse the current situation, we need to modify the educational model and put in place methods that are active and constructionist. In this way, by allowing students to interact and manipulate the models provided, we hope to enable them to construct their own knowledge base in a way that is deeper and more enduring and also allows them to acquire more active classroom behaviours.

It is safe to say that practically all students in our faculty habitually use either a mobile phone or a tablet, and that these devices can all capture images of the world with a high-resolution camera. Consider then, the influence on learning and the acquisition of skills if we could develop an application that could superimpose onto this real image other information relevant to a classroom learning activity, and could do this synchronously, via a certain chain of events or “triggers”; an application with which students can interact, respond to requests for information or ask questions.

Through this educational proposal we hope to deliver a student experience that is both innovative and, at the same time, strengthens their knowledge of Graphic Design and develops their spatial awareness. Our method is based on presenting students with real-life problems, the understanding of which is facilitated through the virtual environment and AR such that they are enabled to use what they learn as a tool for solving problems, generating ideas, and developing improvements in the field of engineering. We hope to increase academic achievement, motivate and encourage student participation by combining the use of mobile phones or tablets, with AR, a cutting-edge technology that is also very attractive due to its links with video gaming. Using our application, students will be able to create, share, and use learning content throughout a given classroom activity; at the same time, this technology will facilitate the dissemination of resources at critical moments of the learning process. In addition, we will evaluate the app through an investigation comparing the achievements of an experimental group (using the app) compared to a control group (not using the app) to determine both qualitatively and where possible, quantitatively, whether our app improves learning outcomes.

2. Experimental Design: Methods and Tools

2.1. Educational Paradigms of Learning

A paradigm is a particular way of seeing the world, of interpreting reality, based on a specific philosophy. According to the author, Martin-Trujillo L., “a paradigm is a specific framework through which we view the world, understand it, interpret it, and intervene in it” [25].

The principal theories upon which we have based the design of our application are the following:

Constructivism embraces a set of psychological and pedagogical theories based on the idea that all educational input should have an impact on Human Development and that this is the fundamental objective of education. Learning takes place within the individual and arises out of the relationships and exchanges individuals have with their environment [25,26]. It is based on a model of active, constructive learning. Students should construct their own understanding of reality linked to their prior knowledge. Constructivism is organised around three basic concepts: the student is actively responsible for their learning process; knowledge is the result of a constructive process at a social level; and the function of the teacher is to enable the student’s process of constructing learning through direction and guidance. This theory maintains that instructive learning must provide macro and micro-level support to the student so they can construct their own knowledge, and be involved in meaningful learning [27].

One of the foundations of constructivism is cognitive development, as proposed by Piaget. Cognitive development is a set of transformations that take place throughout life and through which a person acquires new knowledge and improves their skills in perception, thinking and understanding. These skills are used to solve the practical problems of life. Piaget studied how children interpret the world at different ages and how they acquire knowledge as they develop [28]. The child has their own logic and ways of knowing, and these follow predictable patterns of development as the child interacts with the world and reaches maturity. Children actively construct knowledge about their environment using what they already know and by interpreting new happenings and objects. Piaget divided cognitive development into four stages: sensorimotor, preoperational, concrete operational, and formal operational. It is assumed that the child’s thinking is qualitatively different at each stage of development. According to Piaget, cognitive development does not simply consist of qualitative changes in experience and skills but rather in radical transformations in how knowledge is organised. The theory states that the acquisition of knowledge and the development of thought are achieved through a principle of adaptation whereby thinking is moulded by reality through two basic processes: assimilation and accommodation. Assimilation consists in incorporating those elements exterior to the individual to the structures of knowledge they already possess. Accommodation is the process that completes assimilation, that is, once an experience has been incorporated into an individual’s cognitive structures, a readjustment or re-accommodation is required to integrate the new knowledge with pre-learned knowledge.

Another constructivist theory is the sociocultural theory proposed by Lev Vygotsky which gives greater relevance to the social environment as a facilitator of development and learning. In contrast to Piaget, Vygotsky does not talk about assimilation, but rather appropriation [29]. Vygotsky’s theory is characterised by three fundamental factors and the interactions between them: interpersonal factors, historic-cultural factors, and individual factors. He stated that it is not possible to understand a child’s development without understanding the culture in which they grew up. He thought that the thinking patterns of an individual were not dependent on innate characteristics but rather that they were a product of cultural institutions and social activities. A child is born with basic mental skills, amongst which are perception, attention, and memory, and thanks to interactions with peers and more competent adults, these innate functions develop into superior mental functions. Vygotsky considers five fundamental concepts: mental functions; psychological skills; the zone of proximal development; thinking tools; and mediation [30].

Meaningful learning is a concept developed by David Ausubel and refers to a type of learning in which the student makes associations between new learning and previously learned knowledge, readjusting and reconstructing both sets of information in the process. Meaningful learning is a mechanism that allows both the acquisition and ordering of large numbers of ideas and information in any field of knowledge [31].

Discovery learning proposed by Jerome Bruner is based around the idea that what is going to be learned should not be given in its final form, but rather it must be reconstructed by the student before it can be incorporated into their existing cognitive structures. Discovery learning encourages mental development and there is nothing so personal as a discovery made by oneself, personally. The theory maintains that a student should acquire knowledge for themselves, guided by the teacher, but that their experience should be led by the discoveries they themselves make. Discovery is a form of inductive reasoning since students pass from specific examples to the formation of rules, concepts, and general principals. This type of learning is also known as problem-based learning, inquiry-based learning, or experiential learning [32].

Seymour Papert’s notion of constructionism coincides with constructivism in its conceptualisation of learning as “the creation of structures of knowledge”, independent of the circumstances of learning. However, his constructionism transcends this concept and establishes a set of circumstances in which learning becomes more meaningful, specifically the context of learning by doing. Papert maintained that “learning is hugely facilitated when the learner is consciously engaged in the construction of a meaningful social object or entity which can take many forms from a sandcastle, to a poem, to a computer program” [33].

In his book, “The children’s machine”, Papert talks of the important role these real-world social objects play as catalysts for those constructed inside the mind. As a result, Papert identifies two types of construction, one external and the other internal to the individual. Moreover, a successive cycle of learning is set in train: as the real-world object is built, knowledge is constructed in the mind and this knowledge gives the potential for the construction of more complex real-world constructions, which in turn generate further knowledge, and so on [34].

According to Papert, better learning is not derived through finding better methods of teaching, but through offering the learner better opportunities to construct. Fundamental considerations here are the importance of providing the student with adequate materials, the environment or surroundings, and indeed the social context in which learning takes place.

For a technology to be effective, it is necessary to provide novice learners with an easy way to get started (what Papert terms “low floors”) but also, over time, offer them a way of working on projects that are ever more sophisticated (“high ceilings”) [35].

A constructionist learning tool, therefore, allows students the opportunity to experiment, explore and express themselves freely, without a pre-established script, a fixed itinerary or timetable. By also providing the student with other ways to actively interact, i.e., through a screen, we are increasing the number of ways they can express their ideas and build their projects, In this way we are contributing to their development as Creative Thinkers or, in the words of Mitchel Resnick, creator of the graphical programming language, Scratch, and disciple of Papert, so called X-students [35].

2.2. Problem Definition and Objectives

Over the years that we have been teaching Graphic Design we have observed that many students have difficulties understanding the sections on Systems and Assemblies in regard to understanding their function, the position of pieces and the relative displacement between components. Understanding these issues is fundamental for correct, standardised representation of mechanical assemblies. Errors in understanding the mechanism leads, inevitably, to errors in their graphical representation.

The use of mobile applications has been studied by many authors and it has been shown to produce good outcomes in similar contexts. The characteristics of this technology promote and enrich the teaching and learning process; however, like any other technology it is necessary to have a precise understanding of the steps that should be followed for its successful incorporation into classroom practice. There are numerous investigations and research studies that endorse the didactic use of this kind of technology and analyse the factors that influence its use and uptake [36,37,38,39,40].

Our investigation hopes to achieve the following objectives:

• Taking a constructionist approach to learning, we hope to provide students with an innovative classroom experience through the use of the ARPAID mobile app and, at the same time, reinforce their knowledge of Graphic Design and improve their spatial awareness.
• Adapt learning and the acquisition of skills to the strategic use of ICTs, particularly Augmented Reality and Applications for Mobile Devices. Our method is based on presenting students with real-life problems in a virtual environment, so enabling them to understand and apply what they have learned as a tool to resolve further problems, generate ideas, and propose improvements to the engineering community.
• Increase academic achievement, both in traditional examinations and in individual and group tasks.
• Motivate and encourage student participation through the combined use of mobile phones and tablets and AR, a cutting-edge technology that is attractive due to its associations with gaming. In this way we will enable students to construct their own knowledge through the use of subject content provided via the app throughout the whole classroom learning experience.
• Enable the increased availability of resources to students at critical moments of the learning process.
• Assess results, through an investigation, evaluating qualitatively and where possible, quantitatively, the potential improvement in learning of an experimental group (using the app) and a control group (not using the app).

These objectives are focused on facilitating students’ meaningful learning. The authors [41] demonstrate that the teaching of engineering should not be based solely on theoretical concepts and indeed, that interaction with multimedia applications enables students to acquire knowledge more quickly. As cognitive theory tell us, it is necessary for students to think, understand, touch, interact, and reason through concepts for themselves [42]. As a result, we have designed an application that not only offers theoretical concepts but also shows students the real objects we work with. The application is built such that these objects can be touched, rotated, displaced, etc, and ultimately, internalised and understood. This process simplifies spatial understanding of mechanical assemblies so that students can interpret them correctly and be able to produce cross-section and assembly drawings of these systems. A further advantage is portability. Thanks to mobile phones and tablets, students can access learning content without the need for a computer. The accomplishment of these objectives will provide us with a tool that will be of great help to students in aiding them to understand concepts and resolve theoretical and spatial problems. We will attempt to evaluate these objectives through the assessment tools set out.

2.3. Investigative Method

Hypothesis: 

Use of the mobile application ARPAID in the classroom by students when working on individual projects will help promote meaningful learning of subject material related to technical drawing of mechanical assemblies.

ARPAID is proposed as a means of acquiring knowledge and enabling interaction with two of the assemblies students must resolve and draw as part of the practical assessment task in this subject. During the app design process, it was decided that, besides being able to manipulate mechanical assemblies it would be convenient for the app to have a facility for self-evaluation and to provide a set of activities and exercises focused on the deepening of learning, even some form of gamification: a serious or applied game.

A fundamental characteristic of ARPAID is the combination of synchronous usage in presential classroom sessions with the teacher, with asynchronous usage during individual tasks undertaken individually by students. Synchronous sessions will involve extensive use of the ARPAID’s AR capabilities, as students are provided with material and activities as part of their class-based lesson. A connection to a remote server holding a database will, in addition, allow for individualisation of tasks required of users.

Moreover, it was deemed valuable to enable an iterative process of assessing ARPAID: evaluating its usability and potential for improving the user interface and the app’s functionalities. This assessment was envisaged as a two-pronged process. On one hand, user attitudes were evaluated through interviews and satisfaction questionnaires and on the other, through the discreet and systematic collection of data concerning the variables and parameters of app usage which were stored and tabulated in a database for subsequent analysis.

Lastly, and thinking ahead to future initiatives, it was deemed necessary to open up the possibility for the app to handle a greater number of mechanical assemblies and practical activities such that it could have a higher degree of flexibility. This needs to be achieved without increasing the file storage requirements on the mobile device or reducing its performance, thus, we intend to use so called asset bundles comprising material located on a remote server that can be downloaded to a device as necessary, to be used by the mobile application.

For all these reasons and given the projected complexity of our investigation, we considered it convenient to break it down into three sequential stages. At the time of writing this article, the first stage is complete and the process of preparing for the second is underway.

• Stage I: Implementation of the ARPAID App

During the first stage, work will focus on the implementation of ARPAID with the basic applications that comprise its core functionality. Here, the aim is to provide students with a robust application, with essential functions fully developed, that is easy to use, intuitive and aesthetically attractive. At this stage, the objective is to verify, through a statistical analysis of data obtained, whether ARPAID use has a positive impact on subject learning.

• Stage II: Assessment of usability, user interface and app functioning. Incorporation of new functionalities, especially in AR features.

In the case that we confirm our hypothesis that the mobile app has a beneficial impact on students’ skills acquisition, the second stage of project development will be initiated. During this stage, the level of user satisfaction will be assessed with regards to learning content and practical tasks provided and we will also gather information concerning student opinions and feelings about using ARPAID. This process of assessing user experience is essential and needs to be systematised where possible, complemented by a stealth assessment [43], to obtain indicators for improvement and feedback to help optimise the app. Additionally, new features will be incorporated to complement essential functions. The following augmented reality features are worth noting: recognition of real mechanisms with superimposed virtual labelling (brand numbers, designation of parts, functional dimensions, indication of tolerances and fits, etc.); customized tests for each student during the class with questions depending on the analysis of the use of the app, and collaborative multi-user activities in the same location (co-located experience in a shared space). Once this is achieved, we will again investigate ARPAID’s impact on learning.

• Stage III: Externalisation with Asset Bundles

In this final stage of the project a new version of ARPAID will be created which will make use of UNITY 3D’s asset packages to allow access to a large archive of assembly models, problems, quizzes, tasks, and activities. To conclude the project, it is proposed that the analysis of ARPAID’s effect on learning should be iterated once again.

A summary of the proposed research project, with a stage-by-stage breakdown of essential activities, can be seen on the diagram shown in Figure 1.

Figure 2 shows a flow diagram of the first stage of the investigation. As can be seen, to verify our hypothesis, the student corpus was divided into two groups, one a control and the other an experimental group. A series of instruments (quizzes and practical tasks), were developed to be undertaken by students either before or after the experiment and which provided the data used in our statistical analysis.

2.4. Mobile Application. Development in UNITY3D

The ARPAID app has been developed for use on the two leading mobile operating systems, IOS and Android. It was made available to be downloaded for free on Google Play Store. Due to Apple’s terms and conditions of use, it was only possible to produce a test version of the app for a period of 90 days, using TestFlight. However, our students using iPhones or iPads were able to use the app without any limitations for the duration of this investigation.

In the successive stages of this project’s development, the ARPAID app will make wide ranging use of the possibilities afforded by AR on mobile devices, mainly from the second stage. A list of design elements used in our mobile application is shown in Appendix A. ARPAID was created using the UNITY3D graphic engine for videogame development [44]. The fundamental reasons we decided to use UNITY3D are as follows:

• Multiplatform design: executable programs generated in UNITY3D are supported on a wide range of devices.
• Capability to place three-dimensional models in a three-dimensional virtual space and tailor their properties using C# programming scripts.
• Facility for creating complex and dynamic user interfaces (UI).
• Outstanding graphics quality capable of creating realistic scenes, with high quality textures and complex lighting effects, achieved using the surface shader in the render pipeline.
• Possibility of linking and exchanging information with relational databases, on remote servers via the TCP/IP (MySQL-Apache-PHP) protocol.
• The integration of multiplatform Augmented Reality with AR Foundation [45]. AR Foundation functions as an intermediary enabling the use of AR on different platforms and devices independently of the various manufacturers’ proprietary solutions (HoloLens, Magic Leap, AR Kit or AR Core).

The initial version of ARPAID essentially comprises five UNITY3D scenes.

2.4.1. User Authentication and Registration Scene

This scene (Figure 3) allows the authentication of students so that they can login to use ARPAID, or their registration if they are accessing ARPAID for the first time. It also saves all relevant information, including the date and time that the session was initiated, onto a database hosted on an external server.

2.4.2. Main Menu Scene

Once authenticated, students proceed to a screen informing them of ARPAID’s two modes of use:

Synchronous Mode: Classroom use, for direct tracking, scanning of QR codes or images projected onto the whiteboard.

Asynchronous Mode: To enable ARPAID use outside the classroom so that students can complete individual tasks, and for teachers to provide notes or files containing relevant tracking codes for the subject being studied.

2.4.3. QR Code Scanning Scene (AR Tracking)

Via the main menu, students can access a scene that, through scanning and recognition of the relevant QR code (Figure 4), will upload a scene where they can manipulate a particular mechanical system. The decision was made to use QR codes, because in classroom situations, these monochrome codes were those best recognised by AR packages available on mobile devices.

2.4.4. Assembly Manipulation Scene

Once the tracking image has been recognised, the Assembly Manipulation scene is loaded. In this way, a copy of the mechanical assembly associated with the QR code is presented in virtual space. This is shown in Figure 5.

Besides the model of the mechanical assembly, this screen contains four zones, or panels, in which various different icons appear. These panels constitute the scene’s user interface (UI), and they are superimposed, with some degree of transparency over the view of the virtual space provided by the main camera.

The ARPAID user interface was tested and analysed methodically to achieve maximum convenience, speed, and ease of use.

The user can change the observation viewpoint through specific touchscreen gestures. The ability to visualise the assembly from any direction facilitates spatial understanding and allows students to ascertain the relative positions of one component in the assembly with respect to the others. Using a single finger, touching and swiping across the screen produces a rotation with respect to the X and Y axes (in UNITY3D the Y-axis is vertical). Rotation about the Z-axis is achieved via the horizontal component of finger motion but only when this occurs along a small band on the lower edge of the screen. With regards linear displacement, Z-displacement (depth) is equivalent to zoom and is achieved by touching two fingers on the screen. If the fingertips are moved apart, the camera approaches the object (positive zoom), while if they are moved together the camera recedes from the object (negative zoom). Panorama motion, that is, displacement along the X or Y axes, is achieved by moving sliders that can be found in the Visualisation Panel located in the bottom left-hand corner of the screen.

The relative displacement of assembly components, taking into account certain restrictions with regards to positioning and degrees of freedom for each one, is fundamental to internalising and understanding how the assembly works. In this ARPAID scene, the user can move each one of the component elements, by tapping a finger on the component and dragging it across the screen. To aid interpretation of the system, the movements of each element are restricted to assembly displacements.

If a component requires any other sort of motion necessary for assembly or disassembly of the system, a slider bar is added on the upper edge of the Visualisation Panel.

ARPAID uses two ways to limit the movement of components to within the geometric restrictions and positions of the real-world system: coding restrictions and collisions. Preference is given to coding restrictions such that limitations on the relative movements of components is defined by exact calculation of the system’s kinematics and are programmed using conditional strings. However, assembly displacements of components can leave them in random positions so complicating calculations. In these situations, the second option comes into play, that of defining the range of movement through the detection of mesh collisions between the gameobjects comprising the system.

The Visualisation Panel complements the options to change the observation viewpoint and relative displacement of assembly components which we have just described. This last panel is located at the bottom left-hand corner of the screen.

Special attention has been given to the design specifications of the Visualisation Panel such that it should be compact, easy to use, with self-explanatory icons. The central icon alters the viewpoint to give an isometric view. Around this central icon are six others used to access the six orthographic representations of the assembly being studied. In the bottom right corner of the panel is an icon that displaces the assembly’s components to show the model in exploded view. The icon in the bottom left-hand corner of the panel blocks all camera motion, so enabling a higher precision of component motion. The Reset icon, in the upper right-hand corner of the panel, reinitialises the viewpoint and positions of assembly components.

Lastly, the icon in the upper left of the panel reloads the QR scanning scene. Figure 6 shows a detail of the panel.

The Detail Part Panel is located in the upper right-hand corner of the screen. On this panel is a list of all the components in the assembly on which several actions can be performed. Being able to hide, reveal, or visualise with a degree of transparency individual components of an assembly is a way of deepening knowledge through the manipulation of—in this case, virtual objects and entities [33]. The possibility of showing components with some transparency, as seen in Figure 7, provides a novel view of a fully assembled working system containing internal parts that would not be possible in real life.

Each component listed in the Detail Part Panel (Figure 8) has an associated button that, if pressed, loads the Detail Part Definition and Manipulation scene.

Finally, in the top left-hand corner of the screen, an Information Panel has been added. Here, students can access all the relevant, necessary information to facilitate understanding of whichever assembly they are working on. In addition, they can find course activities that have been set and assessment and self-assessment quizzes so that their depth of learning can be evaluated. The Information Panel, shown in Figure 9, comprises three sections, each of which contains different categories of information. The first icon accesses information to help users navigate ARPAID. The second icon links to documentation concerning the assembly being studied: the types of components it contains; the number of each component it contains; what every component is made of; fits and tolerances; roughness and surface condition.

The third section of the Information Panel contains a series of activities and self-evaluation quizzes designed to determine the level of learning they have attained. The inclusion of a full framework for self-assessment and of stealth assessment [46] is expected to be part of the second stage of project development.

2.4.5. Detail Part Definition and Manipulation Scene

In this scene the focus is enabling user interaction with individual components of the assembly, outside the context of the mechanical system. Manipulating the assembly and how certain components move in relation to others is fundamental, however, it is no less important to have a detailed understanding of the shapes and dimensions of individual component elements in order to understand how each one interacts with other assembly components. This process enables the student to see which of a component’s functional dimensions are intrinsic to its correct performance within the system.

The design of the user experience (UX) [47] and user interface (UI) [48] is similar to the Assembly Manipulation Scene in terms of its organisation and the rules it follows. However, the icons are a different colour so that students can easily identify which scene they are in.

A further panel has been added, in the bottom right-hand corner of the screen, to provide students with all the details of the dimensions and shapes of every part in the assembly. This is the Definition Panel (Figure 10) and enables viewing of an isometric projection of the component, including dimensions, that can be used later to elaborate the necessary drawings for a standardised representation of the component. The image size can be altered using a slider bar.

Figure 11 shows a diagram of the structure and flow of the app, based on the scenes created in the UNITY3D Editor.

The composition of the scenes in the UNITY3D editor requires the use of assets generated in external applications. These assets include 3D meshes, textures, and UI icons. Appendix B describes the process followed for their creation.

To summarise the implementation of the mobile app ARPAID, Figure 12 shows a flow diagram of the process with particular attention to software requirements and file format compatibility considerations.

2.5. Mechanical Assemblies Modelled for Use in ARPAID

For the first stage of the investigation two complete assemblies were modelled. These assemblies form part of the practical work undertaken for this course:

• Magnetic Bit-holder assembly (Figure 7 and Figure 13a)
• Double Articulated Hoist Ring assembly (Figure 4, Figure 5 and Figure 13b)

At present, work is going ahead to model a further set of mechanical assemblies to build up a library of learning resources for our students.

2.6. Study Sample and Participants

The investigation took place throughout the academic year 2019–2020 in the Graphic Design II module common to the first-year degree courses in Electronic Engineering, Electrical Engineering, and Aerospace Engineering at the University of León, Spain (ULE). Our sample comprised 153 students in total.

Due to the way that teaching is organised in the School of Engineering at ULE, the investigation has had to use already existing groups of students, that is, natural groups. Since we were not able to randomly assign individuals to different groups, this could mean that, initially, there are noticeable differences between the groups; in other words, they will not be equivalent. As a consequence the investigation is quasi-experimental [49], and its design involves Pre-Test and Post-Test assessment of two student groups: an experimental group which will use ARPAID, and a control group which will not.

With the aim of evening out potential differences between the control and experimental groups it was decided that the experimental group should contain students from two-degree courses. Thus, the experimental group comprised students in the Aerospace and Electrical Engineering degree programs while the control group comprised students in the Electronic Engineering degree program.

Both groups had the same teacher for this subject (the first author of this article), used the same materials, notes, and book of practical exercises; furthermore, the tests and examination exercises undertaken by all students were of the same level of difficulty. The only difference between the two groups was the experimental group’s use of ARPAID for two weeks. To evaluate the effects of the app on student learning, both groups took a quiz at the beginning of the investigation (Pre-Test) and one at the end (Post-Test). The groups also completed four assessed exercises, one of which was a formal examination and counted towards their final course mark.

The sample is composed of 153 students: 67 studying the Aerospace Engineering degree (43.79%); 16 studying the Electrical Engineering degree (10.46%); and 70 studying the Electronic Engineering degree (45.75%). Thus, the experimental group comprises 83 students (54.25%) while the control comprises 70 students (45.75%). Of the 153 students, 30 are female (19.61%) and the remaining 123 are male (80.39%). The age range of the sample is from 19 to 45 years of age.

2.7. Resources Used

The application we developed can be used on mobile devices such as the latest generation of smartphones and tablets. The minimum requirements for the AR used are IOS version 13, and Android 7.0.

The following instruments were used to determine whether ARPAID had a positive effect on learning Technical Drawing of Mechanical Assemblies:

• Pre-Test Knowledge Quiz
• Post-Test Knowledge Quiz
• Pre-Test Practical Exercise: Separator Assembly
• Post-Test Practical Exercise 1: Hoist ring Assembly
• Post-Test Practical Exercise 2: Bit-holder Assembly
• Post-Test Examination Exercise

2.7.1. Pre-Test Knowledge Quiz

In the absence of the possibility of assigning students randomly to either the experimental or control group, the Pre-Test Quiz was used to determine the level of equivalence between groups prior to the investigation.

The questions on the quiz were of a general nature, relevant to the subject of mechanical assemblies, and was completed in the initial, explanatory classroom session. The quiz was aimed at measuring students’ level of subject knowledge in the initial stages of instruction. Through the test, we hoped to determine whether or not there were significant differences between the levels of knowledge in each group and in the case that large differences were discovered between control and experimental groups, these differences could be quantified.

The Pre-Test Quiz contained 20 multiple-choice questions in which the student had to select the correct answer, or answers, from four to six possibilities provided. It was put together by the teaching team carrying out the investigation (the quiz can be found in Supplementary Materials).

2.7.2. Post-Test Knowledge Quiz

The teaching taking place within this investigation was not the only source of knowledge and learning in this subject; rather, it acted to complement other approaches. As a result, it would be expected that, as the course progresses, we might see a certain amount of levelling out in student achievement between the groups. If the ARPAID experiment has a significant influence on learning, however, we should see the maintenance of initial differences—albeit reduced.

As a result, after working with ARPAID, students were given a Post-Test Quiz to evaluate their progress. It was decided that the Post-Test should follow the same format as the Pre-Test, except that the order of the questions and possible answers was rearranged randomly.

2.7.3. Pre-Test Practical Exercise

The Pre-Test Quiz completed by students attempts to highlight, in a general way, any differences in knowledge level between the two groups in the investigation. The Pre-Test Practical Exercise is a more specific measure of these differences since it comprises a task in which the student must understand the blueprints for a particular mechanical assembly, in this case, the Separator assembly shown in Figure 14a. This practical task takes place after the explanation of concepts and tackling of problems in the classroom. It is scheduled after explanatory sessions, directly before the initiation of the ARPAID experiment. It is hypothesised that the biggest equivalences between experimental and control groups should be seen at this point in the project and in the case where ARPAID has a significant effect on learning the levels of the two groups should then diverge.

2.7.4. Post-Test Practical Exercise

Besides the assembly used for the Pre-Test Practical Exercises, students must complete drawings of two further mechanical assemblies. These three obligatory tasks are designed to be of increasing levels of difficulty, each one having additional elements, containing new standard mechanical components and concepts related to tolerances and fits, and surface roughness. The Separator assembly used for the Pre-Test (Figure 14a), and the two Post-Tests exercises, the Bit-holder (Figure 13a) and the Hoist ring assembly (Figure 13b), represent this pedagogic methodology of gradually increasing complexity.

The Pre-Test and the first Post-Test assemblies must be drawn in pencil, with traditional tools, while the second Post-Test assembly is completed using AutoCAD software.

Bearing in mind that before attempting the Post-Test drawing exercises the experimental group will have had access to ARPAID and, as a result, will have been able to work with the 3D models; if it is true that ARPAID genuinely has a positive impact and promotes meaningful learning, the grades achieved by the experimental group should improve relative to those achieved by the control group.

It must be said that the mechanical assemblies that students are asked to draw are the same as the models used by the experimental group when working in ARPAID.

2.7.5. Post-Test Examination Exercise

In the assessment regime of the Graphic Design course, the section devoted to mechanical assemblies accounts for 35% of the final grade and final assessment involves a formal examination where students must draw blueprints for the assembly shown in Figure 14b.

Furthermore, the examination exercise is totally new, i.e., not related to any of the assemblies students will have worked on in ARPAID. Thus, it is a fairly objective measure of whether ARPAID’s impact on meaningful learning can be generalised and not limited solely to assemblies used in the experiment (Bit-holder and Hoist ring) but to other mechanisms. Positive results here would mean that ARPAID reinforces meaningful learning and the acquisition of skills related to the theory of graphical representation of mechanical assemblies.

2.8. Timeline for the Investigation

The authors of the present investigation have planned and devised a timeline for the completion of Stage I which includes the following steps (Figure 15):

• Step 1: Literature review to delimit the terms of the investigation and discover the state of the art in terms of theory and technology.
• Step 2: Designing the experiment, defining the hypothesis, objectives and methodologies that will be used.
• Step 3: Initial ARPAID design, establishing its structure, methodology and components.
• Step 4: Preparation of research assessment instruments.
• Step 5: Development and implementation of an ARPAID prototype.
• Step 6: Analysis of the ARPAID prototype, unit tests, iteration.
• Step 7: Working with the teacher in the technical drawing class.
• Step 8: ARPAID implementation on the two different operating systems to be used (IOS and Android).
• Step 9: Assignment of students to either the experimental or control group.
• Step 10: Completion of the Pre-Test Quiz by all students.
• Step 11: Completion of the Pre-Test Practical Exercise by all students.
• Step 12: Students in the experimental group download ARPAID to their mobile devices. Students are given two weeks to use the application and work with the material it contains.
• Step 13: Completion of the Post-Test Quiz by all students.
• Step 14: Completion of Post-Test practical drawing activities 1 and 2 by all students.
• Step 15: Completion of Post-Test Examination Exercise.
• Step 16: Analysis and statistical processing of the data.
• Step 17: Examination of the investigation’s results and drawing conclusions.
• Step 18: Proposals made for improvements and planning for Stage II of the investigation.

3. Results and Discussion

3.1. Evaluation of Learning Outcomes

After the tabulation, processing and analysis of the data obtained during the investigation a detailed exploration of the results will be done.

3.1.1. Pre-Test Knowledge Quiz

Descriptive statistical measures applied to the results of the Pre-Test Quiz confirmed the initial hypothesis that there would be a lack of equivalence between the experimental and control groups. This is largely due to the fact that the mean grade of students at intake to the Aerospace Engineering degree is significantly higher than that of student groups starting other degree courses. Even when normalising this group to the group with the lower mean intake grade, the lack of equivalence is maintained.

The mean grade obtained in the Pre-Test Quiz was 8.73 (out of 20) for the control group while the experimental group scored 3.3 points higher gaining mean grade of 12.03.

Analysis shows that the standard deviation of test scores for the control group was smaller than for the experimental group—2.95 and 3.75, respectively. The greater variation seen in the experimental group can be explained by its heterogenous nature, containing students with extremely good results and others with much lower academic achievement (see Figure 16).

3.1.2. Pre-Test Practical Exercise

The next hypothesis to be tested was that the presential classroom sessions, both theory lessons and those involving practical tasks, would have a levelling out effect on the two student groups. Indeed, despite initial differences, in these circumstances learning can be optimal and enable students to reach the same high standards, independent of their initial level of learning.

Analysis of data obtained from students’ performances in the Pre-Test Practical Exercise, undertaken by the whole student sample, verified the levelling effect of classroom learning (see Figure 17).

In this exercise, students in the control group obtained a mean score of 5.09 (out of 10) which was similar to that obtained by the experimental group, 5.76. Furthermore, learning appeared to be uniform between and within groups as shown by the fact that the standard deviation of scores was almost the same for both (1.42 for the control and 1.47 for the experimental group).

3.1.3. Post-Test Quiz

It is necessary to reiterate that the Pre-Test Quiz was used to determine whether there were significant differences between the experimental and control groups prior to the experiment taking place. This and the Post-Test Quiz were not designed specifically to evaluate the impact of ARPAID on the students’ acquisition of technical drawing skills since the questions in the Quiz are generic rather than focussed on evaluating the skills that ARPAID was designed to develop and strengthen.

The Post-Test Quiz reflects the situation immediately after the experiment (see Figure 18).

Although this instrument cannot be used to evaluate the impact of ARPAID, it can suggest trends. If the experimental group shows a greater improvement compared to the control, this will be, fundamentally, due to the effect of ARPAID, since this is the only factor that varies between the groups.

Comparing the results of the quizzes Pre- and Post-Test, it is clear that there is a significant increase in achievement in both groups. This indicates that the personal learning work completed by students added to the classroom-based sessions complemented by tutorials has been instrumental in enabling students to construct knowledge and deepen their understanding of the subject material. The improvement in mean score was 3.18 points (out of 20) for the control group while that seen in the experimental group was 1.48. This is not a surprising result since, as we mentioned, classroom teaching and self-study are factors that level out learning outcomes. However, differences between the variation of scores within groups could be indicative of ARPAID’s influence, acting as an additional factor promoting uniformity of learning. The standard deviation of scores in the experimental group was more or less constant pre and Post-Test: 3.75 and 3.74, respectively, indicating that learning was fairly uniform within this group. However, within the control group, the standard deviation of scores increased from 2.96 Pre-Test to 4.01 Post-Test. This implies that some students had improved their results considerably while others achieved a much smaller increase in their scores.

With respect to the Pre-Test Practical Exercise, after statistical analysis and adjusting values to the same scale (20), there was an improvement in achievement for both groups: 1.73 points in the control group and 1.99 points in the experimental group. Although it is possible that this improvement could be put down to hard work on the part of students in both groups through self-study and practice, the greater achievement of the experimental group suggests that using ARPAID had a positive influence in this improvement.

Figure 19 shows the evolution over time of mean scores measured at key points in the investigation: the initial state, prior to any intervention (Pre-Test Q); the state post classroom teaching intervention (Pre-Test PrEx); and finally, the state post ARPAID intervention experiment (Post-Test Q). As can be seen, before any teaching took place, the difference in level between the two groups was 3.3 points. After the classroom teaching sessions, when students had to complete the Pre-Test Practical Exercise, this difference had reduced to 1.34 points (59.39%). This was the state of affairs before the ARPAID experiment took place, after which the students took the Post-Test Quiz. Here, the difference between groups increased again to 1.67 points, suggesting that ARPAID did indeed contribute to learning.

3.1.4. Post-Test Practical Exercises and the Examination Exercise

These three exercises constitute the basic instruments used to verify whether use of ARPAID has a positive impact on the experimental group and if it has contributed to meaningful, deep learning about the standardised drawing of mechanical assemblies. The exercises constitute a direct application of the skills acquired during the learning process. Student grades in these exercises are therefore an objective measure of their degree of learning and the level of skill attained by the student.

In order to analyse the data, three Supervised Learning Classifiers have been applied [50] to the scores obtained by students in these three exercises: Linear Discriminant Analysis (LDA); Quadratic Discriminant Analysis (QDA) and a Support Vector Classifier (SVC). The aim was that the classifiers should, based on a student’s score, be able to correctly place that student into a class: Class 1 (student used ARPAID) or Class 0 (student did not use ARPAID). After initial training of the classifier with the available data, class assignation was completed using the scores obtained by students in the three assessed exercises.

Each of the analyses has been represented graphically, in a Confusion Matrix Heatmap, from which we extracted indicators (or metrics) that enabled an evaluation of classification quality: the precision and the sensitivity (also known as the true positive rate or recall). Figure 20 shows the Confusion Matrix for QDA (a) and the SVC (b). Results obtained using LDA classifier are not shown as these were inferior to the classifications obtained using the other methods.

Looking at the results for QDA, we see that this classification process asserts a 90% true positive rate, that is to say, when a trio of results from the three exercises is predicted to have come from a student who used ARPAID, the probability that this is a correct prediction is 90%. The probability of false negatives is 10%. The probability of identifying true negatives is slightly low at 68.75% and that of false positives is 31.25%.

The best results were obtained using the SVC. Here we see that the classifier has a true positive rate of 91.4%, while at the same time, the probability of a false negative is 8.6%. The probability of correctly identifying true negatives is down to 69% and that of false positives is 31%.

The classification quality metrics, precision and sensitivity were very promising for both types of analysis shown above. The precision—that is, the ratio of true positives to the sum of these and the false positives, is 0.86 for both QDA and SVC. The sensitivity, the number of true positives with respect to the total number of positives found overall, is slightly better: 0.9 for QDA and 0.91 for SVC. Figure 21 shows the values of all the most commonly used classification quality metrics.

It can be seen therefore that our data sample contains a structure that justifies its classification in the manner suggested; this is shown in general and particularly by the results of SVC, which demonstrates this most clearly. That is to say, the results obtained support the existence of two sub-populations or classes in the sample: one composed of those students who used ARPAID and another composed of students who did not. These two clearly delineated classes can be separated according to the scores obtained in the three assessed exercises. Thus, the use of ARPAID does appear to have an effect on the grades achieved by students, meaning we have reached the objectives of the investigation.

Figure 22 shows histograms of the two classes. It can be seen that, despite some overlap, the classes are well separated and that with the best student achievement and smallest variation is class 1, that is, the class representing individuals who used ARPAID.

Regarding the percentage weight of each one of the assessed exercises as a discriminator for classification, it would seem that the exercise that most distinguishes students who used ARPAID is the Examination Exercise which had a weighting of 60%. The second most influential was Post-Test Practical Exercise 2 (the Hoist ring) with a weighting of 38%, and lastly, Post-Test Practical Exercise 1 (the Bit-holder) with a weighting of 2%. According to these data, the Bit-holder exercise is irrelevant to the classification process indicating that students who did use ARPAID and those who didn’t performed equally well in that assessment. The explanation for this result could be related to the fact that this assembly is the only one that students must draw using the computer assisted design package, AutoCAD.

Many of the students in the control group did not complete all the exercises set, which seems to indicate some difficulties and a lack of confidence when attempting this course work. In contrast, this issue was not seen amongst students in the experimental group.

Once it has been confirmed that use of ARPAID does indeed have an impact on learning skills for technical engineering drawing of mechanical assemblies, it is necessary to establish the nature of this impact: is it negative or positive? The answer is of course obvious looking at the mean scores obtained by students in the three assessed exercises used in the investigation. As shown in Figure 23, these mean scores are always highest in the experimental group.

The fact that the Examination Exercise had a 60% weight in the classification suggests that use of ARPAID had a positive impact, not only on understanding of the mechanisms modelled but also on students’ ability to generalise learning to any kind of assembly. That is, the results suggest that use of ARPAID has a positive impact on meaningful learning in this subject area and on the acquisition of skills for the graphical representation of mechanical assemblies. At a later stage, following this initial experiment and further ARPAID development, it will be necessary to study this issue.

4. Conclusions

This article describes the evaluation of a mobile application called ARPAID. This application can be used to teach the graphical representation of mechanical assemblies in engineering. It is a new tool and an interesting complement to other learning processes. It is an accessible and portable tool to help students understand material in the assembly drawing module of the Graphic Design course at university level.

Results show that ARPAID causes a significant improvement in the achievement of students in the experimental group using the application.

The increase in achievement recorded for the experimental group was not only observed for exercises involving assemblies modelled in ARPAID but rather, it was seen for a range of assemblies given to students. This fact allows us to suppose that this improvement in performance can be extrapolated or generalised to any exercise.

ARPAID has been given a favourable welcome by the students. They have reported that it has been extremely useful in studying course material and has enabled them to understand essential concepts. Furthermore, the fact they have been provided with a novel tool that is intuitive to use and takes a very non-traditional approach, has contributed to an increased motivation amongst students for this part of the course. It must be emphasised that, during the experimental phase of the investigation, the average number of logins per student to the ARPAID app was 7.38, with a maximum of 28 uses by a single student. This shows how well accepted ARPAID has been amongst the student body, and in some cases, students have continued to use it several months after the end of the academic year.

The results obtained allow us to conclude that use of ARPAID is a method that complements traditional methods and that it motivates and helps students in their learning process.

The promising initial results of the project have provided all those involved in this work with an incentive to go ahead with the further stages of its development with renewed energy and enthusiasm.

5. Study Limitations

An important issue in this investigation has been the impossibility of testing out ARPAID’s synchronous capabilities during class-based sessions because of the quarantine imposed due to Covid-19. Indeed, the explanatory sessions and classroom tasks planned for the second fortnight in March and April 2020 had to be cancelled and replaced with online videoconferencing sessions.

A further difficulty that emerged during implementation of ARPAID was its deployment on devices running the IOS operating system. Due to a conflict concerning privacy and security needs for data collected during use of ARPAID, the directors of the Apple Development Program prevented its upload to the official App Store at the point when the investigation was about to take place. Fortunately, it was possible to test a fully functional beta version of ARPAID using the TestFlight app.

The technological requirements for the extensive use of AR on mobile devices could be seen as a disadvantage, given that not all students will have access to top of the line mobile phones. As a result, it is necessary to find a compromise between the available characteristics of ARPAID and mobile device capabilities such that technology does not become a discriminatory barrier to the acquisition of learning.

6. Future Lines of Investigation

As described in the previous sections of this article, our investigation was designed to be developed in three stages. To date only the first of these stages has been completed. The lines of research that we intend to pursue from this point are the following:

• Stage II: Evaluation of ARPAID’s usability, its user interface and general functioning. The incorporation of new features, particularly in the field of AR. Inclusion of mechanisms for stealth assessment.
• Stage III: Externalisation of ARPAID using asset bundles and the creation of a remote database of resources.

A promising line of research, and one which is already in the process of being implemented, involves automating, as far as possible, the uploading of new mechanical assemblies to ARPAID, regardless of the complexity of said assemblies (both in terms of the number of components and in the hierarchical structure of sub-assemblies). In this way, we hope to reduce the level of human intervention to a minimum.

Another possible avenue of work is studying the social and motivational aspects of ARPAID usage. We hope to discover which elements promote a good learning environment and to evaluate the effect of features related to gamification along the line of serious gaming.