Geospatial Virtual Reality for Cyberlearning in the Field of Topographic Surveying: Moving Towards a Cost-Effective Mobile Solution

Abstract

In spite of the tremendous success in artificial intelligence technology and a high level of automation in geospatial data obtaining processes, there is still a need for topographical field data collection by professional surveyors. Understanding terrain topology and topography is a cognitive skill set that has to be demonstrated by geospatial Subject Matter Experts (SME) for the productive work in the topographic surveying field. For training of the mentioned above skillset, one has to be exposed to the theory and must also practice with surveying instruments in field conditions. The challenge of any surveying/geospatial engineering workforce training is to expose students to field conditions which might be limited due to equipment expenses and meteorological conditions that prevent good data collection. To meet this challenge, the Integrated Geospatial Technology research group is working on a geospatial virtual reality (VR) project which encompasses the following components: (a) immersive visualization of terrain; (b) virtual total station instrument; (c) virtual surveyor with reflector installed on the virtual rod. The application scenario of the technology we are working with has the following stages: (1) student is installing total station on the optimal location; (2) students move virtual surveyor on the sampling points they consider to be important (3) contours are generated and displayed in 3D being superimposed on 3D terrain; (4) accuracy of terrain modeling is observable and measurable by comparing the sampling model with initial one.

1. Introduction

Nowadays, geospatial technologies are part of the new paradigm of cyber-infrastructure [1] that demands a geospatial workforce training methodology to accommodate cyberlearning technologies [2]. Topographic surveying is one of the most demanded operational workflows needed for both mapping and engineering geospatial application scenarios. Understanding of terrain topology is a critical skill set that has to be developed by any surveyor who is involved in topographic data collection. Training of those skills for surveyors requires the use of various instruments (total stations/GPS systems) in the field. Such a filled training session is sensitive to the seasons of the year and weather conditions which stimulate demand to move surveying education to the virtual space. Virtual Reality and Augmented Reality (VR/AR) technologies open an opportunity of exploring online training opportunities for geospatial workforce training and education. There are some examples of the possible applications of such a technology in the field of education. Those publications describe both common principles of VR cyberlearning [3] and specific aspects of the surveying VR implementation [4,5]. Specifically, research paper [6] reports on virtual instruments developed for teaching surveyors, which can be enriched by small Unmanned Aerial Systems (sUAS) data and Terrestrial LIDAR Scanner (TLS) point-clouds, as described in [7]. Papers [8], [9], and [10] are devoted to the development of cognitive theories behind virtual reality training and education. Paper [11] discusses various applications of Unity technology for VR simulation generation. In paper [12], authors proposed the original solution of the very important research question on the optimal number and position of the ground control points for the sUAS case. Finally, paper [13] describes immersive virtual reality technology for specifically surveying engineering education purposes. Nevertheless, all of the research listed above was implemented based on standalone computer systems. Nowadays mobile technologies and cloud computing enable another solution to the geospatial VR implementations which are more computationally efficient and can be deployed on networked inexpensive systems. The major difference of the current research is the attempt to deploy inexpensive mobile VR compared to standalone systems described in the papers mentioned above. Of course, those studies are theoretically based on another in-depth research study in the sphere of Augmented Reality (AR)VR, such as [14,15,16]. Where [14] performs an excellent VR/Mixed Reality (MR)/AR technology review, [15] deals with biomedical and attentional aspects of such technologies, and [16] is devoted to the rigorous cognitive analysis of the experiment from the human factors standpoint.

The development of the virtual training tool sets is now prevalent in mainstream research efforts. For example, the manuscript [17] describes the cognitive and technological aspects of surveyors training “gamification” [18,19] that culminated in the development of operation workflows of the 3D multiuser virtual learning environment (3DMUVLE) educational game design framework. This research study demonstrates a very useful connection of the gaming application scenarios with learning outcomes and their assessments in the frame of cyberlearning; however, implementation of the variety of the specific surveying technologies and instruments may serve as a complication factor for the creation of generic application scenarios. The most interesting work was performed in [20] where online training was focused on digital terrain modeling practice. The challenge of current research is to develop a cyberlearning approach for terrain data collection specifically for topographic surveying.

2. Research Methods

To meet this challenge, we developed a cyberlearning framework that was devoted to training students in understanding the proper process to select points for the topographic surveying in a virtual simulation. The cyberlearning environment was based on a VR model encompassing a Digital Elevation Model (DEM), a virtual total station, and a virtual reflector. Unity [11] was used to create the VR environment to build the topographic surveying VR simulation. The operational workflow diagram of the Surveying VR cyberlearning toolset is depicted in Figure 1. Cyberlearning workflow consisted of (a) setting a total station position, (b) repetitive movement of the virtual reflector to the positions where terrain sampling points were selected by the student for the measurements, (c) generation of contour lines based on sampling points selected by students, (d) visual and quantitative comparison of the DEM and contours generated based on student measurements with proper ones generated based on source DEM and instructor-sample measurements. Technical implementation of the operational workflow was achieved by means of integration of the inexpensive mobile computing and state-of-the-art stand-alone inexpensive and open-source terrain modeling toolsets. Elements of gamification were introduced by a score defined by a minimal number of sampling points versus the higher accuracy of the DEM generated as a result of the cyberlearning session. That approach was implemented and discussed in detail in the current section with preliminary results outlined in the Section 3 of current manuscript.

Learning outcomes of the technology outlined here were (a) understanding of the terrain modeling and sampling principles, (b) comprehension of terrain interpolation methods, (c) understanding of the regular and structural terrain modeling, (d) profound understanding of the field data collection during topographic surveying. Per the technology developed, these outcomes were achieved via the following processes: (1) generation of the terrain of the pre-defined level of complexity; (2) preparation of the operational VR model comprising that terrain, total station, and rover; (3) student exposure to the virtual terrain topographic surveying with measurements of timing; (4) exporting of the mobile VR data for processing and accuracy obtaining; (5) assessment of the learning outcomes numerically by means of accuracy, timing, and score analysis; (6) feedback of the assessment results to the student and decision on the next iteration with the terrain of the same complexity or moving to a more complex level.

The design of the virtual total station and reflector/rover was carried out in Blender [21] and are depicted in Figure 2.

The virtual terrain in Unity was transformed into Android Mobile Application [22] and can be executed on any smartphone or tablet with an Android operating system of version 4.4 or higher. Technical implementation of the Surveying VR was performed by means of a low-cost Samsung VR system [23] and a control device [24] and is depicted below in Figure 3.

Screenshots of the operation Surveying VR cyberlearning systems are given in Figure 4 below.

Students involved with the applied cyberlearning process were exposed to the fact that obtaining surveying topo-points on proper elements of the terrain such as the lowest and highest elevation points and structural lines (lines where terrain slopes and aspect is changing), known as breaklines, would lead to the same DEM as a regular grid sampling. Since the computational power of the smartphones we deployed for virtual reality was not sufficient, we exported VR obtained measurements results as a csv (comma separated value file) which was processed by external applications of sufficient computational power. To visualize DEM, contour lines were generated for each experiment by external software application, illustrated in Figure 5.

Numerical estimation of the accuracy for each cyberlearning sample can be computed as root mean square errors of the points interpolated based on the current students’ attempt as

$$RMS=\surd \frac{{(h_i-h_{DEM})}^2}{n-1}$$

(1)

where h_i = interpolated elevation based on current attempt sampling points; h_DEM = interpolated elevation based on DEM; n =number of points included in the interpolation area.

In terms of the gamification score, each student attempt was computed as a weighted inverse sum of accuracy and number of sampling points involved for obtaining those accuracy attempts, as expressed below:

$$SCORE=N_{1 }*\frac{1}{RMS}+N_2*\frac{1}{NP}$$

(2)

where RMS is the error of the attempt (accuracy of elevations); NP is the number of sampling points collected in the current attempt; N₁, N₂ = weights of accuracy and optimal sampling.

We initially used N₁ = 200 and N₂ = 100 to obtain integer numbers for the score. Those numbers can be also deployed as a means of assessment of the learning outcome and grading instrument.

3. Cyberlearning Implementation and Results

To make our cyberlearning experience available to most devices, we implemented it on the Google Android VR platform [23]. Figure 6 shows the Surveying VR App working on the test smartphone.

As was mentioned before, the processing power of smartphones was not sufficient for the digital elevation modeling we deployed an existing inexpensive external terrain modeling toolset—Golden Software Surfer [25]. Sampling points for each gaming attempt were transferred as a text file for the Surfer which enabled the production of DEMs and visualizations of contours for each cyberlearning attempt. Golden Surfer also enables experiments with different elevation interpolation methods such as IDW, Kernel, and Kriging which enabled us to establish another learning outcome of the terrain interpolation method understanding. This can be assessed by means of visual analysis of the results in Surfer. We also encouraged our students to work with the open-source CloudCompare toolset [26] for the numeric and graphical comparison of source DEM and those which were generated based on cyberlearning samples. The use of CloudCompare was optional and helped significantly in both numerical and visual analysis for terrain model comparison.

As the first step, the test area had been surveyed by terrestrial laser scanning, and a high-precision reference DEM was created. The reference DEM in contours and 3D forms is presented in Figure 7.

In what follows, we supposed that the test area was surveyed separately by a professional surveyor and by a student with a lack of necessary skills. Figure 8 presents the contours of the reference test area. These contours were based on the twenty points that were collected by a professional surveyor and by a student in a different manner depending on the level of the skillset. The contour interpolation has been carried out using the Kriging method. For the sake of better presentation, the surveying points (red circles) have been overlaid. In Figure 8, the left image is the professional surveyor’s contours, and the right one is the student’s contours.

It is easy to see that the student has tried to choose the surveying points almost uniformly, whereas the professional surveyor has picked up the points in such a way that they are describing the test area surface as close to reality as possible. Consequently, we obtained two different surfaces that were based on the same volume of the data sample. Figure 9 depicts those surfaces in the forms of overlaid contours and the comparison between the surfaces.

The better way to analyze these data is a calculation of simple statistical characteristics. The results of the calculation appeared instantaneously after the virtual fieldwork had been done; therefore, a student may check the quality of their work and verify their accuracy for self-assessment. Below, in

Table 1. Statistical characteristics.

Parameter	Value, m
Minimum	−0.87
Maximum	1.69
Mean	0.12
Root Mean Square (RMS)	0.35
Conclusion	$\mathsf{\Delta }=tRMS\le h/3$, where t—probability coefficient (2.0), h—contours step (1.0 m) $\mathsf{\Delta }=0.70\ge 0.33$, you failed

, is an example of the assessment.

To check whether the interpolation method affects the final conclusion, the results of surveying for twenty points have been compared with the same points for interpolated surfaces. The comparison results are presented in

Table 2. Comparative analysis of experienced and inexperienced topographic data collection.

Surveyor		Student
Mean, m	0.0	Mean, m	0.0
Root Mean Square, m	0.02	Root Mean Square, m	0.02
Minimum, m	−0.04	Minimum, m	−0.03
Maximum, m	0.07	Maximum, m	0.07

.

The results in Table 2 look quite similar. One may infer that Kriging interpolation has a minimum effect on the final decision, and consequently might be neglected.

To validate properly, the results must not only be compared with each other but also with reference surfaces. Below, the comparison of the professional surveyor’s data and the student’s data are presented.

In Figure 10, the left panel is the professional surveyor’s contours overlaid by the test area contours and the right is the contours’ differences.

In Figure 11, the left panel is the student’s contours overlaid by the test area contours and the right one is the contours’ differences. From the last two figures, we may calculate the accuracy of surveying according to the reference data and determine whether the surveyor or student has passed the accuracy criterion (see Table 1). The checking procedure results are summarized in

Table 3. Statistical characteristics.

Surveyor	Student
Mean, m = −0.03	Mean, m = 0.08
Root Mean Square (RMS), m = 0.15	Root Mean Square (RMS), m = 0.38
$\mathsf{\Delta }=tRMS\le h/3$, $\mathsf{\Delta }=0.30\ge 0.33$, passed	$\mathsf{\Delta }=tRMS\le h/3$, $\mathsf{\Delta }=0.76\ge 0.33$, failed

.

From the results, we may conclude that the professional surveyor has passed the test as expected.

Summarized in

Table 4. Surveying VR gamification experimental results.

Surveying VR Experiment Complexity	Attempt (1)	Attempt (2)	Attempt (3)	Scores
Simple Terrain Experiment 173-286
RMS (meters)	1.2	1.0	0.7	173
Number of points	15	10	9	200
Time (seconds)	240	140	126	286
Average Terrain Complexity Experiment 65-226
RMS (meters)	3.2	1.8	0.9	65
Number of points	37	30	23	114
Time (seconds)	777	570	414	226
High Terrain Complexity Experiment 38-121
RMS (meters)	5.6	3.8	1.7	38
Number of points	39	31	29	55
Time (minutes)	897	651	551	121

are the resulting recorded times based on formulas (1) and (2) for one subject on three different terrain complexity levels. The gamification score can be considered as a summative assessment of the learning outcomes from the cyberlearning lesson described.

Improving accuracy and terrain understanding by reducing the number of sample points and the time to obtain resulting contours indicates the feasibility of our novel Surveying VR cyberlearning methods and its applicability for the training of the topographic surveying skill sets.

4. Discussion

It can be seen from Table 1 that the time to collect one point is increasing with the increase in terrain complexity. That is especially significant compared to the first attempt. However, we may assume that a skill set obtained from the initial work with less complex terrain is transferable for the next level of terrain complexity. To confirm this, we will compare data for subjects who are starting from complex terrain at the beginning with those who initially were trained on lower complexity terrains. Our gamification scoring system compatibility consistency with numerically defined learning outcomes is confirmed by initial experiments. This enables us to draw the observation from the pedagogical side that students should first be exposed to the lower complexity terrain exercises in order to better develop skills defined by the expected learning outcomes. This will reduce the total time needed to gain such an outcome and corresponds to the methodology we use in training the surveying engineers and technicians using traditional practices. These observations can be considered as experimental confirmation of the cognitive science principles of VR applications as they are established in previous cyberlearning research, such as [3], which is supported by cognitive theories [8,9,10]. Moreover, by adding representation to the current list of subjects, our research may garner some interest in cognitive science such as a geospatial case study. We are planning to establish such interdisciplinary collaboration in the future.

COVID-19 restrictions have prohibited us from extensive study with human subjects’ due to social distancing. Our future research will be focused on integrating Surveying VR cloud-based computing capabilities such as Geocloud [27] or M-App [28]. We will identify proper client–server architecture cloud-based technology that will enable us to implement Surveying VR cyberlearning technology deploying a simple smartphone VR apparatus connected to the internet and to avoid the necessity of multi-platforming.

During recent years Virtual Reality has shown good potential to expand learning domains in classroom environments. By using Virtual Reality technology in the field of cyberlearning we can shift the focus from plain learning to an interactive learning environment; however, we have to consider some problems. Despite all of the efforts in the field of Virtual Reality, we still have challenges using those technologies and devices inside learning environments. One of the biggest is providing powerful computers for students inside a classroom for having a good Virtual Reality experience as well as high-quality HMDs. However, providing these requirements might be difficult for students and teachers due to their costs; hence, we tried to implement this application for smartphones. Nowadays, we have a wide variety of smartphones and they are much more affordable than high-end Virtual Reality apparatuses. Even with all of the limitations of smartphones, such as computing power, we developed our virtual reality application and tried to provide a good VR experience for our study participants.

As of right now, the environment only includes the terrain, but in reality, there are other ground features that can be objects of interest or even obstacles during a survey, especially in obtaining a line of sight. Decisions have to be made by surveyors on the locations of the stations and the ground points to be collected. In this case, future work will include more ground features to simulate the decision-making process during a survey. Other instruments and devices, such as GNSS, are also used in surveying fieldwork, and thus, more types of surveying devices may be included for different types of procedures.

In spite of multiple implementation problems, caused mostly by mobile VR computational limitations, we demonstrated the feasibility of the approach for training topographical surveying skills. Given the limitation or complete elimination of the face-to-face classes in academic organizations due to COVID-19, deploying such technologies for training in surveying/geomatics engineering programs can be considered as a reasonable approach.

5. Conclusions

Surveying VR is a feasible method for training the geospatial workforce that enables the achievement of the same learning outcomes specifically in terrain understanding for topographic surveying as a traditional field-based practice. Certainly, there is still a gap between field surveying experience and cyberlearning. The major reason for that gap is missing experience in the fieldwork with surveying instruments including centering the tripod over the setting point, work with tribrachs, instrument leveling, reflectors, crew communications, etc. Even with the increase in the VR application complexity complete hands-on experience is crucial to be ready for performing the surveying. With the increase in computational power towards better processors and cloud-based computing, we anticipate our future efforts will encompass the development of the more realistic cyberlearning simulations to cover the gaps discussed here.

Given the boost of online education demand due to the COVID-19 pandemic, methodology and experimental strategy established in the Surveying VR cyberlearning project can be expanded to the development of skills associated with other geospatial sensors such as LIDAR scanners, UAS, Earth observation satellites, and others.