Feasibility of a Drone-Based Road Network Inspection System ^†

Abstract

Data collection using drones is a widely used practice today. This capability can be used to develop a drone-based road surface inspection system aimed at increasing user (driver) safety and providing information to the owners and maintainers to assess the status of their infrastructure assets. This paper presents an overview of the Turkish–Hungarian U-SOAR project, aimed at the prototype level development of such system. While the system is technically feasible, two key aspects were identified that requires focused further evaluation and development: the business model definition and the management of conflicts in operation. The paper highlights the key findings in these two areas. First, the methodology used to assess the possible business approaches is demonstrated, showing that in the context of these two countries, service provision providing processed data has the highest potential. Second, the requirements towards conflict management as a key part of safe operations are presented. This paper shows the proposed automated conflict management solution and the initial development and testing of the system.

1. Introduction

The use of drones for data collection is considered an established practice today. Collecting data can be performed by various sensors, including standard and hyper/multi-spectral cameras, LIDARS, etc. One of the most common applications is the examination, observation, and management of infrastructure and facilities, for example, in the construction industry, where sites are mapped or checks are carried out during maintenance [1,2]. Looking at the field of transportation, applications include the data collection of road traffic [3], cyclist and pedestrian [4], road surface [5], or railway [6] monitoring, and other goods transport and traffic management applications [7]. Based on the significant available research, it can be stated that there are no technological obstacles to building drone-based monitoring infrastructure as a service. Rather, feasibility depends on economic and operational challenges, such as cost-effectiveness and safe operation, including managing conflicts. In terms of cost, compared to today’s solutions, a drone network needs to be cheaper or provide better quality data for the same price to be competitive. Data on road conditions are being collected regularly, mostly for maintenance scheduling and renovations. Most data collected today can also be collected from a drone platform. In terms of conflict management, drone traffic raises many problems [8] and is not yet been solved. The given region’s legislation [9] also needs to be considered, which increases complexity.

2. Presentation of the Concept

The goal of the project is the verification of the feasibility of a drone-based road inspection system at a prototype level. Drone docking stations are placed at strategically selected locations. Drones fly 10–15 m high above the road surface, depending on obstacles, periodically recording the surface and objects on the road. The collected information is parsed by a GIS (Geographical Information System), which allows for easy and quick access to records. The locations of objects and road quality are recorded and can be accessed at the specified space and time. Computer vision software algorithms are used for the automatic analysis of recordings: the detection of dangerous objects and road surface conditions. Such obstacles can be stones; debris or foreign objects on the road; dangerous road surface due to weather conditions, such as ice, mud, or flood; or damage to the road surface, such as cracks, potholes, or other road defects. Drone-based data collection has many advantages over land vehicle data collection: due to its relative isolation from road traffic, it can be reliably flown regardless of traffic and road quality, the vehicle is cheaper to buy and operate, and autonomous operation is simpler and presents less risk. Risk in this context is defined as the product of event (such as a crash) probability and severity. However, there are also disadvantages: docking stations must be built for take-off and landing, better sensors are required, and they are more sensitive to weather conditions. The service has many advantages: high frequency road condition checks can be facilitated, dangerous road surface conditions can be quickly assessed, and operators’ response time can be reduced. Furthermore, due to the use of GIS, the drivers on the road, including autonomous road vehicles, can receive data on the condition of the road in advance, making driving safer.

In a system that is aimed at collecting and distributing data, the integrity and security of such data are very important. The field of cybersecurity deals with such issues and there is active worldwide research involving not only drone applications [10,11] but almost every other industry as well. Thoroughly investigating cybersecurity aspects was out of scope of the U-SOAR project, delegating it to the possible continuation of the project.

The rest of the article deals with two key aspects of the concept: the possibilities for value creation in the business model and the management of conflicts that arise during operation. While there are many other fields to be addressed to develop the full “production” level system, these two points are regarded as top priorities within the project.

3. Business Models

Today there are numerous research works available on the predicted market size of the advanced air mobility market, such as [12,13,14]. However, in addition to estimating the available market size, a business approach regarding how value can be created and sold to customers needs to be developed. To propose the business model, i.e., in what form the concept can be sold on the market, what value-creating activities it could provide, the ideas of the project participants were first collected. These are divided into three categories:

• Sale type: drones or parts/accessories are sold or rented;
• Service type: information/data collected by the drone is sold;
• Support type: related services are sold.

The individual activities were scored using an expert review within the project consortium (two academic and two industrial partners, one from each country, providing different points of view), evaluating the individual activities from 1 to 5 based on the following criteria: expected profit, market size, lack of competition, difficulty entering the market, simple technology, expert availability, and low investment. A score of 1 indicates very unfavorable, and 5 indicates a very good assessment. The evaluation was carried out by the participants for the Hungarian and Turkish markets. The individual expert scores were recorded and averaged, while deviations were also assessed. It was decided that negotiations between experts were not necessary in this phase of the project. As can be seen in Figure 1, that the availability of experts was scored as the top positive factor, followed by simple technology and expected profit. The biggest weakness was the lack (or rather abundance) of competition: service by ground vehicles on the one hand and other drone service providers on the other. Closely related to this is the low difficulty of entering the market, which was ranked as the second weakest point.

Based on these factors, as Figure 2 indicates, service-type activities performed the best and sales-type activities performed the worst. The reason for this is that, according to the expert participants, when selling drones in any form, one would have to compete with many similar market participants. Support-type activities scored in the middle range, and most participants considered them to be good additional activities next to a main service-type activity. The provision of the processed data as a service was clearly considered the best, since it can be sold and used by a wide range of users. The most popular is to evaluate road quality at a service level and send warnings about possible obstacles and dangers to drivers directly to their navigation application and to provide detailed data for the operator of the road section for maintenance and road renovation scheduling.

Unfortunately, the direct comparison of drone-based and current road-vehicle based methods was not feasible within the scope of the project. This was primarily due to the fact that the ownership and maintenance of the road networks show significant segmentation. In Turkey, three classes of road are defined as highways, according to the General Directorate of Highways, and four other classes are defined as smaller roads, and responsibility for these varies between different ministries and institutions. In Hungary, national public roads are state-owned (except for a limited number of outsourced sections), while local roads are the responsibility of municipalities. Significant further work would be required to evaluate the business structures of the many stakeholders involved.

4. Conflict Management

Operational conflicts should be defined in the broadest general sense, referring to interaction between a vehicle and different objects. These objects can be physical or even abstract, such as:

• Another vehicle: aerial or land-based;
• Prohibited or restricted airspace or road section;
• Obstacle;
• Terrain or environmental hazards.

The management of conflicts not only relates to physical conflict, e.g., impact between the objects but also situations where a change in direction or velocity is required from one participant, generally in any situation where there is an impact on safety, including the workload of the participants.

Conflicts can be managed on many levels, from infrastructure planning and traffic control methods over a long period of time to passive, direct means of mitigating conflicts during conflict. Generally, based on the time aspect, conflict management is divided into strategic and tactical. Worldwide there is active research work on the management of unmanned traffic, including the USA [15], EU [16], or Japan [17]. Within the framework of this project, the goal is the detection and management of conflict situations at 5–10 s before a potential accident or incident. Since traditional air traffic control does not have the capacity to handle conflicts of such small drones, a web-based conflict management service is developed for this purpose, which is capable of resolving conflict situations between connected users and obstacles and closed airspaces stored in the database. The requirements for the system are as follows:

• High performance: many participants handled autonomously;
• Technology independent: compatible with a wide range of manufacturers;
• All participants: drones, obstacles, and restricted airspace definitions can be integrated;
• No low-level code development: vehicles use their own control algorithms;
• Modular solution: integration of several types of current and future users/use cases;
• xITL tools: compatible with XITL (X in the loop) tools to facilitate testing;
• Enable conflict detection and management algorithms development.

The prototype of the conflict management system has already been developed and tested [18]. The system was developed as a web service, using modern, widely used database, task management, and messaging software packages. In the prototype system, conflict situations are detected by the pairwise dynamic projected conflict zones detector. The detector, based on the vehicle’s type, position, speed, and trajectory, predicts its possible locations in the next 10 s. If conflict zones intersect, the system detects a conflict situation and initiates the appropriate conflict resolution commands. In the case of the prototype, conflicts are resolved using the stop and go algorithm, where the commanded vehicle slows down or stops if it is in conflict. This approach would not resolve the conflict in the case of a full or small angle head-on encounter, nor for fixed wing drones, so more advanced resolution algorithms are in development. A new function also in development is the integration of airspace usage data and other geofence objects into the conflict management system. Reading objects from the data published by HungaroControl is already functional, but conflict management with objects has not yet been tested. The representation of the scanned data including current reserved airspace sections is shown in Figure 3.

The prototype was successfully tested on the Zalazone test track, where the resolution of a conflict between a ground vehicle and a drone was demonstrated. The devices used during the test are an X-500 drone (part of a PX4 development Kit, manufactured by Holybro, in Hong Kong, China) and a drone “head” flight-controller-in-a-box device attached to the ground vehicle. The box contains the essential sensors to provide telemetry data and can be mounted on any device. The system includes a web server, generating conflict avoidance commands which are transmitted using telemetry radios [18].

The next steps in the system development include the integration of obstacles and closed or restricted airspace definitions in the system’s database, the improved management of multi-vehicle conflicts and using more advanced conflict resolution algorithms, to not only provide safe, but efficient operations as well.

5. Summary

In summary, the drone-based road network inspection system was considered feasible from a business point of view by the expert system analysis performed by the participants. As a main activity, they proposed the creation of a service where the road surface information (condition and/or presence of foreign objects) collected by the drone network is offered to road users and operators in a processed form, namely as road quality assessment and as notifications of dangerous conditions on the road, respectively. A web-based service was developed to handle conflicts between drones during operation, which is capable of handling conflicts between individual vehicles. As a next step, individual obstacles and airspace definitions will be integrated into the conflict management system, and the solutions to these conflicts will be tested first in a simulated then real world environment. It is also an objective to improve the management of multi-vehicle conflicts by further developing the conflict management algorithm and to further optimize the efficiency of conflict management, reducing the time spent by vehicles executing conflict management commands.