1. Introduction
In recent years, and due to the mandatory need to continuously adapt the production flow, industrial companies are increasingly adopting fully automated internal logistic systems, namely based on AGVs, instead of manual or inflexible mechanical solutions (e.g., forklifts, conveyors, and others).
Considering the Industry 4.0 initiative, both AGVs, as well as mobile manipulators, are seen as strategic tools in the Factories of the Future. In a very competitive industrial environment, these can contribute to increase productivity and reduce the costs associated with the internal logistic system, ensuring an efficient material flow. Likewise, their introduction also allows human operators to be reallocated to more complex and ergonomic tasks, with increasing value to the final product. These set of characteristic makes AGV’s appealing for a wide range of industrial applications, such as goods transportation, end-of-line automation chain, warehouse and distribution. However, and despite their versatility, there is the need to deploy advanced multi-robot coordination algorithms in order to ensure the AGV’s continuous operation, guaranteeing the minimum tasks execution time and the smoothness of the vehicle’s movements.
The AGV’s fleet coordination problem has received wide attention from both the research and industrial fields. Typically, two main systems comprise any multi-AGV application: free-collision routing system [
1,
2] and scheduling system, that encompass both task scheduling and dispatching [
3]. The vehicle routing system is responsible for computing trajectories that minimize the total distance traveled by AGVs considering different constraints such as each vehicle’s carrying capacity and the plant layout where vehicles can circulate, while ensuring free-collision routes. Some approaches are based on time windows as proposed by the authors in [
4,
5,
6,
7,
8]. Here, the AGV route is constructed considering that one point can only be visited one time for only one vehicle at a given time interval. The feasibility of each route is evaluated, checking windows overlapping. In its turn, the scheduling system is associated with the task scheduling and their attribution to individual AGVs. This scheduling and dispatching should take into account some decision criteria, namely the task deadlines and traffic status, among others.
Bearing these ideas in mind, this paper proposes an integrated approach for both AGV route planning and task scheduling and dispatching. More in detail, the proposed routing algorithm called Time Enhanced A* (TEA*) is an extension of our previous work addressed in [
9,
10], that is, in this paper, further integrated with a scheduling module in order to minimize the tasks execution time. The main feature of TEA* is the addition of a temporal component to the known A* algorithm that generates routes efficiently, considering that each robot knows other robots’ positions during the time. Furthermore, TEA* is an on-line approach allowing its integration in dynamic environments.
The major contributions of our paper are to propose a Multi-Robot Coordination System which includes a time-based algorithm to generate free-collision routes based on the A* algorithm and a scheduling module that use the routing algorithm information to minimize a cost function, dependent on the following parameters:
This paper is organized as follows. In
Section 2 the state-of-the-art of path planning algorithms and multi-robot systems are presented.
Section 3 describes the proposed multi-robot routing algorithm.
Section 4 describes the industrial case scenario, followed by the comparison between the proposed approach with an alternative state-of-the-art [
11].
Section 5 and
Section 6 present the results achieved using Tabu-Search Method. Finally, some conclusions and the contribution of this paper are presented in
Section 7.
2. Related Work
In the last decade, the multi-robot coordination problem has been a target of many scientific studies. To solve it, several authors have proposed the use of meta-heuristic approaches to address both the problem of AGV routing and task scheduling. The authors in [
12,
13,
14,
15,
16], propose using a Genetic Algorithm to find the optimal or sub-optimal solution, which satisfies the routing system goals, including the minimization of the tasks completion time, minimal distance, among others. Likewise, ref. [
17] proposes an integrated solution that comprises a routing module based on genetic algorithms and a scheduling system based on bid auctions. Despite the potential of the proposed methods, normally these approaches treat each robot as an individual agent (without physical constraints) [
17,
18], simplifying the problem at hand. Furthermore, the results do not include industrial case scenarios. Alternatively, particle swarm strategies can also be applied to multi-AGV systems as in [
19,
20]. Ref. [
21] proposes a Particle Swarm Optimization (PSO) based algorithm, called Fractional Order Robotic Darwinian Particle Swarm Optimization (FORDPSO), integrated with a fuzzy system to optimize the driving of multi-robots in unknown environments. However, these methodologies are not yet sufficiently tested in real industrial environments.
Considering solely the use of a bid auction strategy, the authors in [
3,
22] propose a solution where each robot constructs ‘bids’ for each task and a central module receives the bids and assigns tasks considering the fitness function’s maximization. Similarly, ref. [
23] proposed different combinatorial bidding strategies, comparing its performance with single-item auctions.
Recognize the use of analytical methods, the authors in [
24] define a robot mission through Linear Temporal Logic formulas (LTL). An LTL approach considers that the truth of a declaration can be changed during the time. This work focus on minimizing the cost function, which is the maximum time between candidate solutions of an optimizing proposition. In [
25] the authors use a model based on Integer Linear Programming (ILP) to find paths that minimize the time until the last robot reaches its goal or minimizes the total traveled distance.
Generally, two different approaches define the architecture of any multi-robot system: a Centralized [
26] or Distributed [
27] methodology. The authors in [
28,
29] use a centralized architecture where one of the robots is the leader, and the others are the followers. Here, the major challenges are related to ensuring communication robustness and the algorithm flexibility to change the leadership. Other works use distributed architecture like in [
27,
30], where each robot calculates its path independently using, for example, a D* Algorithm [
31] and then, the path is broadcast for all robots, making that every robot knows all path information.
Regarding only task scheduling system, it typically aims for minimizing an objective function which includes system characteristics, such as the number of vehicles, the order in which the vehicles execute their missions, etc. In [
32], Kelen et al. compares two scheduling methods that determine the ideal number of vehicles for a given industrial scenario: the Shortest Job First and Tabu Search. Additionally, a routing method based on an enhanced Dijkstra algorithm, was used to manage the AGV’s path. The number of stoppages and the time that each vehicle waits for the mission assignment was not measured.
In [
20], Yu Zhang et al. proposes simple heuristics at the high-level layer, referred to as the ‘coalition’ level, that creates an abstraction layer relatively to specific details in robots’ specifications. Simple heuristics can be ‘MinProcTime’, that gives priority to the missions with shorter processing times, and another one can be the ‘MinStepSum’, similar to the ‘MinProcTime’, but determines the best solution incrementally when the ordering of the assignment are not pre-determined. However, in the simulation experiments presented were not considered an industrial scenario with real-world scheduling problems.
In the past half-decade, new approaches based on Artificial Intelligence (AI) are emerging. AI addressed in [
33,
34], is the science that seeks to study and understand the phenomenon of intelligence and, at the same time, a branch of engineering, as it seeks to build instruments to support human intelligence. In practice, an AI system besides storing and manipulate data can also acquire, represent, and manipulate knowledge. This manipulation concerns the ability to deduce or infer new knowledge from existing knowledge and use representation and manipulation methods to solve complex problems. This area of engineering is vast and has been the subject of huge investment from both business and research institutes. More specifically in Robotics, there is already a recent line of work on Multi-Agent Path-Finding (MAPF) [
35,
36,
37,
38]. On AGVs, the MAPF problem is to find the best paths, for a fixed number of agents, from their current locations to the final task position, where all agents have a free-collision path, as it is described in [
39,
40,
41]. Another use of AI in Robotics, is the combination of non-AI algorithms (time-window based or greedy) for the AGV coordination with AI for the prediction of future tasks [
42].
Despite the many scientific studies carried out, task scheduling solutions often resort to small heuristics (First-in-First-Out, Shortest-Distance), often decoupled from the trajectory planning system. In turn, trajectories are predefined in offline mode and designed to prevent, as far as possible, the occurrence of deadlocks. This type of solution often leads to the logistics system being oversized concerning the operation’s real needs.
3. Proposed Routing Algorithm
To overcome the challenges related with the multi-AGV coordination problem, presented in the previous sections, in this section a new methodology for AGV’s route planning is proposed. This novel methodology is called TEA* Algorithm [
9,
10], in which the paths are recalculated continuously, making it an online method. According to the vehicles’ movements and the environment changes, TEA* updates the paths of each AGV in order to avoid collisions and to guarantee the continuous operation of the logistic system. In fact, as in the majority of the industrial logistic systems, it is expectable that changes on initial task information and/or unpredictable events, such as delays in the transportation system as a result of obstacles presence [
21], can occur. In these scenarios, the adoption of online path planning algorithms, capable of dealing with such events, becomes mandatory.
As referred before, TEA* is based on the traditional A*, where a third dimension was added, the time. The input map has three dimensions: vertex’s coordinates (
x and
y) and a representation of the time, as shown in
Figure 1. The time is represented with temporal layers given by
(
denotes the maximum number of layers). Each temporal graph is a set of free and occupied/obstacles vertexes.
As far as computational complexity is concerned, many factors can influence the system (RAM, CPU, others), however, through [
43] where the test conditions are the same for all algorithms, it is possible to validate that A* is the most suitable algorithm for robot fleet management in different environments. Therefore, taking into account the case study carried out in [
43], and the results obtained in [
9], it can be concluded that TEA* is a practical, versatile and quite optimized algorithm in terms of path planning algorithms.
3.1. The TEA* Method
In multi-AGV systems, time is a crucial component for a better prediction of the vehicles’ positions. Besides the constantly recalculated paths, TEA* determines the route for each AGV during the temporal layers. This grants the identification of upcoming collisions, allowing them to be avoided with considerable anticipation. The path information for each AGV is converted to a busy vertex on the following robot’s map, allowing collision avoidance.
Consider a graph
G with a set of vertexes
and edges
(links between the vertexes), with a representation of the time [
] (as can be seen in
Figure 1). Each AGV can only starts and stops in vertexes and each vertex can only be occupied by only one vehicle at the time.
During the path search for a single AGV the neighboring vertexes are evaluated using a similar approach as A* algorithm [
44]. Moreover, each edge in each temporal layer, has a cost function value, denoted as
, given by the sum of two terms (see Equation (
1)).
Considering the path between and , the first term , represents the distance between the current vertex j to the initial vertex , in the k temporal layer. The second term, denoted as , is a heuristic value that calculates the distance to the final vertex . The terms and assign different weights to the distance and the heuristic function.
Each vertex, in each temporal layer, has different values of
and
, according with the Equation (
2). Here,
is given by the sum of the distance between the current vertex
j and the initial vertex
, being the edge distance between
j and its adjacent vertex
denoted as
.
The main differences between TEA* and the known A* algorithm are mainly concerned with the addition of the time component and can be defined as follows:
Definition 1. The neighbor vertexes belong to the next temporal layer. The neighbor vertexes of a vertex j () are given by the set of all adjacent vertexes in the next time component (). The number of temporal layers depends on the required iterations to achieve the final point of the mission and the map dimensions. Note that the larger the map, more time layers are required.
Definition 2. The neighbor vertexes include the vertex containing the AGV’s current position. The set of neighbor vertexes includes not only the adjacent vertexes but also the vertex corresponding to the position in analysis. This property allows a vehicle to maintain its position between consecutive time instants if any neighbor vertex is free. In this case, assumes a constant value that corresponds to the cost of keeping its position.
The Algorithm 1 describes the TEA* approach for a single AGV with the following parameters:
: Value of vertex j in the time layer k (Free—0 or Occupied/Obstacle—1).
: AGV l occupies the vertex j in the k time layer.
: Open list contains the vertex j in the k time instant. Each item contains the respective cost value, .
: initial vertex.
: final vertex.
: adjacent vertex of vertex j.
: The vertex i in the time layer is the parent vertex of vertex j in the instant k.
: Heuristic Value for vertex j in k temporal layer.
: Distance Value for vertex j in k temporal layer.
: Distance value of the edge .
3.2. Smoothing Trajectories
For TEA* to be applied, the shop floor layout requires to be modeled as a set of vertexes and edges (links between the vertexes). Each AGV travels on these graph paths, from one node to another, through a pre-defined set of edges. Each edge is represented as a cubic Bézier curve (as proposed in article [
45]), given by Equation (
3).
Here,
denotes an integer value between 0 and 1 according with the AGV’s position in the curve,
is the initial point of the curve, and
,
,
, defines the spline’s curvature.
Figure 2 represents a portion of the map which contains Bézier curves and straight lines.
Algorithm 1:TEA* Algorithm |
|
5. Routing Algorithms Comparison
Considering the industrial scenario previously presented,
Table 1 illustrates the mission execution time for each vehicle with the TEA* Algorithm, using 10 AGVs and 30 tasks. For each AGV, the time of advancement (
) and the stopping time (
) are reported. In
Figure 4 is illustrated the final solution found for each vehicle. Two black circles represent possible collision points in different robot paths, but the third dimension of TEA* allows AGVs to share the same trajectory by passing at different times at the common points, i.e., avoiding possible collisions.
Comparing the results achieved with the TEA* (
Table 1), with the results presented by the authors in [
11] (
Table 2), it is possible to conclude that TEA* is advantageous mainly considering the
of the last vehicles (AGVs 6, 7, 8, 9, 10). Conversely, the
of the AGVs 3, 4 and 5 surpasses the value of the same robots in
Table 2. However, in these cases the
and the
are lower in TEA*. Therefore, this algorithm presents itself as a better solution for multi-robot path planning.
The average time (
) to complete the missions list with TEA* is
s and in the case of [
11] this time is approximately
s. The last vehicle in [
11] takes
s to finish its tasks, while in TEA*, the last vehicle performs its tasks in
s.
To highlight the capability of TEA* Algorithm to optimize the routes even with considerable workload in the system,
Table 3 presents the same industrial scenario but now with 20 AGVs and 60 tasks. Note that the difference in the average time between 10 and 20 AGVs are approximately
s. The fact that a vehicle considers the current and future positions of each vehicle as obstacles during the discrete-time, gives it the possibility to wait for some instants for the traveling of previous AGV, instead of calculating a longer deviation.
For the sake of completeness, it is important to refer that the TEA* Algorithm has been designed to be integrated into dynamic industrial environments, thus allowing direct scaling concerning the number of robots to be used. The structure of the algorithm itself is already prepared for different exchanges of industrial scenarios.
6. Task Scheduling Algorithm
As referred earlier, the problem of AGV coordination is not only closely related with the route planning of the AGVs, but also with the task scheduling. The performance of the routing algorithm can be improved. The AGV execution order affects the calculation of routes since the path positions over time for a given vehicle are obstacles for the following AGVs. If the execution order changes, the paths and respective task execution times for each vehicle are different.
6.1. Tabu Search Method
The Tabu Search Method is a ‘meta-heuristic’ adaptive method of local search in continuous exploration within a search space, moving from one solution to another, the Tabu Moves, diversifying the solutions found in this process of the search for an improved solution [
47]. The best permissible movement is the one with the highest evaluation in the vicinity of the current solution regarding target function value and taboo restrictions. Thus, the ‘meta-heuristic’ Tabu Search is an iterative search algorithm characterized by dynamic memory and consisting of two parts: initialization and search.
Starting from an initial randomly generated solution or using a heuristic, the Tabu Search will evaluate a set of different mutations (neighborhood exploration) of the current solution in each iteration. The best mutation will be accepted, and the changes made saved in a Tabu List adopted to store the most used changes, which are classified as prohibited in later iterations. This strategy is necessary to avoid a return to solutions already checked previously.
Therefore, in this method, in each iteration, the evaluation function consists of validating a certain quantity of new solutions, where the best solution, based on the objective function, is accepted, even if its cost is higher than the cost of the current solution. Thus, the algorithm chooses the new solution that produces an improvement or the least deterioration in the cost function (an attempt to evade minimal locations). The Tabu Search algorithm runs until a stop criteria is reached.
Figure 5 presents the main blocks of the Tabu Search method.
In the AGV scheduling problem presented, the final objective is the allocation of the n sub-tasks by the j available AGVs, in order to minimize the total time for the overall task completion. The main goal is to determine/distribute the best sequence of attendance of the sub-tasks by the AGVs to minimize the total time of execution.
For the problem presented, the initial solution (sequence of sub-tasks assigned to each AGV) is generated using the closest neighboring heuristic, as presented in Algorithm 2. In other words, each AGV (and taking into account the last sub-task performed, which influences its position on the map) is assigned to the next sub-task with lower cost (shorter travel time/distance).
This process is executed cyclically until all sub-tasks have been assigned to one, and only one, AGV.
Algorithm 2: Closest Neighbour Algorithm—Pseudo Code |
|
The Algorithm 3, describes the implementation of the Tabu Search used in the proposed approach.
Algorithm 3: ‘Meta-Heuristic’ Tabu Search—Pseudo Code |
|
6.2. TEA* Algorithm with Tabu Search Method—Results
The Tabu Search Method was implemented to find better vehicle configuration and to schedule the order in which vehicles execute their tasks. A configuration comprises the order in which vehicles should be processed by the TEA* Algorithm.
The AGV execution order affects the calculation of routes since the path positions over time for a given vehicle are obstacles for the following AGVs. If the execution order changes, each vehicle’s paths and respective task execution times will be different.
The optimization goal is the minimization of the three following parameters:
Time of the last vehicle, denoted as in seconds;
Average Time of the missions execution, denoted as in seconds;
Number of Stoppages, denoted as ;
That were aggregated in the following cost function (Equation (
4)):
Here, , and are components that are weighting parameters. In the simulation experiments the following values were used, respectively , , . These were manually defined considering an iterative and experimental way, with the main goal of minimizing the cost function c.
The configuration that leads to the lowest cost function is chosen as the better solution. In our approach is not required to achieve the optimal solution, a near-optimal configuration that leads to an efficient TEA* execution is enough.
To obtain the initial configuration of the Tabu Search Method, a heuristic function was defined. It consists of the path computation for each vehicle without consider the other vehicles as obstacles (optimal solution) and ordering it by decreasing the order of execution task times. The objective is to process firstly the longer paths minimizing the number of stoppages. The candidate solutions are generated, changing two by two the vehicle execution order from the current solution.
Table 4 presents the results for the TEA* Algorithm using the AGV configuration solution found by the Tabu Search method. The average time for completing all tasks is lower, but the more significant improvement was the waiting time. In
Table 1 using as initial configuration =
AGVs were stopped 15 s. Using the Tabu Search solution =
, total waiting time was 6 s.
Considering 20 AGVs, besides the stoppage time to be higher, the task completion time is lower than TEA* results without scheduling module. In
Table 3 the total time to complete all tasks was
s and in
Table 5 this time was
s.
7. Conclusions
This article proposes a multi-AGV system that comprises a routing algorithm based on the search method A*. This algorithm is suitable for multi-robot applications, avoiding collisions and deadlocks and guaranteeing any industrial scenario required safety levels. To optimize the results achieved scheduling method (Tabu Search) minimizes a fitness function defined by several parameters calculated by TEA*. The two modules work cooperatively, sharing the TEA* information.
Our work’s major contributions are: (i) Presentation of a promising approach for multi-AGV applications in warehouse environment, improving the flexibility and efficiency of the complete system; (ii) Validation of an on-line Multi-Robot Coordination Algorithm comparing it with a state-of-the-art alternative.
As future work, it will be interesting to validate the TEA* Algorithm in a real environment, with a real robotic system, by comparing it to the simulation results.