Edge Computing-Driven Real-Time Drone Detection Using YOLOv9 and NVIDIA Jetson Nano

Abstract

Drones, with their ability to vertically take off and land with their stable hovering performance, are becoming favorable in both civilian and military domains. However, this introduces risks of its misuse, which may include security threats to airports, institutes of national importance, VIP security, drug trafficking, privacy breaches, etc. To address these issues, automated drone detection systems are essential for preventing unauthorized drone activities. Real-time detection requires high-performance devices such as GPUs. For our experiments, we utilized the NVIDIA Jetson Nano to support YOLOv9-based drone detection. The performance evaluation of YOLOv9 to detect drones is based on metrics like mean average precision (mAP), frames per second (FPS), precision, recall, and F1-score. Experimental data revealed significant improvements over previous models, with a mAP of 95.7%, a precision of 0.946, a recall of 0.864, and an F1-score of 0.903, marking a 4.6% enhancement over YOLOv8. This paper utilizes YOLOv9, optimized with pre-trained weights and transfer learning, achieving significant accuracy in real-time drone detection. Integrated with the NVIDIA Jetson Nano, the system effectively identifies drones at altitudes ranging from 15 feet to 110 feet while adapting to various environmental conditions. The model’s precision and adaptability make it particularly suitable for deployment in security-sensitive areas, where quick and accurate detection is crucial. This research establishes a solid foundation for future counter-drone applications and shows great promise for enhancing situational awareness in critical, high-risk environments.

1. Introduction

Unmanned aerial vehicles (UAVs), another name for drones, are tiny aircraft that may operate remotely or on their own in autonomous mode. These devices come in various forms and sizes, from small consumer models to large, military-grade drones with cutting-edge sensors and weaponry [1]. Over the past 15 years, the drone industry has experienced exponential growth, becoming increasingly accessible to the public and available at more affordable prices [2]. Depending on their payload capacity, drones are employed in a range of applications. These includes, but are not limited to medical aid [3,4,5], disaster response [6,7,8], logistic and transportation [9,10], agriculture [11,12], remote sensing [13,14], space exploration [15], and inspection activities [16,17,18]. The multipurpose uses [19] of drones are illustrated in Figure 1. This versatility, flexibility, and ability to access remote or dangerous areas make them valuable tools in numerous industries. In the US, there were 791,597 drones registered as of 1 October 2024, with 387,746 of them being recreational [20].

The proliferation of drones by terrorist organizations and individuals involved in illegal drug trafficking has increased significantly. Additionally, the growing number of hobbyist drone operators introduces the risk of interference with critical operations such as firefighting and emergency response efforts. Consequently, there has been a surge in the misuse of drones. To counter unauthorized and undesirable drone interventions, automated drone detection is deemed necessary.

For the effective development of a drone detection system, it is crucial to consider the various threats posed by unmanned aerial vehicles. Figure 2 summarizes the primary categories of drone threats, which include unauthorized surveillance, smuggling, harassment, interfering with aircraft, violating no-fly zones, espionage, and collisions.

During a national outdoor event in August 2018, President Nicolas Maduro of Venezuela faced an assassination attempt when two explosive-laden drones targeted him, although they ultimately failed [21]. This incident stands out as the first recorded use of a drone in an attack on a country’s head of state. The closure of Gao International Airport ensued after a recreational drone collided with a runway truck, causing considerable damage and subsequent flight disruptions, emphasizing the urgent need for enhanced drone safety protocols at airports [22]. When suspicious drones were found close to the runway at New Delhi’s Indira Gandhi International Airport in August 2017, there was a serious security violation. This incident forced a temporary suspension of flight operations, leading to widespread disruptions. The event underscored the potential risks and vulnerabilities airports face from unauthorized drone activity [23]. In January 2018, Khmeimim Russian military bases in Syria were targeted by a swarm of 13 homemade aerial drones [24]. The UAV attacks on 14 September 2019 at Saudi Arabia’s Khurais oilfield and Abqaiq processing plant resulted in a 5.7-million-barrel decrease in oil production, escalating crude oil prices by 15% globally [25,26]. A smuggling operation involving drones that transported mobile phones worth over HKD 620 million from Hong Kong has been dismantled by authorities [27]. Moreover, incidents of drone violations have been reported during sports events, with drones illegally flying over football stadiums to cause disruptions [28]. The massive use of drones for offensive, defensive, and intelligence purposes was evident during the Ukrainian–Russian war in 2022 [29,30]. In 2024, there has been a notable increase in drone-related incidents, emphasizing the urgent need for effective counter-drone technologies and sensible regulations for small UAV usage. These incidents, which include smuggling, illegal intrusions, collisions, and technical malfunctions, have impacted various sectors. Most unauthorized drone activities occurred in sensitive locations like airports, border areas, prisons, and residential communities. Since January 2024, approximately 218 drone incidents have been reported [31]. Such incidents highlight the importance of UAV detection technologies in preventing unauthorized drone activity. These systems are engineered to detect drones; determine their position; and gather critical information about their type, direction, and speed to enable timely neutralization.

The tasks of detecting and classifying UAVs present several significant challenges, which researchers [32,33] are actively working to address. Some of these challenges include UAVs varying in size and speed, complicating detection and classification due to their distinct flying characteristics and shapes. High speeds, unpredictable flight patterns, and resemblance to birds or airplanes make accurate UAV identification challenging, necessitating fast and precise detection systems. UAVs operate at various altitudes and ranges, with low-altitude, short-range drones posing unique detection difficulties and potential risks for traditional countermeasure systems. Weather, urban obstructions, and adverse lighting can impair sensor effectiveness, leading to false positives or negatives in UAV identification.

The challenges in drone detection are addressed by utilizing a transfer learning approach with a Jetson Nano and a camera setup at the base station. Using YOLOv9 and deep learning methods, this setup enables the detection of drones at various altitudes and ranges. The camera captures data, which are processed in real time by the Jetson Nano at the base station, improving the detection and classification accuracy even in complex environments. This setup facilitates quick and effective responses to potential threats by analyzing factors such as speed, flight patterns, and environmental conditions.

The key contributions of the paper are as follows:

• We employ the YOLOv9 algorithm, optimized through transfer learning, to train a comprehensive drone detection dataset.
• The proposed method integrates the NVIDIA Jetson Nano platform for real-time drone detection, enabling efficient processing with reduced response time and energy consumption.
• Environmental adaptability: The algorithm is tested across different environmental conditions, including daytime, sunny, and evening settings, demonstrating robustness in varying lighting and weather conditions.
• Altitude-based drone detection: The detection system is validated at different altitudes, including 15 feet, 60 feet, and 110 feet, ensuring high accuracy across varying drone heights.

The rest of the paper is organized as follows: Section 2 outlines our approach to drone detection using various deep learning techniques. A comprehensive overview of the proposed system is provided in Section 3, detailing its architecture, hardware components, software framework, and the dataset employed, as well as the experimental setup that addresses the training and validation of the dataset, including transfer learning on a low-power edge computing device. Section 4 presents the results of the study and provides a discussion of our findings. Section 5 concludes with a summary of our results and discusses future research directions.

2. Overview of Drone Detection Technologies

Drone detection entails the identification and tracking of drones as they navigate through the sky. Drones, during their operation, release a variety of signals, such as noise, radio frequency (RF) emissions, and sound waves. Advanced detection technologies, including radar, LIDAR, and vision sensors, can pick up these signals. Depending on the detection technology used, drone detection can be classified into four key categories [34].

• Acoustic Detection [35,36,37,38]: It captures the unique sound patterns from drone propellers and motors [39]. This method is relatively cheaper. One of its benefits is that it does not need a direct line of sight with the drone to function well in dimly lit areas and difficult environments like fog or dust. In isolated locations with little background noise, this technique performs admirably. However, its accuracy is significantly affected by background noise and weather conditions such as rain and wind [40]. These factors can interfere with the microphone arrays, making it harder to isolate the drone’s acoustic signature and potentially leading to false positives or reduced detection range. The detection capability extends to 200 m, but this range may differ depending on environmental factors.
• RF-Based Detection [41,42,43,44]: Drones typically communicate with their controllers using RF signals in the 2.4 GHz to 5 GHz range [45]. The 2.4 GHz band is commonly used for longer-range communication, while the 5 GHz band can provide higher data rates and real-time control [46]. A drone’s controller can listen in on signals transmitted by the drone using RF sensors. However, this method can struggle with distinguishing between drones and other RF sources like Wi-Fi or Bluetooth devices. Additionally, it may require precise calibration and can be affected by signal interference in urban environments [47].
• Radar-Based Detection [48,49,50,51]: Radars are commonly utilized for aircraft detection in military and civil fields, including aviation, and are thus recognized as reliable tools for detecting drones. It transmits radio waves, typically in the microwave range, and analyzes the reflected waves to determine the presence, distance, and speed of objects like aircraft or drones [52]. The major advantage of radar-based detection is its high accuracy in identifying and localizing objects compared to other methods. However, traditional radars face limitations, as they are designed to detect larger aircraft and often struggle with smaller objects like drones [53]. They may also have difficulty distinguishing between hovering drones and static reflective objects, as well as between small drones and birds. Furthermore, the drawbacks of radar-based drone detection include high costs and the necessity for specialized skills for installation and maintenance [54].
• Vision-Based Detection [55,56,57,58]: A visual detection system captures drone images or videos using daylight, infrared, or thermal cameras and applies computer-vision-based algorithms for detection [59]. These systems rely on computer vision and deep learning to identify drones by analyzing appearance features such as color, shape, contour, and motion across successive frames [60]. This approach leverages both object recognition and motion tracking for effective drone identification in diverse environments. Visual drone detection provides precise identification and tracking through visual cues like shape and markings, which are difficult for acoustic and RF methods to detect. Unlike radar-based systems, it works efficiently despite signal interference and remains reliable in different environmental conditions, including adverse weather. Additionally, vision systems can integrate advanced artificial intelligence [61] for automated decision-making, enhancing overall detection accuracy and efficiency in dynamic airspace environments. Visual-based drone detection is categorized into two primary approaches: traditional techniques and deep learning methods [62]. Traditional techniques rely on filtering [63], threshold segmentation [64], and morphological operations [65] for handcrafted feature extraction. Despite their advantages, these methods are constrained by speed, accuracy, and their ability to adjust to environmental changes [66]. In contrast, contemporary research into visual-based drone detection and identification has increasingly leveraged deep learning models and feature learning to improve accuracy [67].

Deep learning, a cutting-edge method within machine learning, has gained prominence due to its exceptional performance and precision in results [68,69]. It is particularly adept at extracting features directly from raw data. The term “deep” refers to the multiple layers that exist between the input and output layers, where features are extracted in a hierarchical and nonlinear manner [70]. Object detection through deep learning methods fundamentally utilizes CNNs to extract features from images. These techniques are generally classified as one-stage and two-stage detectors [71].

2.1. Two-Stage Detectors

Two-stage detectors [72] initially generate region proposals (RoIs) and subsequently classify and refine the bounding boxes, resulting in higher accuracy but slower performance. Figure 3 illustrates the fundamental framework of two-stage detectors. Some of the most recognized region proposal-based methods are R-CNN [73], Fast R-CNN [74], Faster R-CNN [75], and Mask R-CNN [76].

R-CNN, developed by Girshick in 2014 [64], advanced the field of object detection by combining region proposal methods with deep learning techniques. It detects objects by creating region proposals through selective search, extracting features with a CNN, and then classifying and refining bounding boxes. Despite its effectiveness, R-CNN faces issues like slow processing, a complex training process, high memory demands, reliance on external proposals, and limited adaptability to different object scales.

Fast R-CNN [74] improves on R-CNN’s inefficiencies by processing the entire image in one forward pass and employing ROI pooling to generate fixed-size feature maps from the original maps, which significantly boosts computation speed. It replaces the SVM with a softmax layer for classification and integrates various architecture components, leading to enhanced speed, reduced memory usage, and an end-to-end training process that utilizes multi-task loss for labeled regions of interest.

Building on Fast R-CNN, Faster R-CNN [75] introduces a Region Proposal Network (RPN) that generates region proposals from convolutional feature maps, which eliminates the requirement for a distinct proposal stage. This integration enhances both speed and accuracy in object detection. The architecture also employs ROI pooling, which allows for efficient feature extraction from proposals of varying sizes [77].

Mask R-CNN [76] is an extension of Faster R-CNN that adds a branch for predicting segmentation masks on each detected object in addition to bounding box and class predictions. This enables instance segmentation, allowing the model to identify and delineate individual objects within an image. It enhances object detection by providing detailed pixel-level information for each object instance.

2.2. One-Stage Detectors

One-stage detectors [78] differ from conventional two-stage models by integrating object localization and classification into a single process, which allows for faster detection speeds that are optimal for real-time applications. The architecture of these detectors is depicted in Figure 4. Leading examples of this efficient strategy include the Single-Shot Multibox Detector (SSD) [79] and YOLO variants [80].

The SSD [81] architecture predicts class labels and bounding box offsets for a fixed number of default boxes at various scales across multiple feature layers. By incorporating anchor boxes with a variety of aspect ratios and sizes, SSD is able to detect objects of different shapes and dimensions in a single pass through the network.

YOLO [82] is a pioneering real-time object detection model that was introduced in 2015. It utilizes a fixed grid methodology, allowing the model to process an entire image in one go through a convolutional neural network, thereby simultaneously predicting bounding boxes and class probabilities. By dividing the image into regions, YOLO achieves high speeds and efficiencies, making it well suited for real-time applications while maintaining competitive accuracy across various computer vision tasks. Illustrated in Figure 5, the YOLO family has seen several iterations, with each version designed to improve upon previous models and tackle their limitations [83,84,85,86,87].

Table 1. Comparison of YOLO versions based on architecture, framework, mAP, and FPS.

YOLO Variant	Publication Date	Anchor	Framework	Backbone	mAP	FPS	Detection Model
YOLOv1	2016	No	Darknet	Darknet-19	63.4%	45	Single-stage detector
YOLOv2 [88]	2017	Yes	Darknet [89]	Darknet-19	76.8%	45	Single-stage detector
YOLOv3 [90]	2018	Yes	Darknet	Darknet-53	57.9%	30	Single-stage detector
YOLOv4 [91]	2020	Yes	Darknet	CSPDarknet53	65.7%	62	Single-stage detector
YOLOv5 [92]	2020	Yes	PyTorch [93]	CSPDarknet53	50.7%	140	Single-stage detector
YOLOv6 [94]	2022	Yes	PyTorch	EfficientRep	52.3%	123	Single-stage detector
YOLOv7 [95]	2022	Yes	PyTorch	E-ELAN	56.8%	161	Single-stage detector
YOLOv8 [96]	2023	Yes	PyTorch	CSPDarknet53	60%	140	Single-stage detector
YOLOv9 [97]	2024	Yes	PyTorch	AdvancedCSPNet	71.2%	140	Single-stage detector

provides a detailed comparison of different YOLO versions, showing improvements in their architecture, framework, mean average precision (mAP), and speed (FPS). From YOLOv1’s grid-based detection in 2015 to YOLOv9’s new Programmable Gradient Information (PGI) framework in 2024, each version has enhanced YOLO efficiency and accuracy.

The progression of YOLO models, particularly from YOLOv3 to YOLOv9, showcases improvements in speed, accuracy, and adaptability for complex object detection scenarios like drones. YOLOv9 addresses some of the limitations in previous versions, such as poor performance in adverse conditions and inefficient gradient propagation. This study will compare YOLOv9’s performance on Jetson Nano with earlier versions to determine its practical application in real-world drone detection.

3. Experimental Setup for Drone Detection Using YOLOv9

YOLOv9 is an advanced real-time object detection system that integrates cutting-edge deep learning methods and architectural innovations to deliver exceptional performance in detecting objects. YOLOv9 addresses critical challenges in object detection by incorporating reversible functions for data integrity, Programmable Gradient Information (PGI) [98] for precise gradient updates, and Generalized Efficient Layer Aggregation Network (GELAN) [99] to streamline feature extraction and speed. These advancements resolve information bottlenecks and enhance the model’s flexibility, allowing for a more efficient and accurate detection process compared to previous YOLO versions. Figure 6 shows the overall architecture of YOLOv9, showcasing the streamlined and efficient approach used for drone detection.

YOLOv9 employs a highly efficient architecture to enhance object detection accuracy and speed. The backbone leverages CSPNet [100] for optimizing gradient flow and ELAN [99] for improving processing speed while maintaining a lightweight design. This combination enables the effective extraction of multi-scale features from input images. The backbone also incorporates RepNCSP-ELAN 4 blocks, which integrate RepNBottleneck and CSP modules for detailed feature representation, allowing the model to capture both global and local patterns.

The neck of YOLOv9 improves feature fusion and aggregation, vital for detecting objects of varying sizes and scales. PANet modules are employed here to strengthen feature representation. YOLOv9 also introduces PGI (Propagation Guided Improvement), which enhances gradient backpropagation, facilitating faster training convergence and better model performance.

In the head of YOLOv9, the network predicts the final bounding boxes, class probabilities, and objectness scores. This section uses reversible functions to preserve data integrity and minimize information loss, which in turn boosts prediction accuracy. The architecture includes Adown blocks for efficient downsampling, maintaining critical spatial information while reducing feature map size.

YOLOv9’s customized loss function ensures effective optimization during training, and non-maximum suppression (NMS) is applied to refine detection results, significantly improving both efficiency and accuracy in object detection tasks.

The bounding box, classification loss, and objectness loss smaller loss functions—each addressing a distinct facet of the object identification task—combine to form the loss function in the context of YOLOv9. This is how it is computed [101]:

$${\mathrm{L}}_{\mathrm{Y}\mathrm{O}\mathrm{L}\mathrm{O}\mathrm{v}9}={\mathsf{\lambda }}_{\mathrm{b}\mathrm{o}\mathrm{x}}.{\mathrm{L}}_{\mathrm{b}\mathrm{o}\mathrm{x}}+{\mathsf{\lambda }}_{\mathrm{c}\mathrm{l}\mathrm{s}}.{\mathrm{L}}_{\mathrm{c}\mathrm{l}\mathrm{s}}+{\mathsf{\lambda }}_{\mathrm{o}\mathrm{b}\mathrm{j}}.{\mathrm{L}}_{\mathrm{o}\mathrm{b}\mathrm{j}}$$

(1)

The weights λ_box, λ_cls, and λ_obj control how much each component of the model’s error contributes during training. These weights can be fine-tuned using techniques from previous YOLO models, like YOLOv5 and YOLOv8.

The bounding box regression loss checks how well the model predicts the location and size of objects. It uses the Mean Squared Error (MSE) between the predicted and actual box coordinates to improve how well the model finds objects. L_box is calculated as follows:

$${\mathrm{L}}_{\mathrm{b}\mathrm{o}\mathrm{x}}={\displaystyle \frac{1}{\mathrm{N}}}{\sum }_{\mathrm{i}=0}^{\mathrm{N}}[{\left({\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{i}}^{\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}}\right)}^2+{\left({\mathrm{y}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}}^{\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}}\right)}^2+{\left({\mathrm{w}}_{\mathrm{i}}-{\mathrm{w}}_{\mathrm{i}}^{\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}}\right)}^2+{\left({\mathrm{h}}_{\mathrm{i}}-{\mathrm{h}}_{\mathrm{i}}^{\mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}}\right)}^2]$$

(2)

where L_box represents the bounding box regression loss and N denotes the total number of bounding boxes. The predicted center coordinates, width, and height of the bounding box are given as x_i, y_i, w_i, and h_i, respectively. Meanwhile, $x_i^{true}$, $y_i^{true}, w_i^{true}, h_i^{true}$ correspond to the actual center coordinates, width, and height of the bounding box.

Classification loss measures how correctly the model identifies the type of objects. It uses Cross Entropy Loss to compare the predicted class probabilities with the actual labels to boost classification accuracy. L_cls is calculated as follows:

$${\mathrm{L}}_{\mathrm{c}\mathrm{l}\mathrm{s}}=-{\displaystyle \frac{1}{\mathrm{N}}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{\sum }_{\mathrm{c}=1}^{\mathrm{C}}{\mathrm{y}}_{\mathrm{i}}^{\left(\mathrm{c}\right)}\log {\widehat{\mathrm{p}}}_{\mathrm{i}}^{\left(\mathrm{c}\right)}$$

(3)

where L_cls denotes the classification loss, N represents the number of predicted bounding boxes, and C is the number of classes. The term $y_i^{\left(c\right)}$ is true for class c (with 1 indicating that the object belongs to class c and 0 otherwise), while ${\widehat{p}}_i^{\left(c\right)}$ represents the predicted probability that the object belongs to class c.

Objectness loss checks if the model correctly identifies whether an anchor box contains an object. It uses MSE between the predicted and actual objectness scores to improve object detection. L_obj is calculated as follows:

$${\mathrm{L}}_{\mathrm{o}\mathrm{b}\mathrm{j}}=-{\displaystyle \frac{1}{\mathrm{N}}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}\left[{\mathrm{y}}_{\mathrm{i}}^{\mathrm{o}\mathrm{b}\mathrm{j}}\log {\widehat{\mathrm{p}}}_{\mathrm{i}}^{\left(\mathrm{o}\mathrm{b}\mathrm{j}\right)}+(1-{\mathrm{y}}_{\mathrm{i}}^{\mathrm{o}\mathrm{b}\mathrm{j}}\right)\log (1-{\widehat{\mathrm{p}}}_{\mathrm{i}}^{\left(\mathrm{o}\mathrm{b}\mathrm{j}\right)})]$$

(4)

where L_obj represents the objectness loss, ${\mathrm{y}}_{\mathrm{i}}^{\mathrm{o}\mathrm{b}\mathrm{j}}$ is the true label indicating if the bounding box contains an object (1 if it does and 0 if it does not), and ${\widehat{\mathrm{p}}}_{\mathrm{i}}^{\left(\mathrm{o}\mathrm{b}\mathrm{j}\right)}$ is the predicted objectness score or confidence level.

The drone detection process in YOLOv9 is both streamlined and efficient:

• Input Handling: The image is passed through the backbone, where GELAN extracts multi-scale features.
• Feature Aggregation: These features are processed in the neck using PGI to enhance gradient flow and feature fusion, improving detection across various object sizes.
• Prediction Stage: Finally, the head uses reversible functions to maintain data accuracy while predicting bounding boxes, class probabilities, and objectness scores.

3.1. Training and Validation of Dataset

The proposed approach for drone detection involves four key steps, as illustrated in Figure 7. It begins with dataset preparation, which serves as input to the detection framework. The next step is training the model to accurately detect drones. In the third phase, the trained model is evaluated on various drone datasets to test its detection performance. Lastly, the model’s performance is assessed and then deployed for real-time drone detection operations.

For our drone detection project, we compiled a diverse dataset from several public sources, including Kaggle, Ms COCO, and Google. This dataset features images captured from different altitudes, angles, backgrounds, and perspectives to ensure comprehensive variability. Additionally, we supplemented this dataset with images collected from our drone flights. In total, a dataset comprising 9995 images depicting various types of drones was assembled. This includes 2467 images from Kaggle, 2691 images from Roboflow, 848 images collected during our drone flights, 3471 images from the MS COCO dataset, and 518 randomly sourced images from Google. For training and evaluating the YOLOv9 model, a 70:30 train–test split was applied, assigning 6996 images for training and 2999 images for testing. For training and evaluating our YOLOv9 model, we applied for a 70:30 train–test split, assigning 6996 images for training and 2999 images for testing.

3.2. Training of Custom Dataset with YOLOv9

To train YOLOv9 for drone detection, the experimental environment uses high-performance resources, ensuring repeatable results. The setup involves an Nvidia GeForce RTX 4060 GPU and Intel i9-12900 CPU (8 cores) in a laptop-based configuration. This system, supported by 32 GB DDR4-3200 RAM, runs on a 64-bit Ubuntu platform. The software environment includes Cuda 12.1, PyTorch 2.2.1, and Python 3.11.8, which are essential for the effective execution and real-time performance of the model.

Training the YOLOv9 model requires configuring key hyperparameters such as the learning rate, number of epochs, batch size, input dimensions, and weight initialization strategy. The model is trained using the stochastic gradient descent (SGD) algorithm, which utilizes backpropagation to adjust its parameters. The learning rate, a crucial factor, controls the speed of learning and is generally set to a small positive value between 0 and 1. Proper tuning of these parameters is essential to achieve optimal accuracy and performance of the model.

A higher learning rate accelerates the training process, thereby reducing the time needed to train the model. However, this can lead to increased average loss and decreased accuracy. In contrast, a lower learning rate results in a slower training process and may cause the model to become stuck at a high training error. Thus, it is crucial to find the optimal learning rate to balance training speed with model accuracy [102]. After determining the optimal model during the training phase, it was converted into a TensorRT model for deployment on the Jetson Nano. The model’s performance was then evaluated using the NVIDIA DeepStream SDK. Detailed parameter settings for the training process, including data augmentation techniques, can be found in

Table 2. Parameters for training the model.

Training Parameter	Value
Class	1
Batch Size	8
Epoch	100
Initial Learning Rate	1 × 10−4
Final Learning Rate	1 × 10−3
Optimizer	SGD

, while Algorithm 1 outlines the pseudo-flow graph of the training model. For deployment on the Jetson Nano, we selected the compact variant of YOLOv9.

Algorithm 1: Training of YOLOv9

3.3. Transfer Learning

Transfer learning [103] is an approach in which a model that has been previously trained on a large dataset is adapted for a particular task by leveraging a smaller dataset. In this scenario, a collection of drone-related images is initially processed on a CPU to create a robust model capable of detecting drones. The insights gained from this training phase are then applied to an NVIDIA Jetson Nano, a compact AI edge device, where the model undergoes fine-tuning to optimize it for real-time detection. By utilizing the pre-trained weights, the Jetson Nano is able to perform real-time object detection efficiently, minimizing resource consumption while delivering fast and accurate results in practical applications. Figure 8 outlines transfer learning techniques.

3.4. System Architecture and Test Environment on Low-Power Edge Computing Device

In our system architecture, the Jetson Nano has been selected as the edge computing device for implementing the YOLOv9 algorithm for real-time drone detection. This device provides substantial computational capacity, offering 472 GFLOPs suitable for executing contemporary deep learning assignments, all while being competitively priced. Powered by a 64-bit quad-core ARM A57 processor operating at 1.43 GHz and containing 128 CUDA cores consuming 5 to 10 watts, it also includes 4 GB of LPDDR4 memory [104]. The Jetson Nano is specifically designed for edge computing applications, distinguished by its small size, cost-effectiveness, and minimal energy consumption. Algorithm 2 describes the pseudo-flow graph of the real-time drone detection on Jetson Nano.

Algorithm 2: Transfer learning on Jetson Nano for real-time drone detection

Figure 9a,b illustrate the system architecture for real-time drone detection using the NVIDIA Jetson Nano edge computing platform. The architecture comprises two main components: a high-speed camera and the Jetson Nano. Initially, a high-speed camera is positioned in the detection area to monitor the surroundings continuously. It captures real-time image data, which are then transmitted to the Jetson Nano for processing.

The Jetson Nano is equipped with the YOLOv9 algorithm, which has been pre-trained to analyze incoming image data in real time. This algorithm performs the detection and localization of drone targets within the captured images. Detected drone targets are identified, and their locations are pinpointed accurately. The flowchart of real-time drone detection using YOLOv9 and NVIDIA Jetson Nano is shown in Figure 10.

4. Results and Performance Evaluation

4.1. Assessment Indicators

To evaluate the effectiveness of the proposed approach, various critical performance metrics, such as precision, recall, F1-score, and mean average precision (mAP), are utilized. These metrics collectively offer a thorough assessment of the model’s object detection performance and accuracy.

Intersection over Union (IoU) evaluates the overlap between the predicted and ground truth bounding boxes, indicating how well the predicted box matches the actual object’s location. With IoU values ranging from 0 to 1, a value of 0 means no overlap, while a value of 1 signifies perfect alignment. Higher IoU values represent more precise localization, where the predicted bounding box closely matches the ground truth, as demonstrated by the following equation.

$$\mathrm{I}\mathrm{o}\mathrm{U}={\displaystyle \frac{\mathrm{A}\mathrm{e}\mathrm{r}\mathrm{a} \mathrm{o}\mathrm{f} \mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{l}\mathrm{a}\mathrm{p}}{\mathrm{A}\mathrm{e}\mathrm{r}\mathrm{a} \mathrm{o}\mathrm{f} \mathrm{U}\mathrm{n}\mathrm{i}\mathrm{o}\mathrm{n}}}$$

(5)

The confusion matrix is a square matrix of size n × n, where n is the number of classes, used to evaluate the performance of a model. In the context of a drone detection system utilizing YOLOv9 with two classes of drone and other objects, the columns represent the true classes of detected objects, while the rows represent the predicted classes made by the YOLOv9 model.

Metrics such as precision, recall, F1-score, and accuracy are derived from the value in the confusion matrix true positive (TP), false positive (FP), true negative (TN), and false negative (FN) values which is shown in Figure 11.

For drones, precision is calculated as the proportion of predicted drones that are correctly identified as drones.

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e})={\displaystyle \frac{\mathrm{T}\mathrm{P}(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e})}{\mathrm{T}\mathrm{P}\left(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e}\right)+\mathrm{F}\mathrm{P}(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e})}}$$

(6)

Recall (for drones) is the proportion of actual drones that are correctly predicted as drones.

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e})={\displaystyle \frac{\mathrm{T}\mathrm{P}(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e})}{\mathrm{T}\mathrm{P}\left(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e}\right)+\mathrm{F}\mathrm{N}(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e})}}$$

(7)

F1 score (for drones) is the harmonic means of precision and recall for drones, providing a balanced measure of detection performance.

$$\mathrm{F}1-\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}\left(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e}\right)=2\times {\displaystyle \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\left(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e}\right)\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\left(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e}\right)}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\left(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e}\right)+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\left(\mathrm{D}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{e}\right)}}$$

(8)

Accuracy measures the overall correctness of the model’s predictions across all classes.

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N}}}$$

(9)

where

• True positives (TP) are cases in which drones are correctly classified as drones.
• False positives (FP) are cases in which non-drone entities are erroneously classified as drones.
• True negatives (TN) are cases in which non-drone entities are accurately classified as other objects.
• False negatives (FN) are cases in which drones are inaccurately classified as other objects.

Average precision (AP) is a crucial performance metric that eliminates the need to select a single confidence threshold value. It is defined by the area under the precision–recall (PR) curve. AP is high when both precision and recall are consistently high across various confidence threshold values and low when either precision or recall is low. The AP value ranges from 0 to 1.

$$AP={\int }_0^{1}P\left(R\right)dR$$

(10)

Mean average precision (mAP) measures how well a drone detection model accurately identifies and locates drones by comparing their predicted positions with the actual positions (ground truth). Higher mAP scores indicate that the model performs better at detecting and recognizing drones.

$$\mathrm{m}\mathrm{A}\mathrm{P}={\displaystyle \frac{1}{\mathrm{C}}}{\sum }_{\mathrm{i}=1}^{\mathrm{c}}\mathrm{A}{\mathrm{P}}_{\mathrm{i}}$$

(11)

where AP_i represents the average precision value for the ith class, and C is the total number of classes under evaluation.

4.2. Experimental Results and Discussion

The trained YOLOv9 models over 100 epochs using a dataset comprising over 9995 images, including training, validation, and testing images. The training duration for the large dataset is 28 h, 8 min, and 28 s. Precision, recall, F1-score, mAP, and FPS were evaluated to know the performance of the YOLOv9 model. The loss of training and validation across epochs is shown in Figure 12, where the downward trend of the loss curve reflects minimized losses.

Figure 13 illustrates the accuracy throughout the training process. Initially, both training and testing accuracy increased rapidly. This is followed by a steady improvement between epochs 7 and 33. After 100 epochs of training, the model achieves 95.14% accuracy on the testing data and 94.46% precision on the training data.

In terms of the detection speed, FPS, and model parameters, 30 FPS or higher [37] is recommended to ensure accurate and timely detection, while 24.12 FPS is achieved by the proposed model along with 253.20 M and 51.6 MB.

Figure 14a,b present the YOLOv9 drone detection model’s output of predicted bounding box offsets. In this figure, x and y are the coordinates of the bounding box centers, and width and height define their size. The color variations in Figure 14 denote the data density, with darker colors showing where data are more concentrated. Most bounding boxes are positioned near the center of the image, with a higher prevalence of medium- and small-sized objects.

YOLOv9 marks a significant advancement in drone detection technology, as illustrated by the performance metrics presented in

Table 3. Performance comparison of YOLOv9 with existing drone detection methods.

Model	Precision	Recall	mAP50	F1-Score
CNN [105]	0.96	0.94	0.95	0.9498
Mask R-CNN [106]	0.936	0.894	0.925	0.9145
YOLOv3 [107]	0.92	0.70	0.785	0.795
YOLOv4 [108]	0.950	0.680	0.7436	0.790
YOLOv5 [109]	0.918	0.875	0.9040	0.896
YOLOv8 [110]	0.91	0.94	0.86	0.85
YOLOv9	0.946	0.864	0.9570	0.9030

. With a precision of 0.946, YOLOv9 outperforms both YOLOv5 (0.918) and YOLOv8 (0.91), demonstrating enhanced accuracy in identifying true positives—an essential factor for effective drone detection. In terms of recall, YOLOv9 achieves 0.864, which is a substantial improvement over YOLOv4 (0.680) and YOLOv3 (0.70). This performance underscores its capability to detect actual drone instances more reliably. While it does fall slightly behind CNN (0.94) and Mask R-CNN (0.894), it still offers a robust detection rate compared to earlier YOLO models. Additionally, YOLOv9’s mean average precision (mAP50) of 0.9570 is the highest among all models assessed, highlighting its superior overall detection capabilities and significant progress in precision across various conditions. The F1 score of 0.9030 indicates a well-balanced performance between precision and recall, making YOLOv9 a dependable choice for real-time applications.

Finally, Figure 15 showcases the overall performance metrics achieved during the training of YOLOv9, while Figure 16, Figure 17 and Figure 18 demonstrate the results of real-time drone detection at different altitudes using Jetson Nano, a low-power edge computing device.

Our study on real-time drone detection indicates that altitude significantly impacts detection accuracy. At higher altitudes, smaller drones may appear less defined, complicating detection efforts. Additionally, fast-moving drones at these heights can suffer from motion blur, which might hinder the model’s ability to accurately predict bounding boxes and class probabilities. Environmental factors, such as atmospheric haze, can also become more pronounced at elevated positions, affecting detection performance.

However, our trained model successfully detects drones under a range of challenging conditions, as illustrated in the figures. Figure 19 highlights the model’s effectiveness in nighttime detection at an altitude of 110 feet, while Figure 20 showcases its ability to identify drones in dark environments with artificial lighting, demonstrating robustness in low-light scenarios. Figure 21 emphasizes the model’s performance in high-altitude situations with poor visibility, further confirming its adaptability. Figure 22 presents successful drone detection in low-visibility cloudy environments, and Figure 23 illustrates its capabilities in complex scenes with multiple objects and dynamic backgrounds. Finally, Figure 24 captures real-time drone detection amid intricate backgrounds, underscoring the model’s reliability in varied lighting and cluttered conditions. Overall, the model’s consistent performance across these diverse scenarios affirms its effectiveness for real-time drone detection applications.

Our YOLOv9-based system, optimized through transfer learning, is capable of detecting drones in diverse lighting conditions, including nighttime scenarios, without relying on thermal cameras or additional sensors. Through training on a comprehensive dataset that encompasses various lighting situations, including low-light and artificial illumination, the model has developed strong feature extraction capabilities that enable it to identify drones in real-world applications and challenging conditions. Additionally, the model incorporates contrast adjustments to improve its ability to differentiate drones from complex, dynamic, and low-visibility backgrounds, making it especially suitable for urban environments where background complexity varies considerably. While the model performs well in many challenging conditions, including cloudy and low-light settings, certain limitations remain when drones are camouflaged within dense urban landscapes or complex backgrounds.

In a subsequent experiment, the system was tested with multiple drones flying simultaneously. The results, illustrated in Figure 25 and Figure 26, demonstrate the successful detection of two and five drones, respectively, across various environmental conditions. This capability highlights the system’s versatility in scenarios involving multiple drones and validates its performance across different scenarios.

Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21, Figure 22, Figure 23, Figure 24, Figure 25 and Figure 26 illustrate the variability in IDs assigned to detected drones throughout the detection process. This optimized YOLOv9 algorithm assigns each drone a unique identifier to facilitate tracking as it moves within the camera’s field of view. While these IDs are useful for tracking purposes, they do not have significant implications for the study. The dynamic nature of these IDs can be attributed to several factors. When multiple drones are detected simultaneously, each is given a distinct ID. However, if a drone temporarily exits the frame and then reappears, the system may interpret it as a new detection and assign a different ID. Variations in altitude and position can also contribute to ID reassignment, as these changes may affect the drone’s appearance from the camera’s perspective. Additionally, environmental factors such as lighting conditions, shadows, and background changes can impact the algorithm’s ability to maintain consistent detection. In some cases, adjustments in the drone’s orientation or size may cause the algorithm to interpret it as a different object, leading to further variability in ID assignments.

5. Conclusions

Drone detection is a very challenging task as they are small in size and shaped like birds with dynamic movements. Radar signatures for drones are also very limited and require expensive hardware setups. Therefore, developing an accurate real-time detection model is very essential to achieving a balance between speed and accuracy. In this paper, YOLOv9 is used as a base model to minimize false detections, with a minimum time span. For the training, we created a new dataset by collecting images from available public resources, as well as our drone flights, to enhance the model’s ability to recognize drones and other objects, even at long distances. Additionally, powerful hardware is employed to ensure that the inference speed meets the requirements for real-time detection. The YOLOv9 model has achieved precision, recall, F1-score, and mAP values of 94.7%, 86.4%, 90.3%, and 95.7%, respectively, using the various datasets, after proper training. Jetson Nano, which is a low-power edge computing device, for real-time drone detection is used for backend computing support. Our updated YOLOv9 model demonstrated enhanced performance over previous versions, with improved recall, F1-score, and mAP, reflecting a 4.6% increase in mAP. Furthermore, the detection speed was tested, reaching a maximum frame rate of 24.12 FPS on the NVIDIA Jetson Nano.

The detector’s performance was evaluated using videos captured at three different altitudes—15 feet, 50 feet, and 110 feet—with lighting conditions to assess its effectiveness across varying heights. As the height increases, confidence score reduces, so we tested our drones around 110 feet in the green zone (the permitted limit to fly in India for drones) and found that the confidence score is more than 60 percent and detected properly. This system will be very helpful for the early and speedy detection of drones in crucial places. This system is designed for the early and rapid detection of drones in critical areas such as airports, military installations, and large public events. It can be deployed on both stationary and mobile platforms to activate quick countermeasures against unauthorized drones. However, adverse weather conditions, including heavy rain, fog, and low light, can affect the system’s performance. Future developments may involve incorporating thermal cameras to enhance detection capabilities in such conditions.

Future work will involve creating a robust and diverse dataset that captures a variety of environmental scenarios, such as distinct weather conditions, lighting variations, and seasonal changes, broadening the model’s generalization capabilities. The dataset will include different types of drones and varied flight patterns, ranging from hovering to high-speed flights, to support more intricate detection scenarios. Furthermore, we may utilize YOLOv10 and YOLOv11 to maximize accuracy, speed, and resilience on edge devices like the NVIDIA Jetson Nano, enhancing the model’s readiness for real-world applications.