Radio Frequency Signal-Based Drone Classification with Frequency Domain Gramian Angular Field and Convolutional Neural Network

Abstract

Over the past few years, drones have been utilized in a wide range of applications. However, the illegal operation of drones may pose a series of security risks to sensitive areas such as airports and military bases. Hence, it is vital to develop an effective method of identifying drones to address the above issues. Existing drone classification methods based on radio frequency (RF) signals have low accuracy or a high computational cost. In this paper, we propose a novel RF signal image representation scheme that incorporates a convolutional neural network (CNN), named the frequency domain Gramian Angular Field with a CNN (FDGAF-CNN), to perform drone classification. Specifically, we first compute the time–frequency spectrum of raw RF signals based on short-time Fourier transform (STFT). Then, the 1D frequency spectrum series is encoded as 2D images using a modified GAF transform. Moreover, to further improve the recognition performance, the images obtained from different channels are fused to serve as the input of a CNN classifier. Finally, numerous experiments were conducted on the two available open-source DroneRF and DroneRFa datasets. The experimental results show that the proposed FDGAF-CNN can achieve a relatively high classification accuracy of 98.72% and 98.67% on the above two datasets, respectively, confirming the effectiveness and generalization ability of the proposed method.

1. Introduction

Unmanned aerial vehicles (UAVs), also known as drones, have been widely used in various military and civilian fields in recent years, such as military reconnaissance, rescue missions, infrastructure inspections, and other applications [1,2]. Although the rapid popularization of drones has promoted socio-economic development, it also brings threats to public security and personal privacy, such as terrorist attacks, drug smuggling, and aerial collision [3]. Therefore, the effective regulation of drones will become an important task in addressing these risks. The accurate detection and classification of drones play a vital role in drone surveillance systems [4].

Existing works in the field of drone detection and classification can mainly be divided into four categories: (1) radar-based detection, (2) vision-based detection, (3) acoustic-based detection, and (4) radio frequency (RF) signal-based detection. Among these methods, the RF signal-based approach is regarded as a promising solution for drone detection and classification in urban environments due to it having several advantages. For example, drone detection based on RF signals can work in all kinds of weather compared to vision-based methods. Additionally, compared with radar- and acoustic-based detection, it is not limited by the size of the radar cross section and has a wider detection range. The general process of drone classification based on RF signals usually involves signal acquisition, feature extraction, and signal classification, with the foremost step being the extraction of the useful features from RF signals emitted by drones [5]. According to the processing domain of feature extraction, relevant research can mainly be divided into three categories: time domain-based feature extraction, transform domain-based feature extraction, and automated feature extraction using deep learning (DL) algorithms.

Several RF signal-based drone detection and classification methods have been proposed. For example, in [6], a drone classification method was introduced by extracting fifteen time domain statistical features from the energy transient of the UAV controller signals. These features were then fed to classifiers for subsequent classification. However, it is not easy to intercept the transient remote control signal of a drone because the duration of this signal is very short [7]. Zhang et al. [8] proposed a UAV recognition algorithm that relies on an artificial neural network utilizing the slope, kurtosis, and skewness of UAV RF signals as time domain features. Based on transform domain analyses, such as a spectrogram, and axially and square integrated bispectra in the frequency domain, are also proposed for drone classification. The authors of [9] proposed extracting RF fingerprints from the fixed boundary empirical wavelet sub-band signals to classify drones. In [10], authors extracted the bispectrum feature from RF signals for drone type classification. Nevertheless, these uncomplicated features may be the same in some similar types of drones [11], especially those from the same manufacturer, restricting the application of such methods.

Lately, various DL models have been proposed to improve drone classification accuracy, as DL models can autonomously extract implicit features from drone RF signals. The authors in [12,13] directly use the in-phase and quadrature (IQ) samples of the raw RF signal to train CNN classifiers. Practically, drones may operate on different channels corresponding to different carrier frequencies, which can result in an unknown carrier frequency offset (CFO) due to the noncooperative relationship between the drone and RF signal receiver. The authors in [14] proposed a CFO estimation and compensation strategy for IQ samples and demonstrated that CFO estimation can enhance the performance of DL-based drone detection systems. In [15], a deep complex-valued convolutional neural network (DC-CNN) was proposed to extract implicit features of drone signal samples for recognizing different types of drones. However, feeding raw data directly to DL models results in high training complexity and low accuracy [16].

To address the above issues, this paper aimed to improve the performance of the RF signal-based drone classification method without increasing the computational cost. A frequency domain Gramian Angular Field (FDGAF) and CNN-based RF recognition technology, named FDGAF-CNN, is proposed to classify different drone signals. Unlike most existing approaches that use time–frequency spectrograms as input images for a classifier, in this paper we introduce a novel signal processing method that preserves the feature information in both the time and frequency domains, while also visualizing the features. The main contributions of this paper are summarized as follows:

(1) To visualize drone RF signals as images, we introduce a novel signal processing method called FDGAF. In the frequency domain, the frequency spectrum of the acquired RF signal is mapped onto a 2D image through spanning polar coordinates by a modified GAF transform. To the best of our knowledge, FDGAF images are used for the first time for drone RF signal representation.

(2) To better preserve the feature information of RF signals, we provide a new perspective for the fusion of RF signals received from different channels without increasing computational complexity. Then, a CNN was constructed to perform the task of classifying the fused images.

(3) To validate the effectiveness of the proposed FDGAF-CNN model, we conducted a comprehensive analysis on two public datasets named DroneRF and DroneRFa. This analysis includes visualization and comparison experiments. The experimental results show that our model performs significantly well on different datasets.

The remainder of this paper is organized as follows. Section 2 presents an overview of related work. Section 3 describes the system model and problem formulation. Section 4 presents the proposed method, including the frequency spectrum calculation, frequency domain GAF transform, and the construction of the CNN. Section 5 presents the experimental results, and Section 6 concludes the entire paper.

2. Related Works

In this section, we mainly present the existing works on RF signal-based methods for drone classification. Nie et al. [17] extracted axially integrated bispectra (AIB), square integrated bispectra (SIB), and a fractal dimension (FD) from the time domain signals of UAVs as an RF fingerprint. These features were then processed using principal component analysis (PCA) and neighborhood component analysis (NCA) algorithms for dimension reduction and finally fed to a machine learning classifier. In [18,19,20], time domain features such as the number of packets, average packet size, and average inter-arrival time were extracted to classify drones. However, this method requires prior knowledge of the adopted communication technology and protocol, which is unavailable in practice since many drones use private protocols. In [21], the cross-correlation properties in the physical layer (PHY) preambles were analyzed to classify drones. However, the amount of computation required for cross-correlation increases with the number of preambles.

Furthermore, with the aid of time–frequency analysis, such as the short-time Fourier transform (STFT) and the wavelet transform, the use of spectral features for drone classification has garnered growing interest among researchers. In [22], the authors converted the received uplink control signals into images (scatter-plots) by using wavelet scattering transform. The obtained images were then used as input for a convolutional neural network (CNN) for training. Nonetheless, the uplink control signal was hardly received as it was intermittent and easily blocked by the obstacles, especially in urban environments. Thus, many works have utilized UAV downlink video transmission signals as a complementary means of detecting and classifying drones. In [23], the minimum variance distortionless response (MVDR) spectra of downlink signals were calculated as features, and a new CNN transformer network was designed for drone recognition. In [24], the 1D RF signals were transformed into 2D signals in the form of a spectrogram, a persistence spectrum, and percentile spectrum images, which were then utilized in the proposed Hybrid Model with Feature Fusion Network (HMFFNet) for drone classification. The authors in [25] converted the received RF signal to a power-based spectrogram which served as input for a specially designed CNN for UAV classification. Overall, these methods achieve classification based on CNN, which is only capable of learning local features.

Recently, with the rapid exploration of advanced DL theory, many efficient DL architectures have been proposed for drone RF signal classification. For instance, Thien et al. [26] proposed a high-performance CNN called UAVNet for RF-based drone surveillance systems. The model was designed with grouped one-dimensional convolution layers and a multi-level skip connection structure and achieved a classification accuracy of 98.5. In [27], a multi-channel convolutional network (MC-CNN) was developed by alternately constructing 1D convolutional layers and max pooling layers. In [28], the authors proposed a detection and classification framework that combined YOLO architecture to identify drone signals. In [29], Swinney et al. extracted power spectral density (PSD), a spectrogram, a histogram and raw IQ constellation as features to a ResNet50 network for drone classification. However, the existing CNN-based drone RF signal classification methods have significant limitations: (i) an inability to achieve a superior performance on different datasets, (ii) inefficiency in directly processing the raw RF signals received from various channels, and (iii) high memory consumption and computational complexity. Therefore, it is critical to investigate more representative features to enhance the identification rate.

To address the above issues, this study aimed to improve the performance of RF signal-based drone classification. We propose a novel signal processing method called FDGAF for drone RF signal representation and visualization, which preserves the feature information in both time and frequency domains. Then, the obtained FDGAF images from different channels are fused by taking the arithmetic average of the corresponding pixels of the image. Such an operation can amplify the differences between different RF signals. Finally, a CNN is designed for drone type classification.

3. System Model and Problem Statement

In this section, the RF signal-based drone-sensing system used in this study is delineated, and the problem statement of RF signal classification is also illustrated.

3.1. System Model

The overall framework of the proposed drone classification system, named FDGAF-CNN, is illustrated in Figure 1. It comprises four primary modules: a data collection module, a frequency spectrum calculation module, an FDGAF image generation module and a CNN classification module. The RF signals emitted by different types of drones are first captured by the RF-sensing devices. Then, the frequency spectrum of the captured raw RF signal is calculated based on STFT. Next, the 1D frequency spectrum series is converted into 2D images through FDGAF transform. Finally, the obtained images are used for CNN training and testing.

3.2. Problem Statement

The discrete-time received signal samples at time instant n captured by the sensing device can be expressed as

r(n) = s(n) + w(n), n = 0, 1, …, N − 1

(1)

where s(n) is the drone signal, w(n) denotes environment noise, and N is the length of the received signal. For the received signals, r, the goal of a drone classification model training task is to maximize the probability of accurately classifying the drone type through adjusting the network parameters. Assume that a drone RF signal candidate set is given by $\mathbf{Y}\triangleq \{y_1, y_2, \dots , y_J\}$, where y_j denotes the j-th type of the RF signal, j = 1, 2, …, J. Thus, the mathematical model of the above task can be expressed as

$$\underset{\theta }{\arg \max }P(g_{\theta }(r_i)[y_1, y_2, \dots , y_J]{|y}_{true}=[y_1, y_2, \dots , y_J])$$

(2)

where $g(\cdot ,\cdot )$ denotes the neural network and θ represents the network parameters.

4. Proposed Method

In this section, the details of the proposed FDGAF-CNN method are introduced. There are four main components of this method: frequency spectrum calculation, frequency domain-modified GAF for spectrum representation, multi-FDGAF image fusion, and the CNN design.

4.1. Frequency Spectrum Calculation

As each drone RF signal record contains many sample points, directly using the original RF signal for GAF transform results in the size of the generated images being very large. Hence, to reduce the size of image while preserving the frequency domain features of the RF signal, this paper introduces a time–frequency spectrum calculation scheme based on STFT.

Firstly, we perform STFT on the captured drone RF signals, represented by X[k, m] as follows

$$X[k, m]={\displaystyle \sum _{n=0}^{N-1}r[n]}h(n-m)e^{-j{\displaystyle \frac{2\pi }{N}}nk}, k=1, 2, \dots , K, m=1, 2, \dots , M$$

(3)

where h(n) is the window function. K and M are the number of frequency bins and time bins, respectively. $X[k, m]$ describes the frequency change of signal r[n] over time m. The STFT spectrum, denoted as SP[k, m], is defined as the square of the absolute magnitude of $X[k, m]$ [30].

$$SP[k, m]={\left|X[k, m]\right|}^2={\left|{\displaystyle \sum _{n=0}^{N-1}r[n]}h(n-m)e^{-j{\displaystyle \frac{2\pi }{N}}nk}\right|}^2$$

(4)

The STFT spectrum SP[k, m] is a matrix with K rows and M columns. By adding up the elements of each row of SP[k, m], we can obtain the frequency distribution of r[n]-like discrete Fourier transform, which can be expressed as

$$F(k)={\displaystyle \sum _{m=1}^{M}SP[k, m]}={\displaystyle \sum _{m=1}^M{\left|{\displaystyle \sum _{n=0}^{N-1}r[n]}h(n-m)e^{-j\frac{2\pi }{N}nk}\right|}^2}$$

(5)

As an example, we analyze a segment of RF signal from an AR drone in the DroneRF dataset. A detailed description of this dataset is presented in Section 5.1. Figure 2 shows the frequency spectrum obtained by using fast Fourier transform (FFT) and STFT, where the signal length N = 512, and the Hanning window with a 128 overlap is adopted for STFT. It can be seen from the figures that STFT is less sensitive to noise than FFT. Compared with Figure 2a, the frequency distribution curve in Figure 2b is smoother and shows better continuity.

4.2. Frequency Domain-Modified Gramian Angular Field Transform

To extract effective features from the 1D frequency spectrum, we convert the 1D frequency spectrum into 2D images before identification. Here, a novel image generation method named FDGAF is introduced, which preserves the frequency features of the original frequency spectrum and makes the features visualizable.

The traditional Gramian Angular Field (GAF) transformation was first proposed by Wang and Oates in 2015 [31]. Through GAF, the 1D time series can be encoded as 2D images with two main steps: polar coordinate transformation and GAF transformation. Notice that, unlike the traditional amplitude/phase polar coordinates, GAF uses the time step as the radius and the arccosine of the scaled amplitude as the angle [32]. For the proposed FDGAF method, the steps of converting the original frequency spectrum series into a 2D image are given as follows.

Step 1: To eliminate the adverse effect caused by singular samples, rescale the 1D frequency spectrum data, termed as $\mathbf{F}=[F(1), F(2), \dots , F(K)]$, to the interval [−1, 1] or [0, 1] by

$$\tilde{F}(k)={\displaystyle \frac{[F(k)-\max (\mathbf{F}\left)\right] + [F(k)-\min (\mathbf{F}\left)\right]}{\max (\mathbf{F}) - \min (\mathbf{F})}}, \tilde{F}(k)\in [-1, 1]$$

(6)

$$\mathrm{or} \tilde{F}(k)={\displaystyle \frac{F(k)-\min (\mathbf{F})}{\max (\mathbf{F})-\min (\mathbf{F})}}, \tilde{F}(k)\in [0, 1]$$

(7)

Step 2: The obtained frequency sequence is converted into a polar coordinate system. Here, the value of the sequence data is used as the angular cosine and the corresponding time stamp is used as the radius, which can be expressed as

$$\{\begin{array}{l}{\theta }_k=\arccos [\tilde{F}(k)], \tilde{F}(k)\in \tilde{\mathbf{F}}\\ r_k={\displaystyle \frac{k}{K}}, k\in K\end{array}$$

(8)

where k is the time stamp, K is the sequence data length, and ${\theta }_k$ denotes the arccosine value of the sample F(k).

Step 3: With (8), the signal sequence can be converted into the polar coordinate system. The time information of the frequency spectrum sequence can be maintained by the radius r_k, and θ_k retains the amplitude characteristic of the original frequency spectrum data.

Step 4: Next, we construct the Gramian Angular Summary Field (GASF) or Gramian Angular Difference Field (GADF). The formulas are given as follows:

$$GASF=[\cos ({\theta }_i+{\theta }_j)]={\tilde{\mathbf{F}}}^{\mathrm{T}}·\tilde{\mathbf{F}}-{\sqrt{\mathbf{I}-{\tilde{\mathbf{F}}}^2}}^{\mathrm{T}}·\sqrt{\mathbf{I}-{\tilde{\mathbf{F}}}^2}$$

(9)

$$GADF=[\cos ({\theta }_i-{\theta }_j)]={\sqrt{\mathbf{I}-{\tilde{\mathbf{F}}}^2}}^{\mathrm{T}}·\tilde{\mathbf{F}}-{\tilde{\mathbf{F}}}^{\mathrm{T}}·\sqrt{\mathbf{I}-{\tilde{\mathbf{F}}}^2}$$

(10)

where I denotes the unit row vector $[1, 1, \dots , 1]$, and ${\tilde{\mathbf{F}}}^{\mathrm{T}}$ is the transpose vector of $\tilde{\mathbf{F}}$.

Through (9) and (10), the inner products reflect the correlation of two vectors. According to such a characteristic, the GASF matrix and GADF matrix reflect the correlation between the angle sum and angle difference, respectively. The correlation between the sum and the difference can be used to jointly express the correlation between the sequence values at different time instants.

Unlike the inner definitions in traditional GASFs and GADFs, we introduce in this paper a new inner product proposed in [32] to generate a GAF matrix. The new inner product is defined as [33]

$$<x,y> =x·y+\sqrt{1-x^2}·\sqrt{1-y^2}$$

(11)

In (11), the 0/1 scaling mode is chosen as the rescale formula. Based on Equation (11), the FDGAF is defined as

$$\begin{array}{l}FDGAF=\left[\begin{array}{lll}\cos ({\theta }_1-{\theta }_1)& \cdots & \cos ({\theta }_1-{\theta }_K)\\ \cos ({\theta }_2-{\theta }_1)& \cdots & \cos ({\theta }_2-{\theta }_K)\\ ⋮& \ddots & ⋮\\ \cos ({\theta }_K-{\theta }_1)& \cdots & \cos ({\theta }_K-{\theta }_K)\end{array}\right]+\left[\begin{array}{llll}\cos {\theta }_1& 0 & \cdots & 0\\ 0& \cos {\theta }_2 & \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0 & \cdots & \cos {\theta }_K\end{array}\right]\\ ={\tilde{\mathbf{F}}}^{\mathrm{T}}·\tilde{\mathbf{F}}-{\sqrt{\mathbf{I}-{\tilde{\mathbf{F}}}^2}}^{\mathrm{T}}·\sqrt{\mathbf{I}-{\tilde{\mathbf{F}}}^2}+\mathrm{diag}(\tilde{F})\end{array}$$

(12)

where the first item on the right-hand side contains differential information between multiple time intervals, and the second item on the right-hand side is a diagonal matrix used for recovering the original values based on its main diagonal elements.

The computational complexity of calculating the FDGAF image is mainly determined by the number of samples. When the length of the ID frequency spectrum series is K, there are K times of inverse cosine operation, K² times of cosine operation, and K times of addition. Therefore, a matrix of the size of [K, K] is obtained after FDGAF transformation. For M preprocessed samples, the complexity of the FDGAF is O(M K²). In practical applications, to reduce the computational load and memory space, we can divide the original data into segments to reduce the size of K. On the other hand, the mobile edge computing devices can be introduced to share the computing load of sensing nodes.

4.3. Multi-FDGAF Image Fusion

To preserve more features, this paper fuses the FDGAF images that were generated from the I/Q components of the received RF signals. Moreover, in the open-source DroneRF dataset, the signals of each type of drone are collected from both high and low spectrum bands. The proposed fusion scheme is also suitable for such a scenario. Let FDGAF_I and FDGAF_Q denote the images generated from I and Q channels, respectively. The fused image FDGAF is calculated by taking the arithmetic average of the FDGAF_I image and FDGAF_Q image, which can be expressed as

$$FDGAF={\displaystyle \frac{1}{2}}(FDG{A}_I+FDG{A}_Q)$$

(13)

Through the above formula, the frequency domain GAF transformation produces a unique inverse map in the polar coordinate system, constituting the feature image dataset. Moreover, the additive operation can make the relationship between pixels in the FDGAF denser, which is more helpful for image classification. Figure 3 shows the process of converting a 1D signal to an FDGAF image.

To illustrate the effectiveness and superiority of our proposed scheme for drone RF signal representation, as an example, we convert the RF signals from Bebop, Phantom, and the AR drone in the DroneRF dataset into 2D FDGAF images and compare them to time–frequency spectrograms, where the spectrograms are calculated by low-spectrum-band RF signals. For frequency spectrum calculation, the length of FFT is set as M = 1024, and a Hanning window with a 128 overlap is adopted; thus, the length of the frequency spectrum series K = 513. Therefore, an FDGAF image with a size of $513 \times 513$ can be obtained. The results are shown in Figure 4.

In Figure 4a,b, we can observe that the spectrograms show some similar characteristics, whereas the FDGAF images of three different types of drones are quite distinct after GAF transformation. This distinction is helpful for subsequent classification tasks.

4.4. CNN Design

CNNs have been widely used in image identification due to their excellent recognition performance. Thus, a basic CNN is used in this paper. The basic CNN architecture includes convolutional layers, pooling layers, and fully connected (FC) layers [34]. Among these, the convolutional layer is the most important CNN unit, which contains a number of different kernels with the same size. These kernels perform convolution operation across the input signal to extract features. The pooling layer is used to achieve the secondary goal of extraction and filtering features, as well as reducing the computational burden. The pooling layer does not contain weights compared to convolution layers. After convolution and pooling layers, the features from them are fed into FC layers for classification. In addition, the FC layer uses the softmax activation function to calculate the probability that the classifier input z belongs to class j, j = 1, 2, …, J. The mathematical expression is [35]

$$p_j(z)={\displaystyle \frac{\exp ({\theta }_j^{T}z)}{{\displaystyle {\sum }_{h=1}^J\exp ({\theta }_h^{T}z)}}}$$

(14)

where $\theta$ is the classifier parameter.

The CNN model adopted in this paper is shown in Figure 5. It consists of three convolutional layers, two pooling layers, and one FC layer. Each convolutional layer is followed by a batch normalization layer and the rectified linear unit (ReLU) activation function. The first convolution layer has 12 $3 \times 3$ kernels, and the second and third convolution layers have 24 and 48 kernels of the same size, $3 \times 3$, respectively. For the pooling layers, the number of filters and the size of filters are marked in Figure 5. After the FC layer, softmax is used to output the probabilities for different drones. Finally, we resize the inputs of the FDGAF images to $227 \times 227$ and use the error back-propagation algorithm to train the CNN.

5. Experimental Results

In this section, first, two public drone RF datasets are introduced in detail, and then the visualization results of the drone RF signal are shown. Finally, the classification performances of the proposed FDGAF-CNN and the comparative analysis are described.

5.1. Datasets

(1) DroneRF dataset: This dataset is an open-source dataset, which was first created by Allahham et al. (2019) [36]. The dataset contains 454 drone RF signal records, which were collected from RF background actives without drones and three types of drones, namely Bebop, AR, and Phantom [36], operating in four different modes: on and connected to the controller, hovering, flying without video recording, and flying with video recording. Among the total records, there are 227 low-frequency-band and 227 high-frequency-band RF records. These were received by two 40 MHz bandwidth receivers operating in the 2400–2440 MHz band and 2440–2480 MHz band, respectively. Each drone record was collocated for 5.25 s, while the background noise was collected for 10.25 s. Figure 6 shows the time domain samples of the dataset, where the drones are on and connected to the controller mode.

(2) DroneRFa dataset: This dataset includes 9 categories of moving drone RF signals collected in urban outdoor scenarios, 15 categories of drone RF signals collected in urban indoor scenarios, and 1 category of background reference signals [37]. Each category of the data consists of no fewer than 12 segments, with each segment containing over 100 million sampling points. The data are stored in “.mat” format files, with each file containing data from the I and Q channels of the RF receiver. As an example, Figure 7 shows the time domain samples of two types of drones, where the plotted data are derived from the first 5,000,000 sample points (with a sampling rate of 100 MS/s) of “T0001_D00_S0111.mat” and “T1001_D00_S0110.mat” from DroneRFa [37].

5.2. Visualization of Drone RF Signal

In this subsection, to demonstrate the superiority of the proposed scheme for drone RF signal representation, we visualize the RF signal of the DroneRFa dataset through FDGAF transformation. As an example, we use the first 1,000,000 sample points as a data segment (lasting 0.01 s) for image generation. This data segment comes from six types of drones: the Phantom 3, Phantom 4 Pro, Air 2S, Mini 2, MATRICE 30T, and MATRICE 200. For frequency spectrum calculation, the length of FFT is set as M = 1024, and a Hanning window with a 128 overlap is adopted. Thus, the length of the frequency spectrum series K = 513. Therefore, an FDGAF image with a size of $513 \times 513$ can be obtained. Figure 8 shows the FDGAF images of the RF signals of the abovementioned six types of drones. From the figures, we can see that the obtained FDGAF images have significant differences among drones.

5.3. Results of Proposed FDGAF-CNN for Drone Classification

All the experiments were performed on a computer with 2.3 GHz CPU, 16 GB RAM, and a GeForce RTX 2050Ti. All algorithms were implemented using MATLAB^® 2023b software on a Windows 11 operating system platform.

To reduce the computational cost while ensuring the integrity of the signal frame, each record in the DroneRF dataset is split into ten segments, corresponding to the time-duration of 25 ms. For the DroneRFa dataset, according to [37], every 1 million data points (lasting 10 ms) are considered as one data slice for FDGAF image generation. The other parameters are set the same as in Figure 8. Moreover, the base learning rate is 0.0001, the maximum number of training epochs is 5, and the mini-batch size at each iteration is 32. The obtained image dataset is randomly divided into separate 70% training, 10% validation, and 20% testing sets to verify the recognition performance of FDGAF-CNN.

Figure 9 shows the learning curves in the training process of the CNN model for two datasets. In Figure 9a,c, after 40 iterations, the FDGAF-CNN model in DroneRF converges to the steady-state values, with a level of 95% training accuracy. For DroneRFa, the accuracy and the loss curves converge to the steady-state values when the number of iterations is greater than 100. It can be seen from Figure 9 that the FDGAF-CNN model can converge well, regardless of the dataset type.

To exhibit the classification performance of FDGAF-CNN more comprehensively, we visualize the confusion matrix of FDGAF-CNN under two datasets. The related confusion matrix is shown in Figure 10. It can be seen from Figure 10a that the classification accuracy of Phantom and the background (no UAV) is 100%. The classification accuracy of AR is 99.1%, where 0.9% of AR is misjudged as no UAV. In Figure 10b, the misclassification mainly occurs between (Futaba T6IZ, Futaba T14 SG) and (Air 2S, Mini2). But for other drones, FDGAF-CNN shows a superior recognition performance.

To demonstrate the effectiveness of the proposed method, we compare the classification accuracy of FDGAF images generated from one channel and those generated from two channels in the aforementioned datasets. In the DroneRF dataset, these images are generated from low-spectrum-band RF signals (FDGAF_L), high-spectrum-band RF signals (FDGAF_H), and a fusion of both low- and high-spectrum-band RF signals (FDGAF). In the DroneRFa dataset, these images are generated from the I channel RF signals (FDGAF_I), the Q channel RF signals (FDGAF_Q), and a fusion of both the I and Q channel RF signals (FDGAF).

Table 1. Classification accuracy of different input images for the DroneRF and DroneRFa datasets.

Dataset	Input Image	Accuracy (%)
DroneRF	FDGAFL	97.67
	FDGAFH	94.85
	FDGAF	98.72
DroneRFa	FDGAFI	97.94
	FDGAFQ	94.60
	FDGAF	98.67

presents the classification accuracy.

As can be seen in Table 1, the best classification accuracy is achieved by using FDGAF images for both DroneRF and DroneRFa, with success rates of 98.72% and 98.67%, respectively. Furthermore, compared to FDGAF_H, the FDGAF shows an improvement of 3.87% for DroneRF and an improvement of 4.27% over FDGAF_Q for DroneRFa. The reason for this is that the fused images (FDGAF) preserve more feature information, which contributes more to performance enhancement than images generated from single-channel RF signals.

5.4. Classification Accuracy Comparison of Different Methods

In this subsection, we compare FDGAF-CNN with other state-of-art drone classification methods that use the DroneRF dataset for training and testing, namely BISSIAM [10] HMFFNet [24], PSD-DNN [38], RF-CNN [39], MFCC-SVM [40], and LFCC-SVM [40]. Figure 11 illustrates the classification accuracy of these methods.

From Figure 11, we can see that the classification accuracy of the proposed FDGAF-CNN model is higher than that of the other methods, except for HMFFNet. In HMFFNet, a spectrogram, persistence spectrum, and percentile spectrogram are concatenated as the classifier input, which increases the computational cost at the model training stage. Moreover, the CNN structure used in this paper is simple, resulting in fewer parameters that need to be optimized.

6. Conclusions

In this paper, we propose a novel drone classification method based on a frequency domain GAF and CNN called FDGAF-CNN through RF signals. First, the frequency spectrum of the acquired RF signal is calculated using STFT. Then, we utilize a modified GAF to transform the frequency spectrum to images, converting the sequence classification task into an image classification task. To improve the classification performance, the FDGAF images obtained from different channel signals are fused as the input to a CNN classifier. To the best of our knowledge, this is the first time that an FDGAF has been developed for drone recognition. Finally, the effectiveness and generalization ability of the method has been verified through two public datasets. Experimental results show that the classification accuracy of FDGAF-CNN for identifying DroneRF is 98.72%, and for DroneRFa, it is 98.67%. In future work, we will evaluate the performance of the proposed model in real wireless environments.