Multimodal Few-Shot Target Detection Based on Uncertainty Analysis in Time-Series Images

Abstract

The ability to interpret multimodal data, and map the targets and anomalies within, is important for an automatic recognition system. Due to the expensive and time-consuming nature of multimodal time-series data annotation in the training stage, multimodal time-series image understanding, from drone and quadruped mobile robot platforms, is a challenging task for remote sensing and photogrammetry. In this regard, robust methods must be computationally low-cost, due to the limited data on aerial and ground-based platforms, yet accurate enough to meet certainty measures. In this study, a few-shot learning architecture, based on a squeeze-and-attention structure, is proposed for multimodal target detection, using time-series images from the drone and quadruped robot platforms with a small training dataset. To build robust algorithms in target detection, a squeeze-and-attention structure has been developed from multimodal time-series images from limited training data as an optimized method. The proposed architecture was validated on three datasets with multiple modalities (e.g., red-green-blue, color-infrared, and thermal), achieving competitive results.

1. Introduction

In large-scale photogrammetry and remote sensing observations (e.g., drone imaging), visual target detection is a difficult issue in multimodal images due to the spectral similarities between the target and the background [1]. In real-world applications, the variety of geospatial information from urban areas can cause difficulties when creating and analyzing datasets, since it is hard to find the right method that matches their learning preferences.

Drone imaging is a good solution for static and dynamic target detection from different objects on an urban scale [2]. Drone and quadruped mobile robots equipped with multimodal sensors (e.g., RGB, color-infrared, and thermal) are an efficient and low-cost method for high-resolution scene understanding in real-world scenarios based on time-series images [3,4,5] such as crop and weed monitoring, traffic and vehicle management, and search and rescue missions for crisis management. In this regard, target detection is an important task in scene understanding from time-series images [6,7,8]. Finding an optimized algorithm with a small training dataset is the key challenge for target prediction and its real-world applications.

In drone imaging, a new encoder-decoder deep learning approach based on Fully Convolutional Network (FCN), Dilated Convolutional Neural Network (DCNN), U-Net, and MultiScaleDilation is proposed for multitarget prediction from oblique time-series images [9]. Moreover, a multitask learning method based on an encoder-decoder model is proposed for vehicle and building detection from multimodal drone images [10]. Gao et al. (2021) proposed a few-shot detector based on fully convolutional one-stage object detection (FCOS) for vehicle, storage tank, and plane detection in drone images [11]. Moreover, a new plant location and counting technology, based on a few-shot learning method that uses RGB images acquired from unmanned aerial vehicles (also known as CenterNet), has been proposed. The quantitative assessments of this study showed that the average precision for a modified CenterNet architecture is about 95% [12]. A multi-stream framework and a deep multimodality learning method has been designed for drone video aesthetic quality assessment from spatial appearance, drone camera motion, and scene structure [13]. Lu and Koniusz (2022) proposed a versatile Few-Shot Keypoint Detection (FSKD) pipeline for unseen species based on uncertainty learning from RGB images [14]. In quadruped mobile robot imaging, an efficient study was proposed based on ERFNet, MAVNet, U-Net, Fast-SCNN, MFNet, and RTFNet from multimodal time-series images for fire-extinguishers, backpacks, hand-drills, and survivors [15]. The encoder block in CNNs for time-series images, represented by an encoding function $y = f\left(x\right)$, compresses the time-series input into a latent space, and the decoder block, $z = h\left(y\right)$, aims to predict the target from the latent space. Unal (2021) proposed a new visual target detection method based on the Kalman filter and deep learning for human tracking from RGB street-level images. In this study, the Kalman filter was used to estimate the position, velocity, and acceleration of a maneuvering target [16]. In Kiyak and Unal (2021), four deep learning models (i.e., deep convolutional neural networks (DCNN), deep convolutional neural networks with fine-tuning (DCNNFN), transfer learning with deep convolutional neural network (TLDCNN), and fine-tuning deep convolutional neural network with transfer learning (FNDCNNTL)) were proposed to detect small aircraft [17]. In Han et al. (2022), a few-shot learning method (also known as FSOD) was applied to object detection from RGB street-level images based on meta-learning with cross-modal representations [5]. To achieve a stable target detection accuracy regardless of the sensing devices’ viewpoint and orientation (drone or quadruped mobile robot), a multimodal few-shot learning method can be utilized [18,19,20,21,22]. According to the related literature, several investigations have demonstrated that there are still some crucial problems, such as: (1) poor model performance due to the lack of pixel-wise training in few-shot learning models in real-world applications, which have not yet been well considered in the relevant studies; and (2) poor generalization ability in different platforms due to the use of single-modal data for training.

In this study, a few-shot machine learning architecture based on squeeze-and-attention structure [23] is proposed for multimodal target detection based on time-series images from drone and quadruped mobile robot platforms with a small training dataset. To build robust algorithms in target detection, a squeeze-and-attention structure has been developed from multimodal time-series data from limited training data as an optimized method. We aim to fill some of the gaps in the supervised method with small training data in the field of target detection of multimodal time-series data from drone and quadruped mobile robot platforms. We summarize our contributions as follows:

• A few-shot learning model based on an uncertainty analysis of feature extraction is proposed with an encoder-decoder structure and squeeze-and-attention module. The proposed model is composed of two components in the encoder block, including the residual representation extraction and the attention layers;
• It proposes a novel approach to extracting inherent and latent representation from multimodal images;
• We conducted several multimodal datasets for different real-world scenarios to investigate the behavior of the proposed few-shot learning method.

2. Materials and Methods

2.1. Definition of the Problem

Despite recent breakthroughs in drone imaging and few-shot learning, confident and robust target detection remains a challenge for remote sensing engineers [20,24]. The purpose of static and dynamic target detection from drone images is to locate the target on large-scale images by using pixel-wise segmentation [25]. Figure 1 shows a comparison of oblique and vertical views from drone and quadruped mobile robot platforms. In Figure 1, targets at different distances need to be processed in different scale spaces and spatial-spectral representations. Due to the expensive and time-consuming nature of multimodal time-series data annotation in the training stage, multimodal time-series image understanding from drone and quadruped mobile robot platforms is a challenging task for remote sensing and photogrammetry [26]. In this regard, robust methods must be computationally low-cost, due to the limited data on aerial and ground-based platforms, yet accurate enough to meet certainty measures.

Target localization with the use of uncertainty modeling in few-shot learning for drone images may improve scene understanding from a limited training dataset, while many target detection methods appear to understand single-time localization with large amounts of training data [27]. Few-shot learning can perform unseen tasks after training on a few labeled images and can consider several tasks to produce a predictive function; it is also an inductive transfer system, whose main purpose is to improve generalization ability for multiple tasks [21]. Target localization of the trained network can be assumed as accurate, but not for decision-making in real-world applications [28,29,30]. For example, timely weed detection in farming tasks is crucial to obtain high-quality crops. In this regard, the uncertainty estimation map should be a crucial stage of the predictive map.

In recent years, the number of studies on the civil applications of few-shot learning in image processing has rapidly increased. A general few-shot learning approach is a pre-trained convolutional neural network [31] which is trained on a large dataset and fine-tuned on another task. A deep network trained on the iSAID dataset [32] can be used in another instance segmentation, since the iSAID contains over 655,451 object instances. This approach is known as transfer learning, and it requires a large training dataset and parameters in order to fine-tune [33]. Transfer learning includes the transfer of knowledge from a preliminary task to a newer task that has many similarities with the old one [34]. In meta-learning [35], in order to learn from a limited amount of training data with annotated images, a new strategy called few-shot learning was proposed. In this regard, few-shot learning models were developed, and they have since proven to be robust methods which use only limited training data.

A model-agnostic meta-learning approach has been developed for image segmentation [36]. The model-agnostic meta-learning extends gradient descent by optimizing for a model initialization that leads to good performance on a set of related tasks. A gated encoder-decoder convolutional neural network has been proposed for pixel-wise weed detection from multispectral drone images [37]. Although these related models are fairly powerful for some target localization tasks, their performance in confidential static and dynamic target detection is, to the best of our knowledge, still not excellent.

2.2. Proposed Few-Shot Learning Network

A graphical summary of the proposed few-shot learning network for robust target localization is presented in Figure 2. The proposed model contains the following three sets, where each set contains $M$ images: a training set $D_{train} = {\left\{x_i,y_i\right\}}_{i = 1}^M, x_i:D\to {ℝ}^3$, an input image, and $y_i:D\to \left\{0,1\right\}$, its corresponding ground truth map; a support set $D_{support} = {\left\{x_i,y_i\right\}}_{i = 1}^{M_{support}}$; and a test set $D_{test} = {\left\{x_i\right\}}_{i = 1}^{M_{test}}$. The proposed model takes the DMF-Net [37], the SA-Net [38], and the MF-Net [26] as the main backbone model for weed detection, vehicle monitoring, and human detection.

In this study, we designed and trained DMF-Net, SA-Net, and MF-Net for learning augmentation and for improving the generalization ability of the trained model. DMF-Net is a lightweight gated convolutional neural network model with a model size of 70 MB. This model is trained using a lightweight dataset that includes 150 samples of multispectral drone images with a size of 480 × 360 pixels. The gated CNN was efficient at minimizing the unnecessary transmission of data by using a convolutional layer for extracting optimized single-date or time-series images. SA-Net and MF-Net are few-shot learning models based on the region-based temporal aggregation Monte Carlo dropout [39], which can further improve the uncertainty modeling to guide vehicle and human monitoring.

Our model is different from similar networks in multiscale feature extraction likewise based on convolutional neural networks, since it overcomes the constraints of convolutional blocks and binary mask generation for static and dynamic target localization. The proposed method utilizes a pair of encoders and decoders. The encoder is composed of four squeeze-and-attention modules and the decoder is composed of one transpose convolution module and interpolation, which learns non-local spatial-spectral features. To aggregate multistep non-local representations, we adopted four squeeze-and-attention modules on the multistep outputs of the backbone model, resulting in better target boundaries. In the proposed method, we focus on region-based temporal aggregation Monte Carlo dropout as the uncertainty estimator for target localization. We propose an encoder-decoder network based on a new squeeze-and-attention block $B_n$, and a transposed coevolution $T_D$ for target detection from time-series images with different platforms for few-shot learning (Figure 2). The convolutional layers include 5 × 5 kernels, which are applied to the input representation maps using stride one. The proposed few-shot learning network is composed of 10 squeeze-and-attention layers followed by batch normalization (Bt) and rectified linear unit (ReL) functions to generate feature maps, as well as max-pooling layers to reduce the size of feature maps (

Table 1. The proposed architecture for multimodal target detection. n@C × R: number of channels, columns, and rows.

Encoder	Components	Size	Stride	Result
Input	-	-	-	n@CInput × RInput
${\mathrm{B}}_1$	CV1 + Bt + ReL	64@5 × 5	1	64@C × R
	CV2 + Bt + ReL + B	64@5 × 5	1	64@C × R
	MP	2 × 2	2	-
${\mathrm{B}}_2$	CV3 + Bt + ReL	128@5 × 5	1	128@C × R
	CV4 + Bt + ReL + B	128@5 × 5	1	128@C × R
	MP	2 × 2	2	-
${\mathrm{B}}_3$	CV5 + Bt + ReL	256@5 × 5	1	256@C × R
	CV6 + Bt + ReL	256@5 × 5	1	256@C × R
	CV7 + Bt + ReL + B	128@5 × 5	1	256@C × R
	MP	2 × 2	2	-
${\mathrm{B}}_4$	CV8 + Bt + ReL	512@5 × 5	1	512@C × R
	CV9 + Bt + ReL	512@5 × 5	1	512@C × R
	CV10 + Bt + ReL + B	512@5 × 5	1	512@C × R
	MP	2 × 2	2	-
Decoder
${\mathrm{C}}_1$	UP	2 × 2	2	-
	DCV1 + Bt + ReL	1024@5 × 5	1	1024@C × R
	DCV2 + Bt + ReL	1024@5 × 5	1	1024@C × R
	DCV3 + Bt + ReL	512@5 × 5	1	512@C × R
${\mathrm{C}}_2$	UP	2 × 2	2	-
	DCV4 + Bt + ReL	512@5 × 5	1	512@C × R
	DCV5 + Bt + ReL	512@5 × 5	1	512@C × R
	DCV6 + Bt + ReL	256@5 × 5	1	256@C × R
${\mathrm{C}}_3$	UP	2 × 2	2	-
	DCV7 + Bt + ReeL	256@5 × 5	1	256@C × R
	DCV8 + Bt + RL	256@5 × 5	1	256@C × R
	DCV9 + Bt + ReL	128@5 × 5	1	128@C × R
${\mathrm{C}}_4$	UP	2 × 2 128@5 × 5 64@5 × 5	2	-
	DCV10 + Bt + ReL		1	128@C × R
	DCV11 + Bt + ReL		1	64@C × R
Output	UP	2 × 2	2	-
	DCV12 + Bt + ReL	64@5 × 5	1	64@C × R
	DCV13 + Bt + ReL	2@5 × 5	1	2@C × R
	Loss	-	-	-

).

In action, we define a training dataset $T_D$ employed for the training step, a test dataset $Q_D$, and a support set $S_D$ employed for the testing step.

We summarize the proposed network’s components as follows:

Gated Module. This module is based on a guided feature extractor, feature fusion layer, dilation CNN, inception layer, and encoder-decoder blocks. The gated module extracts the object-level representation from time-series images from multimodal data. We used the DMF $G_i = \left[Bt\left(L\otimes k_{3 \times 3}\right)\left]\odot \right[UP\left(U\otimes k_{3 \times 3}\right)\right]$ as a gated module [37];

Squeeze-and-Attention Layer. To aggregate multi-scale non-local representations, we adopt squeeze-and-attention layers $B = U(RL\left(f_{att}\left(Pool\left(I\right)\right)\right) \times x_{res} + U(ReLU\left(f_{att}\left(Pool\left(I\right)\right)\right)$ on the multi-scale outputs of the gated module model, resulting in better target boundaries;

Weighted Binary Cross-Entropy. This loss function $E\left(cp,\widehat{cp}\right) = -\left(\gamma \log \left(\widehat{cp}\right) + \left(1 - cp\right)\log \left(1 - \widehat{cp}\right)\right)$ is used for the issue of the target imbalance in which all positive pixels get weighted by an amount close to one;

System Implementation Details. The proposed method was trained, using PyTorch, on a single NVIDIA TESLA K80 with a batch size of 12 for 150 epochs for target prediction. The learning rate and momentum were about ${10}^{-2}$ and 0.9, respectively, for the stochastic gradient descent approach;

Backbones. To train the proposed network from three multimodal datasets for target detection, two backbones were selected. The experiments were carried out with DMF-Net, SA-Net, and MF-Net backbones.

2.3. Datasets

The experimental areas include oblique and vertical images from drone and quadruped mobile robot platforms.

Table 2. Details of datasets for multimodal target prediction.

Dataset	Modality	Patch Size (Pixels)	Samples
			Train	Support	Test
UAVid	RGB	3840 × 2160	500	55	45
WeedMap	CIR	480 × 360	320	100	60
PST900	Thermal	1280 × 720	190	100	60

shows the details of multimodal datasets for target detection.

In this study, three datasets from different scenarios (

Table 3. List of datasets for target detection.

Dataset	Type	View	Data Source	Target Class
UAVid	Video	Oblique	Drone	Vehicle
WeedMap	Orthophoto	Vertical	Drone	Weed
PST900	Video	Oblique	Quadruped robot	Human

), including vehicle detection, weed mapping, and human detection, were utilized to evaluate the proposed method. The UAVid dataset [9] consists of RGB images with an oblique view from a drone platform for vehicle detection. The UAVid dataset consists of a time-series dataset targeting semantic labeling for urban scene analysis from an oblique drone perspective. The characteristics that make the UAVid dataset a standard dataset are (1) the time-series high-resolution images and (2) the different landscape types, including different types of vehicles. The WeedMap dataset [40] consists of color-inferred images with a vertical view from a Sequoia sensor located on a Mavic Pro platform in Eschikon, Switzerland, and it is used as an example to study crop and weed detection assessment. Crops on the Eschikon fields were sowed on April 5, 2017 and were arranged in 50 cm rows. Their growth stage was about one month at the moment of data collection, and the sizes of the crops and weeds ranged from 5 to 10 cm. The PST900 dataset [15] consists of multimodal images (thermal) with an oblique view from a quadruped mobile robot platform. This dataset comprises synchronized and calibrated thermal time-series images with a size of 1280 × 720 pixels for real-time target detection.

3. Results

3.1. Metrics

To evaluate the predicted targets and anomalies, the standard quality measures of the Jaccard index (J), correctness (C), and entropy (E) of the probabilities $p_n$ associated with the histogram of input $I$ were used, and they are computed as follows:

$$J = \frac{True Positive}{True Positive + False Negative + False Positive}$$

(1)

$$C = \frac{True Positive}{True Positive + False Negative}$$

(2)

$$E = -{\displaystyle \sum }_{n = 0}^{255}{p}_n\left(I\right).{\log }_2\left(p_n\left(I\right)\right)$$

(3)

3.2. Uncertainty Analysis of Feature Extraction

In this subsection, we evaluate the qualitative uncertainty of the feature extraction step for multimodal images. In action, the CNN models by learning the geometric shapes in the first layer and then evolving to learn representations of the input data in the deeper blocks, resulting in a more accurate target prediction. The proposed method consists of the input layer (multimodal data), hidden block (the proposed layer), and output (target), whereas the hidden block can include a multilayer of low-level properties to high-level properties. Figure 3 shows the difference between low-level and high-level representation extraction by the proposed model.

3.3. Exprimental Results

To evaluate the performance of the trained network for each scenario, a test set and a support set were selected outside the train set. The results of the multimodal targets and anomalies are shown in Figure 4. Vegetation regions and shadow regions in the test set are an important challenge in target localization. To improve the training set, we added time-series data that included various grass and tree covers to make the model robust for target prediction in the presence of vegetative regions. Moreover, the proposed method was trained using shadow-included multimodal time-series images from the various drone and quadruped robot images to learn how the model tackles that problem.

Table 4. The quality metrics for the predicted targets of multimodal datasets.

Data	Target	Samples	Backbone	J (%)	E	T (ms) 1
UAVid	Vehicle detection	Set-1, n = 15	SA-Net	83.4	0.41	80
		Set-2, n = 15		89.3	0.32	76
		Set-3, n = 15		94.6	0.21	78
WeedMap	Weed and crop mapping	Set-1, n = 20	DMF-Net	89.6	0.28	59
		Set-2, n = 20		96.9	0.19	62
		Set-3, n = 20		92.0	0.26	49
PS500	Human detection	Set-1, n = 20	MF-Net	83.2	0.37	43
		Set-2, n = 20		87.7	0.36	48
		Set-3, n = 20		85.3	0.29	39

¹ On an NVIDIA Tesla K80.

shows the numerical predictions of the multimodal test performed using the proposed method compared to the different Jaccard index and entropy metrics. As shown in Table 4, the Jaccard index and entropy metrics have values of 89.1% and 31.30% for vehicle detection from RGB images, 92.83% and 24.33% for crop and weed mapping from color-infrared images, and 85.40% and 34.00% for human detection from thermal images, respectively. The low computational cost required by the proposed model to classify a given multimodal image makes feasible its integration into real-time applications such as drones.

3.4. Comparative Study of Few-Shot Learning and Classical Methods

In this section, two classical methods, including Support Vector Machine (SVM) [41] and Random Forest (RF) [42], were used for comparisons. For a fair comparison, all methods were trained in this study from the beginning using the same training dataset (Section 2.3) and feature extraction method (Section 2.2) that was applied for the training of the proposed few-shot learning model. The quantitative assessments of this study show that the average IoU scores for the proposed method, SVM, and RF are about 78%, 47%, and 39%, respectively. Moreover, the qualitative comparison of the proposed few-shot learning model prediction with SVM and RF is shown in Figure 5. The qualitative results show the ability of the proposed few-shot learning model to detect smaller target regions in a scene while producing a confident result.

4. Discussion

Table 5. Static and dynamic target detection comparisons on different types of test scenes from the proposed method and PFE-Net.

				IoU		C		E
Methods	Scenario	Backbone	Samples	1 shot	10 shot	1 shot	10 shot	1 shot	10 shot
Ours	Vehicle	SA-Net	45	46.7	77.2	83.7	93.1	26.4	17.3
PFE-Net				47.2	73.1	80.1	86.4	34.1	24.8
Ours	Weed	DMF-Net	60	51.5	89.4	73.2	91.2	29.1	18.3
PFE-Net				51.9	84.2	72.1	83.5	36.3	27.1
Ours	Human	MF-Net	60	63.4	91.5	79.4	93.1	24.1	17.4
PFE-Net				57.3	87.3	74.3	83.4	34.8	29.6

shows the quantitative results for the weed and vehicle detection from the drone images. In this study, we compared the proposed method against the new method PFE-Net [32]. A visualization of static and dynamic target detection results of the test stage is shown in Figure 6, Figure 7 and Figure 8. The proposed model achieves mean correctness for the vehicle, weed, and human detection of 88.4%, 82.2%, and 86.25% for the 45 RGB, 60 CIR, and 60 thermal tested images, respectively, while the mean entropy for the vehicle, weed, and human detection is 21.9%, 23.7%, and 20.75%, respectively. As a result, there is an inverse relationship between IoU and entropy value in target detection from large-scale remote sensing images. Experimental results verified this point for the proposed few-shot learning model. Three unique aspects of this work are:

• Our findings indicate that the proposed few-shot learning model offers better generalization performance and outperforms other methods. The proposed few-shot learning model achieved the highest mIoU in comparison to other evaluated methods, such as PFE-Net (as a few-shot learning model), SVM, and RF, for each dataset in multimodal images. Therefore, compared with the state-of-the-art and classical models, the target detection accuracy was improved;
• To our knowledge, we have presented the first multimodal few-shot learning method with a low computational cost for RGB, CIR, and thermal modalities based on uncertainty estimation;
• The proposed method uses unique features of multimodal images during the feature extraction from squeeze-and-attention layers.

In this subsection, we present an ablation study to compare some different model variants, such as different loss functions for training, multiple modalities, and CNN backbones, and justify our design choices.

Table 6. Ablation study on using different loss functions, modalities, and backbones for the proposed method.

Feature Extractor	Backbone	Loss	Modalities	IoU	C	E
Proposed layer ${\ell }_{out}$	ResNet-101	Dice	RGB (Vehicle)	64.2	84.2	23.4
	HRNet.v2			63.1	81.3	24.6
	ResNet-101	WBC		70.2	84.3	21.6
	HRNet.v2			69.1	82.1	22.7
	ResNet-101	WeC		73.5	85.4	19.8
	HRNet.v2			76.1	87.4	18.2
	ResNet-101	Dice	CIR (Weed)	79.3	84.1	21.4
	HRNet.v2			82.5	84.9	20.7
	ResNet-101	WBC		83.4	85.2	19.8
	HRNet.v2			84.1	86.6	18.9
	ResNet-101	WeC		86.2	89.3	18.7
	HRNet.v2			87.1	90.2	18.5
	ResNet-101	Dice	Red + Green + Red edge + Near-infrared (Weed)	82.4	87.2	19.6
	HRNet.v2			85.5	88.3	18.5
	ResNet-101	WBC		84.2	88.7	18.3
	HRNet.v2			86.8	88.5	18.2
	ResNet-101	WeC		87.8	90.7	18.6
	HRNet.v2			88.0	91.0	17.9
	ResNet-101	Dice	Thermal	71.3	84.2	26.1
	HRNet.v2			74.2	85.5	20.3
	ResNet-101	WBC		70.3	83.1	27.3
	HRNet.v2			72.9	85.3	24.7
	ResNet-101	WeC		83.4	89.4	18.3
	HRNet.v2			86.7	90.6	17.6

shows some ablation study results to investigate the behavior of the proposed method.

4.1. Loss Functions for Training

In this subsection, all models were evaluated with different loss functions. Although the proposed method with a Weighted Cross entropy (WeC) loss function delivers good results, the proposed method was tested with another two loss functions, consisting of Dice Loss (DiL) [43] and Weighted Bootstrapped Cross-entropy (WBC) [44]. We trained our network using the absolute error between the ground truth map and the model’s prediction.

4.2. Multiple Modalities

The proposed target detection method was tested with different data modalities, such as red, green, blue, near-infrared, and red edge channels.

4.3. CNN Backbones

The proposed model can be set up with different backbones for target detection. We selected two backbones for the ablation study. The experiments were carried out with the ResNet-101 and HRNet.v2 [45] backbones.

5. Conclusions

Target localization from drone and quadruped robot platforms has become the dominant tool for real-world applications such as vehicle monitoring for traffic management, weed mapping for smart farming, and human detection for search and rescue missions. Target segmentation with the use of uncertainty estimation in few-shot learning for drone images can potentially improve scene understanding with a small training dataset, while many target detection methods appear to understand single-time localization with a big training dataset. This paper presents a new few-shot learning architecture for some core remote sensing problems. Our method outperforms the state-of-the-art models on many challenging remote sensing datasets.

In this study, we have performed three multimodal experiments for target detection from the drone and quadruped robot time-series images based on a new squeeze-and-attention method for few-shot learning from a small training dataset. The method has achieved impressive performances in various real-world tasks, which can be grouped into three categories: weed and crop mapping; vehicle and traffic monitoring; and human tracking. To improve the generalization of the trained models for 3D target reconstruction, model architecture needs to be developed in future studies.