Object Localization and Sensing in Non-Line-of-Sight Using RFID Tag Matrices

Abstract

RFID-based technology innovated a new field of wireless sensing, which has been applied in posture recognition, object localization, and the other sensing fields. Due to the presence of a Fresnel zone around a magnetic field when the RFID-based system is working, the signal undergoes significant changes when an object moves through two or more different Fresnel zones. Therefore, the moving object can be sensed more easily, and most of the sensing applications required the tag to be attached to the moving object for better sensing, significantly limiting their applications. The existing technologies to detect static objects in agricultural settings are mainly based on X-ray or high-power radar, which are costly and bulky, making them difficult to deploy on a large scale. It is a challenging task to sense a static target without a tag attached in NLOS (non-line-of-sight) detection with low cost. We utilized RFID technologies to sense the static foreign objects in agricultural products, and take metal, rock, rubber, and clod as sensing targets that are common in agriculture. By deploying tag matrices to create a sensing region, we observed the signal variations before and after the appearance of the targets in this sensing region, and determined the targets’ positions and their types. Here, we buried the targets in the media of seedless cotton and wheat, and detected them using a non-contact method. Research has illustrated that, by deploying appropriate tag matrices and adjusting the angle of a single RFID antenna, the matrices’ signals are sensitive to the static targets’ positions and their properties, i.e., matrices’ signals vary with different targets and their positions. Specifically, we achieved a 100% success rate in locating metallic targets, while the success rate for clods was the lowest at 86%. We achieved a 100% recognition rate for the types of all the four objects.

1. Introduction

Foreign object detection is widely used in the industrial field, mainly because industrial products generally have high output value, so people are more willing to invest expensive technologies and devices in this field. However, compared to industrial products, the cost-effectiveness of agricultural products is relatively lower, and some expensive devices are rarely used in the agricultural field. Especially in the early stage of agricultural production, there are foreign objects such as metal screws and rubber from ageing machines, and clod and rock from farmland; these objects will damage the equipment if they are not moved in time. Due to the complex environment, some precise and active devices are difficult to deploy. These foreign objects in agricultural products are static and invisible and invisibly buried within the medium, which poses a challenge for foreign object detection.The current detection methods are mostly based on technologies such as X-ray [1] and high-power radar [2] to achieve targets sensing. These devices are expensive and inconvenient to deploy on site, making it difficult to apply on a large scale in smart agriculture. Currently, foreign objects detection relies mainly on manual sorting methods, which are time-consuming, labor-intensive, inefficient, and unable to detect the objects hidden inside a medium. Therefore, in agricultural object detection, there is an urgent need for a detection method that is fast, accurate, low-cost, and convenient for on-line deployment. In recent years, wireless sensing methods based on radio frequency have attracted great attention. It is a widely used technology in the field of the Internet of Things (IoT), smart agriculture and so on, and it is considered one of the most influential technologies this century. RFID-based wireless sensing is one the core technologies in many fields due to its low cost, labeling ability, easy deployment, and high sensing accuracy, such as indoor scene sensing and positioning [3,4,5], smart agriculture [6,7,8,9], humidity detection [10], trajectory tracking, and attitude recognition [11,12,13]. In the application of metal detection and material sensing [14], especially in localization [15,16], RFID technologies have been developed for many devices due to their low cost and convenient deployment [17]. More recently, the widespread use of these applications in indoor positioning has gradually received much attention from researchers and has generated very meaningful ideas.

The LANDMARC system [18] uses RFID technology to locate objects inside buildings. It is an early RFID-based indoor positioning system. However, the RFID tags in this system are customized active tags, which are costly and difficult to deploy. Subsequently, Ref. [19] utilized commercial RFID tags to locate the target; it achieved localization without hardware modification, which can greatly reduce the costs. However, due to the coarse-grained nature of RSS, the localization accuracy is not satisfied. Ref. [4] proposed a hierarchical classification localization system to accurately locate the target position, which divides the environment into multiple regions to reduce the number of training and prediction classes for the classifier. The system can achieve precise positioning, while its robustness is challenged because it significantly relies on classifier design and the number of training samples. The authors of Refs. [20,21,22] all utilized RFID methods to achieve the localization of moving objects, while Ref. [20] offered a way to identify objects without any contact; it proposed a new and innovative approach for the localization of moving objects using a particle filter by incorporating RFID phase and laser-based clustering. Ref. [21] proposed a method based on an RFID tag array and laser ranging information to locate human movements, which is similar to the method used in our paper. However, it requires the human to be equipped with a tag array, limiting its application. Ref. [22] presented an approach to integrate RFID phase and laser range information for the tracking of dynamic objects in an environment; it determined the locations of objects by comparing the velocities estimated by two different systems. These proposed approaches use the commercial off-the-shelf RFID devices and do not require the modelling of radio signal propagation, which attempts to overcome the shortcomings of Ref. [4] and uses mathematical models to locate the target. However, the mathematical model failed to be universally applied to various tasks. In addition, although the phase can accurately sense the target position, its periodic characteristics limit its application. Refs. [23,24,25] all proposed a 3D localization scheme based on RFID tag arrays, where [24] only used one mobile RFID antenna for target localization. Ref. [23] used the signal arrival angles as features for localization. In addition, Ref. [25] deployed multiple tags on the upper parts of human limbs, used dual antennas to obtain the tag phase, and used phase changes to track the 3D position of human bones. They have achieved relatively satisfactory accuracy, but the tag needs to be attached to the target, which limits the practical application.

Summarized from the above references, to improve the accuracy of localization, some utilized the fine-grained phase as a feature for sensing the target [26,27,28], although it is troubled by phase ambiguity. To achieve low-cost and convenient applications, some utilized the coarse-grained RSS as an indicator to sense the targets [29,30,31], although its accuracy is affected by the multi-path. To sense the static object, some tried to use both phase and RSS to sense the target [32], but they encounter the issue of insensitivity to the static object in the Fresnel zone. Different from the above works, we use RSS and S11 as feature vectors to form a two-tag array to sense and locate the objects in NLOS imaging. The vector (RSS, $S_{11}$) can overcome the phase ambiguity issue and can effectively remove the influence of the multi-path. More importantly, the signals from the two-tag arrays deployed in different directions are sensitive to the position of the static objects. The principle of sensing and localizing foreign objects in a medium is that when RFID tags receive electromagnetic waves from RFID antenna, they will be activated and emit signals back to space in the form of scattering. When the signal propagates in the medium and encounters foreign objects, they will be changed. The system detects and senses foreign objects using the changing features in the signals when there are objects in the medium. The main contributions are summarized as follows:

• To the best of our knowledge, this is the first system that utilizes RSS and $S_{11}$ as a feature vector to simultaneously sense and locate the foreign objects, it can sense the number of objects, determine their types, and locate their positions.
• This paper presented a low-cost and easily deployable system prototype using RFID tags in non-line-of-sight scenarios, achieving the sensing of static objects by deploying two tag arrays with one RFID antenna, although the static object is less sensitive to electromagnetic signals in the Fresnel zone.

2. Architecture of the Sensing System Based on Tag Matrix

There are three modules in this system, including data collecting, virtual spatial coordinates division, and foreign object sensing.

The data collection module uses a wireless and passive sensing method based on RFID, which is easy to deploy at a low cost. The tag matrices are deployed within the sensing area, as shown in Figure 1; they are deployed at the bottom and side of the space, respectively. The RFID antenna is placed above the sensing space so that they can radiate to all tags. When tags receive electromagnetic radiation, they are activated and emit signals into space, and these signals are further processed by RFID readers.

In the virtual spatial coordinates division module, there are two sub-modules including a tag deployment sub-module and virtual space coordinate division. In the tag deployment sub-module, as shown in Figure 1a, we deployed tag arrays at the bottom and side of the space, respectively, which makes it easier to capture more sensitive information about foreign objects. In the virtual space coordinate division sub-module, the sensing space can be divided into multiple virtual coordinates based on the tags’ matrices deployed on two different planes, i.e., the XOZ plane and the XOY plane, as shown in Figure 1a. The blue solid points represent the virtual position, the black hollow points represent the tags’ matrix points deployed in the XOZ plane and the XOY plane, respectively. If there is a foreign object, it will be located near these virtual intersections, and the intersection closest to the foreign object is considered as its coordinate. Here, we take $X_1$ and $X_2$ as foreign objections in the medium, as shown in Figure 1b; they can be “projected” onto planes XOY and XOZ, respectively, using the located algorithm proposed in Section 4. The system determines the virtual spatial coordinates of foreign objects by projecting their positions at virtual intersections in space.

Figure 2 shows the system flowchart of the three modules. For the object sensing module, there are three sub-modules including the sensing and counting algorithm, the localization algorithm, and the BP model-based recognition. In the sensing and counting sub-module, the system achieves sensing of the object if there are foreign objects in the medium, and to count the number of objects. In the localization sub-module, the system achieves the location of the objects through the virtual coordinates, which helps people quickly locate and remove them. In the recognition sub-module, the system utilizes two collected feature values ($S_{11},RSS$) when foreign objects are presented in the medium to map the types of the foreign objects, which can be easily achieved through a BP neural network model.

Here, it should be noted that the object to be sensed is unknown (the presence or absence of foreign objects is unknown), so the system first collects the signal features when there are no foreign objects. The signal features collection is one-time, and they are stored in the database for comparison to determine the presence of foreign objects. The system detects an object by comparing the signal changes before and after the appearance of the foreign object in the sensing space. Then, the next steps are to locate them and to remove them. The system determines the object’s position in the coordinates by virtual coordinates, so that people can remove objects in time. A “projection” method is used to determine the virtual coordinates of foreign objects in the sensing space by deploying tag matrices on two planes (XOZ plane and XOY plane). In the end, the BP network model is utilized to sense the types of the objects using the collected data in NLOS.

We use the vector V($S_{11}$,RSS) as features and propose the Euclidean-distance-ratio (EDR) method to detect whether there are foreign objects. A projection method is proposed to determine the position of the object. The cost of the system is low, and each tag only costs about 0.15 RMB, making it possible for large-scale application. Moreover, the system can work well in many deployment environments; as long as the RFID antenna can communicate with the tags, the system can sense foreign objects in NLOS with bad and complex environments.

3. Feasibility Analysis

This section presents the preliminary study of the sensing and localization of objects using scattering coefficient ($S_{11}$) and received signal strength (RSS) in a medium (such as cotton and wheat).

3.1. $S_{11}$-Based Sensing Model

$S_{11}$ represents scattering parameters in a system; it is an important parameter in microwave transmission. $S_{11}$ is also known as a reflection coefficient, which describes the frequency domain characteristics of the transmission channel. Through the S-parameter, almost all the characteristics of the transmission channel can be known; as shown in Figure 3a, it denotes the two ports network, where $Z_O$ denotes load impedance, $a_1$ and $b_1$ represent the incident and reflected signals from Port 1, $a_2$ and $b_2$ represent the signals of Port 2. We use Equation (1) to represent the signals propagation in a circuit, where $a_n$ and $b_n$ represent $a_{1,2}$ and $b_{1,2}$, respectively, and $V_n$ and $I_n$ represent the voltage and current of the corresponding ports. If $S_{11}$ is used to represent the input reflection coefficient, i.e., input echo loss. Then, we have the following expression $S_{11}=\frac{V_{in}}{V_{out}}=\frac{b_1}{a_1}|a_2=0$. In general, the S-parameter is a complex number. In this design, Equation (2) is used to represent the value of $S_{11}$, where $mag\left(S_{11}\right)$ denotes the amplitude of $S_{11}$, which is a real number.

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}& a_{1,2}=\frac{1}{2\sqrt{z_0}}(V+Z_0{I}_{1,2})\\ & b_{1,2}=\frac{1}{2\sqrt{z_0}}(V-Z_0{I}_{1,2})\end{array}\right.\end{array}$$

(1)

$$\begin{array}{c}\hfill dB\left(S_{11}\right)=20\ast log\left(mag\left(S_{11}\right)\right).\end{array}$$

(2)

For this system, a single port (Port 1) is utilized to perform the work of $S_{11}$; as shown in Figure 3a, the $S_{11}$ can be read from a VNA device. We aim to sense the object by observing the changes of $S_{11}$ using Equation (2). According to the theory of electromagnetic wave transmission, signals will change if they are passing through two different media; thus, the signals will be scattered when the foreign objects are present in the medium. Some of the signals are reflected, some transmitted, and some are absorbed by the foreign object. $S_{11}$ will be changed for the above reasons, indicating that it can be used to represent the presence of the object.

3.2. RSS-Based Sensing Model

Received signal strength (RSS) denotes the signal intensity when passing through a transmission line in a clean wireless electromagnetic environment; the strength of the RFID received signal is obtained using the formula $P_R=P_T{\left(\frac{\lambda \sqrt{G_T}\sqrt{G_R}}{4\pi d}\right)}^2$, where $P_R$ and $P_T$ represent the power of the receiving and transmitting ports, respectively, and $G_R$ and $G_T$ represent the antenna gain at the receiving and transmitting ports. d denotes the distance between receiving and transmitting ports. The above formula indicates that the RSS is only related to the distance d if the receiving and transmitting power is known. However, obtaining RSS through the above formula can be challenging. Typically, Equation (3) is used to illustrate how RSS changes with d [33],

$$RSS\left(d\right)=RSS\left(d_0\right)-10\ast \gamma {lg}_{10}\frac{d}{d_0}+S_{0,\sigma },$$

(3)

where $RSS\left(d\right)$ denotes the strength with distance d from antenna to tag, in decibels. $d_0$ denotes the reference distance from a reference tag to the antenna. It is a constant, so $RSS\left(d_0\right)$ is also a constant. $\gamma$ denotes the path loss factor, $S_{0,\sigma }$ denotes Gaussian noise with a mean of 0 and a variance $\sigma$. To simplify the model, we use $\mathsf{\Delta }$RSS to show the RSS changes before and after there is/are no foreign objects as illustrated in Equation (4).

$$\begin{array}{c}\hfill \mathsf{\Delta }RSS=RSS\left(d\right)-RSS\left(d^{\prime }\right)=10\ast \gamma {log}_{10}\frac{d^{\prime }}{d},\end{array}$$

(4)

where d and $d^{\prime }$ represent the paths of signal propagation in the medium before and after the objects are present. Assuming that the propagation path d′ is not changed after objects are present in the medium, then we have $d^{\prime }/d=1$, and there should be $\mathsf{\Delta }RSS=0$. However, there is $\mathsf{\Delta }RSS>0$ upon observation, indicating that the objects indeed change the signals paths. We plotted Figure 4b to illustrate RSS when there are objects in the medium, indicating that $\mathsf{\Delta }RSS$ can be a feature to sense foreign objects.

We provide a visual representation of the signal propagation path when a foreign object is present in the sensing space, as depicted in Figure 4a. In the figure, points A, B, and C denote the tag, the RFID antenna (responsible for receiving and transmitting), and the foreign object, respectively. When the electromagnetic wave from antenna B activates tag A, the tag responds by transmitting the signal back into the medium. In a uniform medium without object C, the signal path of the tag would be $\{A\to B\}$ with a distance of d. However, in the presence of object C, the transmission path can be mainly divided into two ways: $\{A\to B\}$ and $\{A\to C\to B\}$. The propagation paths of the signal are the superposition of the two paths, indicating that the object alters the signal’s trajectory. This alteration is in line with the triangle theorem, which states that the third side of a triangle is greater than the sum of the other two sides, i.e., $d_1+d_2>d$. Consequently, any changes in the signal path can lead to a corresponding change in $\mathsf{\Delta }RSS$.

4. Design of Sensing and Localization Algorithm

This section presents the design of the sensing and counting algorithm, the localization algorithm, and the BP neural network-based recognition method.Firstly, we investigate a method to determine the number of foreign objects in the medium by observing changes in the tag signal when there either are or are no objects. Secondly, leveraging the propagation characteristics of signals in the medium, we develop a mathematical model to determine the object’s position in a non-line-of-sight (NLOS) scene.Finally, the system utilizes the collected data (previously used in sensing and localization modules) to achieve the recognition of foreign object types.

4.1. ‘Euclidean Distance Ratio’ Sensing Algorithm

We have previously demonstrated the feasibility of sensing the object using RSS and $S_{11}$. In this section, we introduce the design of a foreign object sensing algorithm called the $D_{er}$ algorithm, which relies on the Euclidean distance ratio. Assuming the Euclidean distance between two vectors $V_1(x_1,y_1,z_1)$ and $V_2(x_2,y_2,z_2)$ in space is defined as $d_{Eu}(V_1,V_2)=\sqrt{{(x_1-x_2)}^2+{(y_1-y_2)}^2+{(z_1-z_2)}^2}$. We utilize $V_d\left(i\right)$ to represent the feature vector of tag (i) collected by the system, and d to represent the distance of signal propagation path, $V_{d^{\prime }}\left(i\right)$ = $V(S_{11\_x},RSS\_x)$ and $V_d\left(i\right)$ = $V(S_{11\_0},RS{S}_0)$ represents the collected feature vectors of tag (i) when there is or is no object in the sensing medium. Then, the Euclidean distance between $V_{d^{\prime }}\left(i\right)$ and $V_d\left(i\right)$ is expressed as Equation (5),

$$\begin{array}{c}\hfill d=d_{Eu}(V_{d^{\prime }},V_d)=\sqrt{{(S_{11\_x}-S_{11\_0})}^2+{(RS{S}_{\_x}-RS{S}_{\_0})}^2}\end{array}$$

(5)

$$\begin{array}{c}\hfill D_{er}\left(i\right)=\frac{d_{Eu}(V_{d^{\prime }},V_d)}{|V_d|}\%.\end{array}$$

(6)

Then, the sensing algorithm $D_{er}$ is formulated as Equation (6), where $d_{Eu}(V_{d^{\prime }},V_d)$ represents the Euclidean distance between vectors $V_{d^{\prime }}\left(i\right)$ and $V_d\left(i\right)$ for tag (i), and Equation (6) illustrates the Euclidean distance ratio between the two feature vectors with and without foreign objects in medium. If there is an object in the medium, the $D_{er}$ will change according to Equation (6) because the object affects the propagation path of d′. In theory, when there is no foreign object near the tag, i.e., the signal path is not altered $(d^{\prime }=d)$, then $D_{er}=0$, otherwise $D_{er}>0$, which is referred to as abnormal $D_{er}$ in the tags. Thus, $D_{er}\left(i\right)$ can be used to sense the object near $tag\left(i\right)$, and its value vaguely indicates how far the object is from the tag. If there are multiple foreign objects in the sensing medium, their number can be determined by observing the abnormal $D_{er}$. However, in practical applications, $D_{er}$ is usually greater than zero for there is noise in the lab, and we address this issue by setting a threshold. If $D_{er}$ from a tag exceeds the threshold, it is considered an abnormal $D_{er}$ that can be used to sense the foreign objects.

4.2. ’Projection’ Localization Algorithm

We have presented a sensing algorithm that can determine whether there are foreign objects or not in Section 4.1. However, in practice, we urgently need to know their positions in order to remove them promptly. Therefore, in the following section, we introduce a method to locate the object based on $S_{11}$ and RSS in NLOS scenes.

To verify the feasibility using $S_{11}$ and RSS for localization, we conducted the following experiments to test the sensitivity of the above two features to position. We placed the foreign object C between the tag and the RFID antenna, moved its position, and observed the changes in $S_{11}$ and RSS. As shown in Figure 4a, C was gradually moved from A to B, and the impact of the position of C on $S_{11}$ and RSS was observed separately ($S_{11}$ can be read from an analog vector network analyzer, and RSS can be read from an RFID reader). The results showed that $S_{11}$ and RSS vary with the foreign object position, indicating that the foreign object position is sensitive to both $S_{11}$ and RSS, as illustrated in Figure 5.

Based on the above observations, we conclude that $S_{11}$ and RSS can not only sense the foreign objects, but are also sensitive to their positions. To obtain the 3D position of foreign objects more accurately, we divided the sensing space into grids, as shown in Figure 1a. We deployed $M\times N$ and $M\times L$ RFID tags on the planes XOY and XOZ, respectively (represented by black hollow dots). These tags intersect with each other in the vertical direction of their respective planes, forming virtual intersection points (represented by blue solid dots). We use P(X, Y, Z) to represent the 3D position virtual coordinates, while $P_{XY}(X,Y)$ and $P_{XZ}(X,Z)$ represent the coordinates of RFID tags on the plane XOY and XOZ, respectively. If the point ’O’ is set as the coordinate of P(1,1,1), then the other points in the sensing space can be determined.

There are two steps to locate the object’s position; first, the system projects the objects onto the tag positions in plane XOY and plane XOZ, respectively, then the intersection of the tags from the two planes is taken as the object’s position in the 3D coordinate. Equation (7) is used to obtain the tag projection position through the largest $D_{er}$, where tag (i) denotes the object’s projected position with tag number (i) whose $D_{er}$ is the most abnormal. $P_{tag}(i,j)$ denotes the possibility that the object is projected on tag (i), ${\sum }_{i=1}^M{\sum }_{j=1}^N{P}_{tag}(i,j)=1$. It is reasonable to use probability to represent the projection position, as this system cannot accurately project foreign objects onto the tags but only in their vicinity. In fact, a object’s 3D position is also a probability that is nearest to the grid, because the system uses the $P_{tag}(i,j)$ from two planes to determine the position of the object in 3D. So, this method may fail to locate the object. For example, if the system obtains the positions $P_{XY}(X,Y)$ and $P_{XZ}(X^{\prime },Z)$, and there is $X\ne X^{\prime }$, then no virtual intersection can be found, and the localization fails.

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}& tag\left(i\right)=\underset{i}{max}{D}_{er}\left(i\right):\left\{\left(i\right|\forall j:D_{er}\left(j\right)⩽D_{er}\left(i\right))\right\}\\ & P_{tag}(i,j)=\frac{D_{er}(i,j)}{{\sum }_{i=1}^M{\sum }_{j=1}^N{D}_{er}(i,j)}.\end{array}\right.\end{array}$$

(7)

To improve the success rate of localization, we propose a “sub-optimal” method to address the above issues. When the initial localization fails, the system selects the tag (k) and tag (i) with the second maximum $D_{er}$ from planes XOY and XOZ, respectively. The corresponding plane positions are $P_{XY}(X,Y)$ and $P_{XZ}(X^{\prime },Z)$; if the system succeeds in localization, it will stop and return to position $P(X,Y,Z)$ or $P(X^{\prime },Y,Z)$, otherwise, if the localization is unsuccessful, the system persists in selecting the third maximum $D_{er}$ until successful localization is achieved.

4.3. BP Neural Network Based Object Recognition

The back-propagation (BP) network is the most widely applied model; it is a multi-layer feed-forward network trained using the error back-propagation algorithm, comprising input, hidden, and output layers. In comparison with deep learning networks with complex structures, the BP network tends to perform more reliably on smaller datasets due to their relatively simple structure, which makes them less prone to overfitting. Here, we utilized RSS and $S_{11}$ as a feature dataset to ’learn’ and ’train’ the network using a provided batch of corresponding input–output data to establish a specific network model.

The system collects data from two tag matrices as the training dataset. Each tag captures two features, i.e., RSS and $S_{11}$, forming a vector V(RSS, $S_{11}$). Assuming the size of each tag matrix in a plane is $M\times N$. With each data collection, the system obtains an $M\times N\times 2\times 2$ dataset, representing two feature values for each of the two matrices. When foreign objects appear in the sensing space formed by the tag matrices, these feature data can indicate the presence of foreign objects.

For the aforementioned collected data, information about foreign objects is available. However, establishing a direct relationship between the gathered data and the objects poses challenges for human perception. Specifically, understanding how distinct data features correspond to different objects becomes problematic; in essence, describing the underlying mechanism using mathematical methods is intricate. Thus, our approach involves employing machine learning techniques to address this nonlinear problem. Among the various machine learning methods, the back-propagation (BP) network stands out as an excellent choice. It possesses the capability to mitigate over-fitting risks when handling smaller datasets by fine-tuning its parameters. Moreover, it exhibits rapid training speed and high prediction accuracy. Notably, several other machine learning approaches are suitable for such data structures, detecting the four types of objects outlined in this paper effectively. Our emphasis lies in highlighting that numerous machine learning methodologies can adeptly detect foreign objects through training on this modest dataset, with the BP network being one of the more straightforward options.

The establishment of network models currently lacks a unified and comprehensive theoretical method and is generally determined by experience. In fact, the determination of parameters such as the input layer, hidden layer, and output layer constitutes the process of building the BP network model:

(I)

Input layer: The numberof neurons in the input layer of the network is consistent with the tag feature parameters, which means that each feature value corresponds to a neuron in the input layer. This paper takes RSS and $S_{11}$ as feature parameters; therefore, the input layer contains two input neurons.

(II)

Hidden Layer: The number of neurons and layers in the hidden layer is generally determined based on the size of the data and personal experience. After repeated experiments, we set the optimal configuration for this layer as a single hidden layer with 25 neurons.

(III)

Output layer: This layer is determined by the output target of the network, which is determined by the number of foreign objects to be identified. There are four types of targets to be identified: metal, rock, rubber, and clod. Therefore, the number of neurons in this layer is set as four.

After determining factors such as the number of layers, neurons in each layer, initial weight coefficients, transfer functions, learning algorithms, etc., a BP network is thereby established. Training a neural network system with ’expert knowledge’ is achieved only through the continuous modification of network parameters. The network model is illustrated in Figure 6, where ${\nu }_{i,j}$ and ${\omega }_{i,j}$ denote the weight of the BP network. The excitation function is set as ’sigmod’, the transfer function as ’tansig’, the gradient descent BP training function as ’traingd’, the network learning function as ’learngdm’, and the mean square error performance analysis function as ’mse’. The BP network model is integrated and encapsulated in the Matlab 6.5 toolbox, making it convenient to use for modeling.

5. Experiment and Analysis

5.1. Experimental Setup

The experiment is performed in the sensing space with a size of 945 mm × 520 mm × 510 mm. Seedless cotton and wheat are used as a transmission medium to observe the properties of signal propagation in different media. Metal screws, rock, rubber block, and clod are taken as foreign objects for the sensing target, as shown in Figure 7. Alien 9640 with a Higgs 3 chip is chosen as a tag, Laird2 S9028 polarized antenna as an RFID antenna, Impinj Speedway R220 as an RFID reader, and tektronix TTR506A as an analog Vector Network Analyzer. The system operates on a desktop computer with Windows 10 OS. The RSS is read from a Speedway R220 device, and $S_{11}$ is read from Tektronix TTR506A.

Two tag matrices are deployed in the sensing medium, with one placed at the bottom (XOY plane) and the other on the side (XOZ plane). These tag matrices are designed to mitigate the coupling effect between the tags, following the recommendations mentioned in Ref. [34]. According to the reference, it keeps a horizontal spacing of 22 cm and a vertical spacing of 17 cm between the tags. Consequently, two tag matrices are set up, each comprised of a 5 × 4 matrix of tags, placed at the bottom (XOY plane) and on the side (XOZ plane), respectively, as shown in Figure 7. To facilitate the data collection, one S9028 RFID antenna is placed above the XOY and XOZ planes, at a 45-degree angle to the two planes (illustrated without a dash box in Figure 7). To enable effective communication with each tag from the two matrices on both planes, the RFID antenna moves along the X axis of the container, collecting the feature vectors $V\left(k\right)=(S_{11},RSS)$, each 50 mm, where k = 1, 2 … 19. For a container with a length of 945 mm, a total of 19 sets of feature vectors are recorded. In the deployment of this system, $S_{11}$ represents the overall reflection coefficient in the tag matrix because the VNA cannot read $S_{11}$ for a single tag. However, during the movement the antenna, the tag closest to the antenna tends to contribute the most to the reflection measured by $S_{11}$. This information proves valuable as it helps accurately associate the sensed objects with their corresponding tag positions. On the other hand, the RFID reader has the capability to individually read $RSS$ from each tag. This allows the RSS to provide signal indicators for a single tag, further enhancing the overall performance and functionality of the system.

We set the coordinates as illustrated in Figure 1a. The 3D coordinate is represented as P(X, Y, Z), with the origin coordinate point ’O’ as the first point. If ’O’ is set as P(1,1,1), the positive direction of the X-axis should be P(2, 1, 1), P(3, 1, 1), P(4, 1, 1), P(5, 1, 1), respectively. Similarly, the positive direction of the Z-axis should be P(1, 1, 2), P(1, 1, 3), P(1, 1, 4), respectively. Therefore, there are 5 × 4 × 4 virtual point coordinates in space, illustrated in Figure 1.

5.2. Foreign Object Sensing

In this section, we validate the performance of the $D_{er}$ sensing algorithm when there are one or more foreign objects in the sensing space with media of seedless cotton and wheat, respectively.

Foreign objects are placed in the sensing space (filled with cotton or wheat), and the $D_{er}$ values of each tag are obtained using the proposed algorithm. The $D_{er}$ are plotted for each tag, as shown in Figure 8, which presents the $D_{er}$ when there is one object that appears in the medium. The indices of the X-axis and Y-axis represent the number of the tags, and the Z-axis represents tag $D_{er}$, respectively. Figure 8a,b represent tag $D_{er}$ in cotton and Figure 8c,d represent tag $D_{er}$ in wheat. Figure 8 illustrated that the abnormal tag $D_{er}$ can be an indicator to sense the target both in cotton and wheat, and the object in cotton is more easily sensed than that in wheat because there is more noise $D_{er}s$ in wheat than in cotton. In addition, the properties of the media are also the factors that impact the $D_{er}$ algorithm’s performance.

However, in practice, there may be more objects in the sensing space. So, in the following, we present the performance of the sensing algorithm when there are more objects. Figure 9 is plotted to show the abnormal $D_{er}$. Figure 9a–d represent tag $D_{er}$ corresponding to objects of {rubber block}, {rubber block, rock}, {rubber block, rock, clod}, and {rubber block, rock, clod, metal screw}, where the red dashed line denotes the threshold for sensing objects ($D_{er}$ > 1.4). $D_{er}$ is considered abnormal if it is greater than 1.4.

The system can accurately determine the number of objects by counting the number of abnormal tags $D_{er}$, as shown in Figure 9a–c. However, it fails to count the correct number when there are more than four objects in the sensing space, as shown in Figure 9d. The reason for this limitation is that the deployed tag matrix is too small to distinguish such a large number of targets. To address this issue, it is possible to consider increasing the size of the tag matrix, which would enhance the system’s ability to sense more targets. Furthermore, when metal foreign objects are present in the medium, there are many abnormal $D_{er}$ due to the strong shielding effect on the electromagnetic signal. Therefore, for single object sensing, we predict that the system performs better when one metal object appears both in cotton and wheat media because a metal object has a great impact on the signal. We will validate the idea in the following section using the projection localization algorithm.

5.3. Foreign Object Localization

In this section, we present the performance of the proposed localization algorithm through two steps: projecting the targets onto planes and achieving the virtual intersection (coordinate) in 3D space.

5.3.1. Projecting Objects onto Planes

Equation (7) illustrated that the object’s position on planes can be represented using a probability issue. Taking plane XOY as an example, in a matrix with $4\times 5$ tags, each tag is assigned the probability of $P_{tag}(i,j)$. A linear interpolation method is used to interpolate the matrix to create a heat map that visualizes the possible projecting positions on the XOY plane, as shown in Figure 10 and Figure 11. These figures show the object projected onto the XOY plane in cotton and wheat media, respectively. For metal and rock objects, the figures show an accurate position both in cotton and wheat, as shown in Figure 10a,b and Figure 11a,b, with a position of $P_{XY}(4,2)$. For a rubber block, Figure 10c shows an accurate position in cotton medium, while it achieves exact projection position difficultly in wheat medium, as shown in Figure 11c. For clod in wheat, we cannot achieve its projecting position at all, as shown in Figure 11d.

From these observations, we can conclude that the accuracy of projection varies depending on the properties of the objects. Metal objects can be accurately projected onto a plane, while the projection position of the clod is relatively blurry in wheat. Figure 10 and Figure 11 also depict the influence of the medium on the system. The projection accuracy is better in cotton medium than in wheat medium, which can be attributed to the dielectric properties of cotton and wheat. Cotton, being a non-magnetic substance without moisture content, allows for easier penetration of electromagnetic waves compared to wheat, which is a weakly magnetic substance due to its moisture content.

5.3.2. Locating Objects in 3D Space

To locate the object in 3D space, we need to obtain the projection position on both XOY and XOZ planes. So, we plot the projection position in plane XOZ as shown in Figure 12a–d, which represent the possible position of metal, rock, rubber, and clod in wheat, respectively. By observing the figures, we obtained the projection positions of the object on two planes and recorded these positions in

Table 1. The possible positions in two planes in wheat.

	Foreign Objects	Metal	Rock	Rubber	Clod
Projection Position
XOY:$P_{XY}$(X,Y)		(4,2)	(4,2)	(4,2), (5,2), (5,3)	(5,3), (5,4), (4.2), (4,3), (2,3)
XOZ:$P_{XZ}$(X,Z)		(4,3)	(4,3)	(4,2), (4,3)	(4,2), (3,1), (3,4), (2,2)
3D position:P(X,Y,Z)		(4,2,3)	(4,2,3)	(4,2,2) or (4,2,3)	(4,2,2) or (4,3,2)

. According to the localization algorithm, when $x=x^{\prime }$, the localization is considered successful. Table 1 illustrated that metal and stone can be easily located, while rubber blocks have two possible positions (one of which is the incorrect position of $P(4,2,2)$), whereas clod fail to be located. This once again leads to the conclusion that clod is difficult to locate.

For comparison, we take non-magnetic cotton as an example and use the same method to test the localization performance of foreign objects in cotton. The results are shown in

Table 2. The possible positions in two planes in seedless cotton.

	Foreign Objects	Metal	Rock	Rubber	Clod
Projection Position
XOY:$P_{XY}$(X,Y)		(4,2)	(4,2)	(4,2), (5,2)	(5,4), (4.2), (3,3)
XOZ:$P_{XZ}$(X,Z)		(4,3)	(4,3)	(4,3)	(4,2), (4,3), (2,2)
3D position:P(X,Y,Z)		(4,2,3)	(4,2,3)	(4,2,3)	(4,2,3) or (4,2,2)

. The metal, rubber, and rock blocks were also accurately located; however, the clod was positioned with two possible locations, one of which is correct. This indicates that clod failed to be located in wheat, but was successfully located in cotton. This verifies that the object localization algorithm is not only related to the properties of the objects themselves, but also to the media in which the objects are placed.

To quantify the positioning performance of the algorithm, we employed a statistical approach to determine the system’s localization accuracy. We randomly placed the objects in the sensing space, obtained the positions of these objects using the projection localization algorithm, repeated this process 88 times, and compared the results with the actual positions. The successful localization rate is presented in

Table 3. Initial localization accuracy in both wheat and seedless cotton.

	Foreign Objects	Metal	Rock	Rubber	Clod
Localization (%)
3D position in wheat (%)		87.6	81.8	61.4	55.6
3D position in cotton (%)		87.6	85.2	77.3	77.3

. It indicates that the initial localization accuracy of metals is highest in both wheat and cotton; however, the localization rate of clod is only 55.6%, which is not satisfactory. Overall, the localization accuracy in cotton is higher than that in wheat, which once again validates the conclusion that electromagnetic wave signals perform better in non-magnetic media than in magnetic or weakly magnetic media.

Table 1 and Table 2 present the cases where the algorithm fails in localization for clod. For example, while clod in wheat was not successfully located, clod in cotton was located at two different positions. This affects the reliability of the system. To address this issue, we are considering two methods to enhance the system’s performance. The first approach involves implementing the “sub-optimal solution” method mentioned in previous sections, i.e., if the initial localization fails, the system searches for the second potential location of the object until successful localization is achieved. The second approach is to add one more RFID antenna and adjust the positions of the two antennas (as shown in Figure 7, the RFID antennas with black dashed boxes). This adjustment is made to align them directly with two tag matrices (in XOY plane and XOZ plane), ensuring that the electromagnetic wave propagation direction is perpendicular to the tag matrices. This aims to improve the overall reliability and accuracy of the system.

Figure 13a shows the success rate of object localization after applying the sub-optimal algorithm using only one RFID antenna. The vertical axis in the figure represents the success rate of localization, and the horizontal axis represents the number of re-locations, where ’1st’ represents the success rate for the initial localization. The localization success rate of metals and stones reached 100% after the second and the third re-locations; the rubber and clod improved from 55.6% and 61.4% to around 85% after three re-locations. However, the accuracy improvement of rubber and clod is not significant after three more re-locations. This indicates that the system performance has reached its upper limit.

To further improve the system’s performance, we adjusted the deployment of RFID antennas, as illustrated in Figure 7. The two dashed boxes represent the deployment positions of the two RFID antennas. They are oriented towards the tag matrices separately, collecting the corresponding tag features for each. The original intention of this deployment is to facilitate the projection of foreign objects near the correct tags, with the expectation of improving projection accuracy. The result is plotted in Figure 13b and shows the variation in localization success rate as the number of re-locations increases when two RFID antennas are deployed in the sensing space. Compared with Figure 13a, it can be seen that the first localization success rate has been significantly improved when using dual RFID antennas. However, after using sub-optimal algorithms to improve system performance, it was found that the overall localization success rate of the system did not significantly improve. Although the design of dual antennas increases system costs, it fails to bring significant improvements to system performance. On the contrary, although a single antenna has a lower success rate in initial localization, the sub-optimal algorithm can compensate for this deficiency and also reduce system costs. Therefore, the single antenna combined with sub-optimal algorithm can work well to achieve a satisfactory performance.

5.4. Objects Recognition

The BP neural network is a tool packaged in Matlab 6.5, which is very convenient. Based on the parameters set by developers, a network model can be established through data training. We represent the output target of the network model in the form of a vector. As there are four types of objects to be recognized, we set the output targets as $T_1{(1,0,0,0)}^T,T_2{(0,1,0,0)}^T,T_3{(0,0,1,0)}^T,T_4{(0,0,0,1)}^T$; they represent the output targets for metal, rock, rubber, and clod, respectively. The closer the output vector is to the target vector T, the greater the similarity.

In this experiment, we selected tags from two planes as the collection targets. There were $4\times 5$ tags in each plane, and two features sensed from each tag. In one RFID antenna method, it needs to move 19 steps to scan the entire sensing space (two planes of XOY and XOZ), resulting in a data volume of $4\times 5\times 2\times 19\times 2$ being collected. Furthermore, to make the BP model more robust, the experimental data from cotton and wheat (two media) were mixed together, resulting in $4\times 5\times 2\times 19\times 2\times 2$ = 3040 samples. Therefore, for such small-scale samples, the BP neural network model is fully capable with better efficiency. We set the maximum training steps to 2000, with a training minimum error of 0.001. The learning rate was set to 0.005. The training process and results are illustrated in Figure 14 and

Table 4. Identification results of four types of objects.

${\mathit{T}}_2$	${\mathit{T}}_1$	${\mathit{T}}_4$	${\mathit{T}}_3$
0.004022	0.999691	0.001281	0.000012
0.99959	0.0018945	0.000203	0.000018
0.000031	0.000429	0.634173	0.97990
0.00099	0.000102	0.910530	0.303177

.

We placed the four types of objects in the media, collected data, and fed them into the model. The output vectors are illustrated in Table 4; from left to right, they represent rock, metal, clod, and rubber. It can be observed that, for the output target vectors of rock ($T_2$) and metal ($T_1$), their corresponding elements are very close to 1 (0.99959, 0.999691), while the other elements are close to 0, indicating that the model can recognize these two types of objects well. The output vectors of the clod ($T_4$) and the rubber ($T_3$) correspond to elements 0.97990 and 0.910530, respectively. However, the highest values for the other elements are 0.634173 and 0.303177. This implies a slightly weaker capability of the BP model for identifying clod and rubber.

According to the data structure in this paper, the input and output neurons of the designed model are fixed. However, the neurons in the hidden layers can be manually adjusted based on the training results of the model. To verify the impact of the number of neurons in the hidden layer on the model’s performance, taking the most challenging clods as an example, we set the neuron numbers as 10, 15, 25, 30, and 40, respectively. We observed the parameters of model’s output results, training time, and best training performance to determine the optimal neuron numbers in the hidden layer. The results are presented in

Table 5. The performance using different neurons in hidden layer.

Neurons Numbers	The Results of Output	Time (s)	Training Performance
10	[0.0301, 0.6627, 0.0416, 0.6712]	0.21	0.016484
15	[0.0012, 0.0286, 0.6103, 0.7859]	0.68	0.00138
25	[0.0012, 0.0002, 0.4341, 0.9105]	0.76	0.025573
30	[0.0001, 0.0000, 0.2259, 0.9218]	11.25	0.02281
40	[0.0000, 0.0001, 0.0398, 0.9267]	42.28	0.02884

. In the table, the training time increases with the number of neurons, and the increment is significant when neurons are increased from 25 to 30. Furthermore, output results are confusing when neuron numbers in the hidden layer are set as 10 and 15, while the results are clear when neuron numbers are more than 25. From the above discussions, we set neuron numbers as 25, as the optimal numbers in the hidden layer.

We compared other related works, as shown in

Table 6. Comparison of related work.

The Authors	Applications	Accuracy	Object State	Scene
Shafiq [21]	Localization	0.38 m	Moving	LOS
Kaltiokallio [35]	Localization, Tracking	0.12 m–0.53 m	Moving	LOS
Yulu Fu [20]	Localization	0.26 m	Moving	LOS
ChuYu [25]	Tracking	0.44 m	Moving	LOS
Shaonan [22]	Tracking	0.37	Moving	LOS
Eric [23]	Localization	0.38 m–0.73 m	Statics	LOS
Linqing [29]	Localization	0.26 m–0.45 m	Statics	NLOS
Our Approach	Localization, Recognition, Count	0.22	Statics	NLOS

, which represents the comparison between our approach and the other works. The application and accuracy of our approach are better than those of the other works, and our approach focuses on static objects sensing in non-line-of-sight scenarios.

6. Conclusions

We introduced a method for wireless object localization and recognition under NLOS scenarios, and investigated the propagation patterns of RFID signals in weakly magnetic and non-magnetic media, revealing the principle of foreign object recognition and localization based on tag feature signals. Unlike traditional sensing methods based on RFID, this method does not require attaching tags to the target; we achieved non-contact sensing, recognition, and localization through a tag matrix. The research results indicate that using a single RFID antenna combined with a sub-optimal algorithm can successfully sense, locate, and recognize four common objects in wheat and cotton. This study has a wide range of practical applications and can be directly applied to other fields, such as airport security and liquid foreign material detection, etc. However, it is still in the preliminary stages, and there are still some limitations. For instance, this paper did not conduct research on the sensing depth in media, resulting in the inability to sense the maximum measurement range. Furthermore, a more extensive range of media should be examined. The current presentation of system performance is limited to two media (cotton and wheat), which may not be sufficiently representative. Testing the system’s robustness would require the inclusion of a broader range of media in future studies.

However, there are still further studies that should be conducted in various environments to make the system more robust for applications, such as on the sensing depth issue, the performance when sensing smaller objects, and the impact of the environment on the system.