Differentially Private XGBoost Algorithm for Traceability of Rice Varieties

Abstract

Privacy protection in agricultural traceability has received more and more attention. Most of the existing methods only protect the original data information from the perspective of cryptography and ignore the availability of the protected information. In fact, after data is processed by cryptography, blockchain, and other technologies, it cannot be directly used for machine learning model training. Therefore, differential privacy has great potential value for privacy protection in agricultural traceability, which can enable data to participate in classification tasks under privacy protection. In this paper, we propose an integrated algorithm for agricultural traceability called Differentially Private XGBoost (DP-XGB), which can protect the privacy of the original data during the training process and obtain high model utility under the condition of a small sample size. We inject Gaussian noise into the gradient operator and Hesse operator of the original XGBoost and give the calculation method of the resulting privacy budget. Experiments show that our method can effectively obtain differential privacy guarantees and achieves very high classification accuracy when the noise is small.

1. Introduction

The security of rice is an important issue related to human security and the global economy as an important cereal crop. Tracing the variety, origin, and authenticity of rice has become a hot research topic at home and abroad [1,2]. The spectroscopy [3], stable isotope analysis [4], volatile matter analysis [5], mineral element analysis [6], and metabolomics [7] combined with data mining are often used to identify and protect the authenticity of the variety and origin of rice, and to ensure the safety and quality of rice [8,9].

In the process of traceability, blockchain [10], partial least squares [11], cluster analysis [12], deep learning [13], neural network [14], and other methods were used for discriminant modeling, in order to realize the traceability of origin or variety through the established model.

However, in the process of using the model to trace the variety or origin, the characteristic information will be easily leaked or falsified. The characteristic data of the established model is lost or the established discriminant model is invalid. These pose a serious threat to the quality and authenticity of variety and the origin of rice. Therefore, it is urgent to protect the privacy of the data to ensure the authenticity of the data and the effectiveness of the model.

To solve the privacy problems in the process of establishing the traceability discrimination model, a variety of privacy protection methods have been proposed. Qu [15] proposed a new composite chaotic key design, which can better meet the needs of information encryption and ensure the security of agricultural information on the basis of retaining the original chaotic nature in order to ensure the information security in the process of agricultural information transmission [16,17].

Due to the computational complexity of the above methods, a large number of encryption and decryption calculations are required when privacy protection is carried out, which is costly and the computational efficiency is limited, and it faces great limitations in practical applications. Differential privacy [18] technology is widely recognized as a strict privacy protection method, and its privacy protection and robustness have good theoretical support. It prevents the adversary from inferring the existence of specific records by adding noise to the query results, so as to protect data privacy and information security. For example, Chukkapalli et al. [19] proposed a framework in which the privacy of individual owners of smart farms is protected by adding noise to the data through data perturbation techniques such as white Gaussian noise. In this method, noise is directly added to the data, and the authenticity and integrity of the data cannot be guaranteed after perturbation.

In this work, we innovatively proposed a tracing model of the XGBoost rice variety based on a differential privacy perturbation mechanism, which can protect the tracing feature data well. The experimental results show that the improvement of differential privacy perturbation mechanism does not affect the accuracy of the discriminant model, and can ensure the authenticity and integrity of rice variety data and discriminant model.

Our main contribution can be summarized as follows:

(1) We propose a differential privacy XGBoost algorithm, which realizes the traceability of rice varieties under privacy protection, and measures the risk of privacy leakage.

(2) We propose a privacy protection method by adding Gaussian noise to the first derivative and the second derivative of the Taylor expansion, and can adjust the privacy protection level by changing the parameters.

(3) We carried out experiments on rice variety traceability with our privacy protection algorithm, which further confirmed the effectiveness and realizability of the method we proposed.

2. Related Work

2.1. Agricultural Traceability

To ensure the quality of agricultural products, the traceability of agricultural product information is becoming more and more important [20,21]. The key to the traceability of agricultural product information is how to ensure the accuracy and integrity of the traceability information. A large number of intelligent agricultural information traceability systems with their own characteristics have emerged to enable the public to easily query and track the whole process information of agricultural products from planting to listing [22]. In this process, the security of data is particularly important. It is necessary to ensure that the data collected in all links are accurate and unaltered [23]. Only in this way, the whole process of information traceability can be truly achieved, and the final consumers can accurately obtain the relevant information on agricultural products.

2.2. Encrpytion Tachniques

Network information encryption based on cryptography is a common method to ensure the security of the information used by traceability system, including a privacy protection method based on symmetric encryption (e.g., data encryption standard, DES) and asymmetric encryption (such as RSA) [24]. The privacy protection method based on symmetric encryption can encrypt and protect the basic data well, and the hardware requirement is low, and the encryption speed is fast. However, it is not suitable for data encryption in the traceability model because it cannot prevent data tampering before encryption. The encryption function of the privacy protection method based on asymmetric encryption is strong, but the encryption hardware requirements are high and the encryption speed is slow. Moreover, these two methods cannot prevent illegal tampering of data by legitimate personnel inside the traceability model, so they are not suitable for data encryption processing in the traceability model.

The other method is the protection method based on hierarchical security policy (such as RBAC) [25]. This method can assign different information processing rights according to different personnel attributes, but it cannot completely prevent high-level personnel from illegally tampering with relevant agricultural product data, so it is not suitable for data encryption processing in the traceability system.

2.3. Blockchain Technology

As a new technology in the field of data security [26], blockchain technology has been developing rapidly in recent years, and its application is becoming more and more extensive [27]. Kamble et al. [28] identified traceability as the most important enabler of blockchain technology in agricultural supply chain applications, and provided a reference for blockchain technology development strategy to ensure a real-time driven agricultural supply chain. Behenke et al. [29] proposed the necessary boundary conditions before the application of blockchain technology, including traceability process, unified interface standards, joint platform, and independent governance. Ho et al. [30] built a parts logistics management platform based on blockchain technology to improve the quality of logistics data and ensure the security of information sharing.

It can be said that blockchain technology ensures the sharing and security of all links of data circulation through decentralized core algorithms. At the same time, in the whole blockchain system, each node can be used as the central node, and each computing node is independent of the other, so that even if a node fails, the security and integrity of the whole data can be guaranteed. However, the open and transparent nature of blockchain has led to the problem of user identity and data privacy disclosure.

2.4. Differential Privacy

As one of the most popular privacy protection methods, differential privacy [18] has solid mathematical theoretical support. Maruseac et al. [31] abstract the dataset workflow as a set of points in a multi-dimensional space and use differential privacy to protect the path confidentiality of traceability information. The differential privacy is completely independent of the background knowledge of the attacker. After centralized collection, random noise is added to distort sensitive data before release. Some data after noise is added still retain some statistical characteristics of the original data [32], so it is more suitable to establish a traceability system to protecting data privacy.

2.5. XGBoost

Xgboost is widely used in data mining analysis as a popular classifier [33]. Wang et al. [34] proposed a distributed privacy-protection boosting algorithm suitable for various classifiers. The algorithm uses local differential privacy (LDP) as a building block to build the basic learner using the aggregation of the perturbed data shares to ensure that the privacy of the participating data owners is well protected.

For better data mining, many people combine privacy protection with a classifier [35,36], for example, Patil et al. proposed the DiffPRF algorithm [35], which is mainly a combination of differential privacy protection and random forest, but the algorithm can only process discrete features, which need to be discretized before processing continuous features.

This paper proposed a privacy-protected XGBoost algorithm for rice variety traceability. Compared with the random forest algorithm, this algorithm has better performance in preventing overfitting. The results show that the fusion of the XGBoost algorithm and differential privacy protection method can not only maintain data privacy, but also analyze the privacy-protected data, which can well solve the privacy protection requirements in the process of rice variety traceability.

3. Methodology

3.1. Differential Privacy

In this section, we review some basic concepts of differential privacy, which leads to our definition of this agricultural privacy protection issue. Differential privacy is a representative method in privacy machine learning, and it is the most universal definition can be described as follows.

Definition 1

(Differential Privacy [18]). Suppose o is an outcome that satisfies $o\in \mathcal{O}$, and  $\mathcal{O}$ is the output space. A randomized algorithm $\mathcal{M}$ is said to preserve ϵ-differential privacy if any two adjacent datasets which only differ in a single record satisfy

$$\begin{array}{c}\hfill Pr[\mathcal{M}\left(D\right)\in o]\le e^{ϵ}Pr[\mathcal{M}\left(D^{\prime }\right)\in o]+\delta .\end{array}$$

(1)

where $D,D^{\prime }\in \mathcal{D}$ are the adjacent datasets, and δ is a failure probability. f is a certain algorithm waiting to be protected. When it comes to the difference in the resulting level caused by adjacent datasets, we can describe that by sensitivity as follow.

Definition 2

($l_p$-sensitivity [37]).Suppose $f:\mathcal{D}\to \mathcal{R}$ is an algorithm, where $\mathcal{D}$ and $\mathcal{R}$ represents dataset space and the real number space. $l_p$-sensitivity can be defined as

$$\begin{array}{c}\hfill {\Delta }_p=\underset{D,D^{\prime }}{max}{\parallel f\left(D\right)-f\left(D^{\prime }\right)\parallel }_p.\end{array}$$

(2)

where ${\parallel ·\parallel }_p$ represents the p-norm, and p can be any positive real number. With the above theoretical preparation, we can define some substantive privacy protection measures. The Gaussian mechanism is a technique to realize differential privacy protection by injecting Gaussian noise, and its definition can be described as follows.

Definition 3

(Gaussian Mechanism [38]). Suppose a randomized algorithm $\mathcal{M}$ satisfies $\parallel \mathcal{M}\left(D\right)-\mathcal{M}\left(D^{\prime }\right){\parallel }_2\le {\Delta }_2$ for any adjacent datasets $D,D^{\prime }$, the Gaussian mechanism

$$\begin{array}{c}\hfill \mathcal{M}\left(D\right)=f\left(D\right)+\mathcal{N}(0,\Delta f·{\sigma }^2)\end{array}$$

(3)

with scale

$$\begin{array}{c}\hfill \sigma \ge \sqrt{2ln\left(\frac{1.25}{\delta }\right)}·\frac{{\Delta }_2}{ϵ}\end{array}$$

(4)

satisfies $(ϵ,\delta )$-DP.

3.2. XGBoost Algorithm

Suppose $D=\left\{(x_i,y_i)\right\},i=1,2,\cdots ,N$ is the training set, where $x_i\in {\mathbb{R}}^d$ represent the features and $y_i$ represent the labels. d represents the dimension of the feature. We develop a new XGBoost algorithm named DPXGB to obtain $ϵ$-differential privacy guarantee. To achieve this goal, we need to define the basic model of the boosting method. The strong classifier model composed of K weak classifiers can be described by the following expression

$$\begin{array}{c}\hfill {\widehat{y}}_i=\varphi \left(x_i\right)=\sum _{k=1}^K{f}_k\left(x_i\right),f_k\in \mathcal{F}.\end{array}$$

(5)

where $\mathcal{F}=\{f\left(x\right)=w_{q\left(x\right)}\}(q:{\mathbb{R}}^d\to T,w\in {\mathbb{R}}^T)$ is the hypothesis space of the weak classifiers. q represents the structure of each weak classifier. T is the number of leaf nodes in a decision tree or regression tree.

$$\begin{array}{c}\hfill \mathcal{L}=\sum _{i=1}^{N}l({\widehat{y}}_i,y_i)+\sum _{k=1}^K\mathsf{\Omega }\left(f_k\right).\end{array}$$

(6)

where $\mathsf{\Omega }\left(f\right)=\gamma T+\frac{1}{2}\lambda {\parallel w\parallel }^2$ is a regular term that limits the complexity of the model, thus ensuring that the model will not be over-fitting. By using Taylor expansion to expand to the second order, Equation (6) can be approximately expressed by the following form

$$\begin{array}{c}\hfill \mathcal{L}\simeq \sum _{i=1}^N[l(y_i,{\widehat{y}}_i^{(t-1)})+g_i{f}_t\left(x_i\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)]+\sum _{k=1}^K\mathsf{\Omega }\left(f_k\right).\end{array}$$

(7)

where $g_i={\mathcal{L}}^{\prime }(y_i,{\widehat{y}}_i^{(t-1)})$ and $h_i={\mathcal{L}}^{\prime \prime }(y_i,{\widehat{y}}_i^{(t-1)})$ represent the first-order and second-order derivatives of loss function.

Since $l(y_i,{\widehat{y}}_i^{(t-1)})$ is obtained at the $t-1$-th iteration, which is a certain value at the t-th iteration, we can omit this constant term.Let $I_j=\left\{i\right|q\left(x_i\right)=j\}$ be the set of all samples in the j-th leaf node. Thus, we can obtain the expression of the loss function as follow

$${\tilde{\mathcal{L}}}^{\left(t\right)}=\sum _{j=1}^T\left[\left(\sum _{i\in I_j}{g}_i\right)w_j+\frac{1}{2}\left(\sum _{i\in I_j}{h}_j+\lambda \right)w_j^2\right]+\gamma T.$$

(8)

To obtain the optimal leaf weight $w^{\ast }$, we take the derivative of the Equation (8) and let it be equal to zero. Then we have

$$\left\{\begin{array}{cc}\hfill & W_j^{\ast }=-\frac{G_j}{H_j+\lambda },\hfill \\ \hfill & Ob{j}^{\left(t\right)\ast }=minOb{j}^{\left(t\right)}\approx \gamma T-\frac{1}{2}\sum _{j=1}^T\frac{G_j^2}{H_j+\lambda }.\hfill \end{array}\right.$$

(9)

where G and H are the cumulative values of the first derivative and the second derivative on the j-th leaf node, respectively. We can also express them mathematically as $G_j={\sum }_{i\in I_j}{g}_i$, $H_j={\sum }_{i\in I_j}{h}_j$. $Ob{j}^{\left(t\right)\ast }$ represents the optimal objective function at the t-th iteration, which is obtained by substituting the optimal weight into the objective function.

3.3. Differentially Private XGBoost

Through the above derivation process, we have obtained the optimal weight and objective function. However, this boosting algorithm process does not have the property of privacy protection. The implication is that some privacy attack algorithms can steal sensitive information about input features from the training set.

To protect the privacy of information in the original data, we consider using the Gaussian mechanism. We inject noise into the first derivative and the second derivative of the loss function (Figure 1), making it more difficult for the attacker to infer the training set information through the intermediate operator or the operation result.

$$\left\{\begin{array}{c}\hfill G\leftarrow \tilde{G}+\mathcal{N}(0,{\sigma }_G^2),\\ \hfill H\leftarrow \tilde{H}+\mathcal{N}(0,{\sigma }_H^2).\end{array}\right.$$

(10)

where $\tilde{G}={\sum }_{i\in I_j}{g}_i$ and $\tilde{H}={\sum }_{i\in I_j}{h}_i$ are the cumulative value on a leaf node of the first derivative and the second derivative with privacy-preserving property respectively. ${\sigma }_G$ and ${\sigma }_H$ are the scale parameters injecting noise into $\tilde{G}$ and $\tilde{H}$ respectively.

If we continue to divide the subtree at the leaf node, we can obtain the cumulative gradient (first derivative of loss) and Hessian (second derivative of loss) on the left leaf node and the right leaf node. We denote them as ${\tilde{G}}_L$ and ${\tilde{H}}_L$, and they satisfies the condition ${\tilde{G}}_L+{\tilde{G}}_R=\tilde{G}$. We subtract the objective function after division from the objective function before division and define it as Gain as follows.

$$\begin{array}{c}\hfill Gain=Ob{j}_{before}^{\ast }-Ob{j}_{after}^{\ast }=\frac{1}{2}\left[\frac{{({\tilde{G}}_L+{\tilde{G}}_R)}^2}{{\tilde{H}}_L+{\tilde{H}}_R+\lambda }-\left(\frac{{\tilde{G}}_L^2}{{\tilde{H}}_L+\lambda }+\frac{{\tilde{G}}_R^2}{{\tilde{H}}_R+\lambda }\right)\right]-\gamma .\end{array}$$

(11)

Since Equation (11) is an objective function, we can simplify it as follows as long as we do not change its curve shape

$$\begin{array}{c}\hfill Gain=\frac{{({\tilde{G}}_L+{\tilde{G}}_R)}^2}{{\tilde{H}}_L+{\tilde{H}}_R+\lambda }-\frac{{\tilde{G}}_L^2}{{\tilde{H}}_L+\lambda }-\frac{{\tilde{G}}_R^2}{{\tilde{H}}_R+\lambda }\end{array}$$

(12)

Then we can outline the main steps of our Differentially Private XGBoost Algorithm in Algorithm 1.

3.4. Privacy Computing

Since the boosting algorithm needs to be calculated through multiple iterations, we need to calculate the cumulative value of the privacy budget through composition theorem.

Lemma 1

(Basic Composition [39]). Let a series of randomized algorithm ${\mathcal{M}}_i,i=1,\cdots ,t$ satisfies $({ϵ}_i,{\delta }_i)$-differential privacy. Then $\mathcal{M}=\{{\mathcal{M}}_1,\cdots ,{\mathcal{M}}_t\}$ satisfies $({\sum }_{i=1}^t{ϵ}_i,{\sum }_{i=1}^t{\delta }_i)$-differential privacy.

According to the property of the Gaussian mechanism (see Equation (4)), we can calculate that the privacy budget provided by injecting Gaussian noise with scale parameter $\sigma$ once is $ϵ=\sqrt{2ln\left(\frac{1.25}{\delta }\right)}·\frac{{\Delta }_2}{\sigma }.$ Then, according to the basic composition theorem, we can obtain the cumulative privacy budget during t iterations as

$$\begin{array}{c}\hfill \sum _{i=1}^t\sqrt{2ln\left(\frac{1.25}{\delta }\right)}·\frac{{\Delta }_2}{\sigma }.\end{array}$$

(13)

Algorithm 1 Differentially Privated XGBoost Algorithm

Input:    The set of instances for current node I;        The customized scale of Gaussian noise ${\sigma }_g$, ${\sigma }_h$; Output: 1: $\tilde{G}\leftarrow {\sum }_{i\in I}{g}_i,\tilde{H}\leftarrow {\sum }_{i\in I}{h}_i$. 2: Inject noise in gradient $G\leftarrow \tilde{G}+\mathcal{N}(0,{\sigma }_G^2)$ 3: Inject noise in Hessian $H\leftarrow \tilde{H}+\mathcal{N}(0,{\sigma }_H^2)$ 4: for each $k\in [1,m]$ do 5:    $G_L\leftarrow 0$, $H_L\leftarrow 0$; 6:    for j in $sorted(I,by{x}_{jk})$ do 7:      Compute gradient of loss function $g_j\leftarrow {\partial }_{{\widehat{y}}_i^{(t-1)}}l\left(y_i,{\widehat{y}}_i^{(t-1)}\right)$ 8:      Compute Hessian matrix of loss function $h_j\leftarrow {\partial }_{{\widehat{y}}_i^{(t-1)}}^{2}l\left(y_i,{\widehat{y}}_i^{(t-1)}\right)$ 9:      $G_L\leftarrow G_L+g_j$, $H_L\leftarrow H_L+h_j$ 10:      $G_R\leftarrow G-G_L$, $H_R\leftarrow H-H_L$ 11:      $Gain\leftarrow \frac{{({\tilde{G}}_L+{\tilde{G}}_R)}^2}{{\tilde{H}}_L+{\tilde{H}}_R+\lambda }-\frac{{\tilde{G}}_L^2}{{\tilde{H}}_L+\lambda }-\frac{{\tilde{G}}_R^2}{{\tilde{H}}_R+\lambda }$ 12:      Determine whether to divide leaf nodes by $Gain$. 13:    end for 14: end for 15: return A privacy-preserving decision tree satisfies $(ϵ,\delta )$-DP, cumulative Privacy Budget $ϵ$, failure probability $\delta$.

Although the cumulative privacy budget in multiple iterations can be calculated through Equation (13). However, basic composition is usually not efficient, which means that we are likely to overestimate the cumulative privacy budget. Therefore, we introduce the advanced composition theorem, expecting to achieve a tighter privacy budget.

Lemma 2

(Advanced Composition [40]). Suppose $ϵ,\delta ,{\delta }^{\prime }\ge 0$ and ${\mathcal{M}}_i,i=1,\cdots ,t$ satisfies $(ϵ,\delta )$-differential privacy. Then $\mathcal{M}=\{{\mathcal{M}}_1,\cdots ,{\mathcal{M}}_t\}$ satisfies $({ϵ}^{\prime },t\delta +{\delta }^{\prime })$-differential privacy and

$$\begin{array}{c}\hfill {ϵ}^{\prime }=\sqrt{2tln(2/{\delta }^{\prime })ϵ}+tϵ(e^{ϵ}-1).\end{array}$$

(14)

Similarly, we can also obtain the cumulative privacy budget during t iterations from Lemma 2 as

$$\begin{array}{c}\hfill \sqrt{2tln(2/{\delta }^{\prime })\sqrt{2ln\left(\frac{1.25}{\delta }\right)}·\frac{{\Delta }_2}{\sigma }}+t\sqrt{2ln\left(\frac{1.25}{\delta }\right)}·\frac{{\Delta }_2}{\sigma }\left(e^{\sqrt{2ln\left(1.25/\delta \right)}·\frac{{\Delta }_2}{\sigma }}-1\right)\end{array}$$

(15)

Comparing with Equation (15), the advantages of the Equation (13) will continue to appear with the increase of iterations.

4. Experiments

4.1. Data Collection and Preprocessing

The experimental samples were selected during the rice harvest period in 2020, and 340 samples including Qijing 11, Longyang 16, Longdao 18, and Longqingdao 21 were collected by a five-point field sampling method. Sample information is shown in

Table 1. Rice sampling information for traceability experiments under differential privacy guarantee.

Variety	Origin	Amount	Number
Longyang 16	Wuchang, Heilongjiang, China	50kg	LY16
Longdao 18	Harbin, Heilongjiang, China	50kg	LD18
Longqingdao 21	Daqing, Heilongjiang, China	50kg	LQ21
Qijing 11	Daqing, Heilongjiang, China	50kg	QJ11

.

The flow chart is shown in Figure 2. The collected samples were processed by air in the laboratory, and then removed impurities, selected, husked into brown rice, and ground into white rice. The processing accuracy was in line with the first-grade national standards. The moisture content of the samples was controlled below 14%, and the samples were sealed and stored in a ziplock bag and stored in a refrigerator at 4 ℃. During measurement, the external detector of the diffuse Fourier near-infrared spectrometer is connected, and the rotating measuring device is placed on the top. The sample is put into the optical plate, and the bottom of the plate is cleaned with a mirror wiping paper. During the test, it is necessary to ensure that the rotary measuring device rotates at a constant speed.

To reduce the interference of the instrument itself, it is necessary to preheat 2 h in advance before measuring the instrument. The ambient temperature of the experiment is (25 ± 1) °C, the relative humidity is 25–35%, the spectral wave number range is 12,000∼4000 cm${}^{-1}$, and the resolution is 8 cm${}^{-1}$. The scanning is repeated 64 times. Calibrate the signal and save the peak level to eliminate the noise interference generated by the operation of the instrument.

4.2. Analysis of Near-Infrared Spectrogram of Rice

The absorption of the near-infrared spectrum is mainly caused by the change of vibration state inside the molecule. The frequency characteristics of molecular vibration determine the information range and characteristics of near-infrared spectrum analysis.

Chinese rice varieties are rich, and different varieties of rice genes are different, which will affect the internal chemical composition of rice content, such as sugar, protein, fat, and other subtle differences, the taste and quality of rice will be very different, which will also lead to different varieties of rice to the absorption of near-infrared spectrum differences. Figure 3 shows the near-infrared spectra of four varieties of rice.

It can be seen from Figure 3 that the four varieties of rice samples have the same absorption peak in the same wave-number range, but the absorption intensity is different. The absorption peaks in the wave-number range of 9000∼8000 cm${}^{-1}$ represent $C{H}_3$ in aliphatic hydrocarbons. The absorption peaks in the wavenumber range of 7000∼6000 cm${}^{-1}$ represent the information of free NH groups, which can reflect the type and content difference of amino acids in the sample. The absorption peak in the wave-number range of 5000∼4000 cm${}^{-1}$ is in the C-H first combination spectrum region, including free OH, CH${}_2$, and CH${}_3$, which can characterize the protein and amylose contents in rice samples.

4.3. Evaluation on Privacy

Next, we study the effectiveness of our DP-XGB algorithm at the level of privacy protection. We compare the changes in the privacy budget with the number of iterations under the Gaussian mechanism with different scale parameters. The results are shown in Figure 4a. We find that the privacy budget decreases with the increasing intensity of Gaussian noise, which indicates that the privacy guarantee is strengthened after this change.

In addition, we conducted experiments to continuously increase the scale parameters under different failure probabilities. The results in Figure 4b not only show that the privacy budget decreases with the increase of scale parameters, but also show that appropriately tolerating higher failure probability can reduce the privacy budget to a certain extent. Therefore, we can adjust the scale and failure probability appropriately to ensure differential privacy within a reasonable range.

4.4. Evaluation on Utility

We compared the attack performance of DP-XGBoost and several integrated learning methods with Gaussian mechanism, including: (1) Random Forest (RF): the most popular bagging method constructed by decision trees, (2) AdaBoost: the most common boosting algorithm composed of decision trees. We stipulated that the depth of each tree should not exceed 5. We compare the performance of these algorithms on 3-classification tasks and 4-classification tasks respectively.

In 3-classification tasks, the highest accuracy is 100% for both AdaBoost and DP-XGBoost when $\sigma$ = 0.001, higher than the performance of Random Forest. When $\sigma$ = 0.002, DP-XGBoost achieve the accuracy of 96.8%, higher than 93.5% for RF, and 91.9% for AdaBoost . When it comes to 4-classification tasks, DP-XGBoost has the highest accuracy for both $\sigma$ = 0.001 and $\sigma$ = 0.002. The above results show that our DP-XGBoost outperforms Random Forest and AdaBoost on the task of privacy-preserving rice traceability (See

Table 2. Comparative experiment of privacy-preserving algorithms on the task of rice traceability, where GM is the abbreviation of Gaussian mechanism.

Type	Scale	RF+GM	AdaBoost+GM	DP-XGB
3-class	$\sigma =0.001$	96.8%	100.0%	100.0%
	$\sigma =0.002$	93.5%	91.9%	96.8%
4-class	$\sigma =0.001$	81.7%	80.5%	82.9%
	$\sigma =0.002$	69.5%	73.2%	76.8%

).

4.5. Ablation Study

To better explore the impacts of differential privacy on the utility of our algorithm, we propose the ablation study of the first-order operator and the second-order operator. We find that when the differential privacy guarantee of the first-order operator is removed, the classification accuracy of the model does not show a significant downward trend with the increase of noise intensity. However, when the differential privacy guarantee of the second-order operator is deleted, the changing trend of the curve is similar to that of using the differential privacy guarantee in the first-order operator and the second-order operator at the same time.

The above phenomena show that the main reason for the decline of the utility of the model is the differential privacy protection for the first-order operators. This is an empirical phenomenon, because usually the absolute value of the first-order operator is much larger than that of the second-order operator. In practice, scale $\sigma$ should be set below 0.4 to ensure that the accuracy of the rice traceability model reaches more than 94%, which corresponds to the privacy budget of $ϵ=12.11$. Nevertheless, the privacy protection of the first-order operator is far more important than the second-order operator, because it is easier to infer or recover information about the original data from the first-order operator [41] (See Figure 5).

5. Conclusions

In this paper, we propose a private machine learning method for the traceability of rice varieties, named DP-XGB. We suggest applying differential privacy as a measure of the risk of privacy leakage, and endowing the model with the function of privacy protection through the Gaussian mechanism. More specifically, we select the first derivative and the second derivative of the loss function generated in the iterative process of the XGBoost model perturbed by random noise that follows Gaussian distribution, so that the model can obtain a privacy guarantee in the traceability task. Experiments on the data we collected verified that DP-XGB has the privacy-preserving property of a differential privacy guarantee. In addition, we also give a method to calculate the cumulative privacy budget, and show how to achieve different levels of differential privacy guarantee by adjusting parameters.