PriRepVGG: Privacy-Preserving 3-Party Inference Framework for Image-Based Defect Detection

Abstract

Image classification is widely used in industrial defect detection, medical diagnosis, social welfare, and other fields, in which privacy and security of models and data must be involved. For example, in diamond synthesis, the diamond substrate image annotation data and the defect detection model are of value for conservation. Based on ensuring inference efficiency and the security of these private data intellectual property, the 3-party secure inference based on secure multi-party computation (MPC) can be adopted. MPC allows parties to use neural networks while preserving their input privacy for collaborative computing, but it will lead to huge communication and memory consumption. This paper propose PriRepVGG, a lightweight privacy-preserving image-based defect detection framework for 3-party. In this work, firstly, This work optimized the division and added an AdaptiveAvgpool layer in MPC framework FALCON; then, This work ported the inference architecture of the RegVGG network into FALCON creatively. Our work applied PriRepVGG to the secure inference of the diamond substrates defect detection under the data server, model server, and compute server settings, which can be carried out in batches with a low misjudgment rate and verify the feasibility of image-based secure inference with a lightweight network in an industrial case under MPC.

1. Introduction

MPC was proposed by Yao in 1982 with the millionaire’s question [1]. MPC makes sure that participants receive the public function’s output in accordance with the protocol but cannot learn any intermediate information in order to reveal the input of other participants or any valuable private information. Data are the foundation for machine learning, and MPC offers a secure computing environment for data using, making it possible to combine the data assets of various parties to train more precise models, which is beneficial for the practical application of machine learning. The focus of MPC research has shifted to support secure computation frameworks for neural networks. By designing cryptographic protocols for linear functions as well as nonlinear activation functions for each layer of neural networks, MPC can protect the data assets of the parties involved in the training process. At present, frameworks have been able to provide secure training and secure inference for deep neural networks (DNN) and convolutional neural networks (CNN), such as security inference based on CryptFlow [2] for detecting lung diseases based on chest X-ray images and using CT images for tumor diagnosis [3]; SecureNN [4] supports security training and secure inference as a 3-party MPC framework with semi-honest; ABY3 [5] supports MPC protocols with the conversion of Arithmetic sharing, Yao’s sharing, and Boolean sharing; MinioNN [6], a privacy-preserving inference framework based on Oblivious Transfer (OT); in addition, Crypten [7], a MPC software framework with Pytorch that supports secure inference with GPU; EPIC [8], an image classification framework based on MPC using SVM et al. Although MPC frameworks already support secure inference, the practical application of machine learning as a service in industrial cases needs to be supplemented. The instantiation of this work improves the automation and detection level of diamond synthesis, while protecting data security of both researchers and practitioners.

Synthetic diamonds are synthesized by chemical vapor deposition (CVD) [9]. CVD is a process where carbon atoms are deposited from carbon gases under the action of high temperature plasma by a chemical reaction occurring on or in the vicinity of a heated substrate surface. In the process of producing synthetic diamonds, the surface quality inspection of substrate material is relying on experience of experts. The surface quality of the single crystal substrate will be the most important factor, which influences the quality of the epitaxial growth. Therefore, the quality of the synthetic diamonds is determined by the surface quality of the single crystal substrate. The images of the substrate are used to judge the growth quality of diamond. Identification of diamond substrate requires a high degree of knowledge reserve and experience, and the detection process is complex, which will lead to low efficiency of detection. However, the research on diamond substrate cultivation and defect detection technology is relatively exclusive, such as due to the protection of diamond substrate images and labels, commercial partners are not expected to get any data of diamond substrate, which seriously hinders the application of the research progress in the field of computer vision to the diamond defect detection task. At the same time, companies with advanced detection technology do not want their models to be leaked. Therefore, it is of great application value to use the MPC to realize the diamond substrate defect detection under the premise of protecting the data privacy of each party, as Figure 1.

2. Overview

2.1. Service Scenarios

In diamond synthesis, on the one hand, modeling diamond substrate defects necessitates engineers with expert experience in cultivation and inspection. In particular, inspection engineers and cultivation engineers model several typical diamond substrate defect types by observing potential characteristics influencing diamond substrate regrowth, locating, photographing, and recording, and then cultivating, growing, and judging again to summarize the shape and grade of defects that are present, a process that is accomplished through continuous experimental accumulation. On the other hand, for the purpose of high-quality synthesis, there are some requirements for diamond substrate defect detection of enterprises, but they are not allowed to expose their diamond substrate images due to the fact they must protect the commercial value of these images.

In this instance, The provider of the diamond substrate images is the data holder, and the engineer who summarizes and trains models is the model holder. Considering the inference efficiency, there is a trusted third party as a compute server. To perform secure inference under the conditions of the three-party server service, This paper employs FALCON to convert the private data of each party into secret shares and uses it as the privacy input of the PriRepVGG framework.

2.2. Contributions

Notably, our work mainly focuses on the construction of PriRepVGG and improvement of MPC protocols in FALCON. Our contributions are summarized as follows:

(1)

This paper proposes a 3-party secure inference framework for image-based defect detection, PriRepVGG. This paper verifies the feasibility of PriRepVGG in the industrial diamond synthesis case, which also contributes to the integration of MPC in industrial defect detection technology.

(2)

This paper optimizes the division and its subprotocols based on the Goldschmidt algorithm. In terms of operation efficiency, the protocol in this work is $2x$ and the effective range of division operation is increased by at least four orders of magnitude compared to the origin work in FALCON.

(3)

This paper improves FALCON compatible with more neural networks by adding the Avgpool protocol and utilizing it to implement the AdaptiveAvgpoolLayer to construct PriRepVGG. It helps FALCON to be integrated with more sophisticated neural networks.

2.3. Organization of the Paper

Subsequently, This paper will discuss the related work. After elaborating the nesessary preliminaries of the protocols in Section 4, Section 5 will describe the protocols in detail. Finally, This paper provides experiments and analysis in Section 6.

3. Related Work

The variety of defects makes it difficult to use unified mathematical methods or traditional machine learning methods to distinguish and locate the features of defects while avoiding model overfitting. The use of typical CNN such as AlexNet [10], VGG [11], and ResNet [12] for defect detection cannot meet the industrial needs. Lightweight CNNs such as MobileNets [13,14], ShuffleNet [15], and RepVGG [16] gradually come out. In particular, RepVGG can use parameter refactoring to convert the parameters of the training model into the parameters of the inference network of the plain structure. The inference architecture incorporates the original batch normalization (BN) layer of the training framework, which can prevent overfitting of the model. Because there are complex operators such as division and square root in the BN layer, simplifying the BN layer is beneficial to reduce the computational complexity of secure inference, and the effect of network inference still maintains the level of inference of the training model. In addition, in need of privacy protection, there has been some work about data security. For example, Liu [17] adopts Paillier encryption to realize classification and recognition method of encrypted electroencephalography data based on a neural network; software defect prediction using differential privacy and deep learning [18,19], etc. Based on the service scenario discussed in Section 2.1, this paper uses the MPC framework FALCON [20] to protect the privacy of data holders and model holders at the same time. This MPC framework combines the features of SecureNN and ABY3, using the replicated secret sharing (RSS). It is superior to the previous work [4,5,21] in terms of time and communication cost for secure inference, and has a distinct advantage for the secure two-party computing framework [6,22,23].

Division is a fundamental but crucial arithmetic operator in MPC; it acts as the core component of several activation function and pooling layers. A number of MPC frameworks have proposed their own different implementations of division: division in SecureML [24] used a circuit based on the cryptographic tool EMP toolkit [25]; SecureNN [4], which constructs a subtractive division based on the underlying secure operators, and ABY2 [26] used a division implemented in a garbled circuit; in addition, division can be implemented in MPC using functional iteration, such as Crypten [7], which uses a Newton–Raphson iteration method to implement division. SIRNN [27] and FALCON are based on the Goldschmidt [28] method to realize division; and initial approximation is vital in functional iteration. In order to determine an initial approximation, SIRNN and FALCON construct the MSNZB and the POW algorithm to deflate the initial values (denominators). Thus, as for the division algorithm, their implementations are probably divided into the following two ways:

• Repeated Subtraction, also known as Subtractive. Assume that until the remainder is zero or a lesser amount than the number being eliminated, the exact number is continually subtracted from the other more important digit; it is computationally intensive when high accuracy is required.
• Functional iteration is more suitable for secure computation with division. such as Newton–Raphson and Goldschmidt are used in division. The limitation of these methods is that the initial value requirements are relatively harsh, but the advantage is that the second order iteration is faster compared to the repeated subtraction.

However, the iteration rounds of division depends on the approximation degree of the initial value, that is, there is a trade-off between the complexity of implementing the approximation function with a large range of digital inverse in MPC and the iteration rounds of division. Therefore, if the initial approximation can be determined precisely, then the rounds of iterations will be reduced, which in turn will improve the efficiency of the division algorithm.

4. Preliminaries

Notation $P_0,P_1,P_2$ are used to represent the parties involved in the protocol, which denote the compute server, model server and data server, respectively. In addition, the algorithm needs to know that $P_0$ is the next party of $P_2$ and the previous party of $P_1$ is $P_0$. To ensure protocol inheritance, the algorithm uses the notation described in FALCON. Use prefix $\mathsf{\Pi }$ to denote a function with participation of $P_0,P_1$ and $P_2$. This paper uses $⟦x{⟧}^m$ to denote the replicated secret sharing (RSS) of modulo m for a general modulus m, such as the arithmetic share $⟦x{⟧}^L$ or the boolean share value $⟦x{⟧}^2$, and uses $x\left[i\right]$ to denote the i th component of vector x. All of the protocols are based on three specific rings which are ${\mathbb{Z}}_L$, ${\mathbb{Z}}_p$ and ${\mathbb{Z}}_2$, where (L = $2^{\ell }$), ℓ = 64, p = 67. This work uses a fixed-point parameter f with 13, which is for scaling shares $x^{\prime }$ forming a real value x, where $x=x^{\prime }·2^{-f}$. The fixed-point parameter will be used in multiplication for truncating the result.

4.1. Threat Model and Security

The threat model in this paper is similar to previous work, like GAZELLE [23] and SecureML [24]. The work combines the consideration of the application scenario and chooses to design the protocol based on semi-honest security with an honest-majority. A semi-honest adversary is described as honest and curious because the adversary will follow the protocol instructions honestly but attempt to learn the private input of other parties. This security model can guarantee the security when a semi-honest adversary corrupts one single party. Since the protocols in this work invoke a number of sub-protocols which have proved the security in the prior work FALCON in the real world-ideal word simulation paradigm, and the security of the protocols is implied supported with previous standard composition theorems [29,30].

4.2. Secret Sharing Scheme

Secret sharing is a crucial security primitive of MPC, and it is the foundation of various two-party and three-party protocols. Two-party frameworks such as Cryptflow2 and SecureML use a 2-out-of-2 additive secret sharing scheme. Three-party framework such as SecureNN also uses 2-out-of-2 additive secret sharing between two parties with a third assist party that only provides relevant randomness for two-party protocols.

A t-out-of-n threshold secret sharing scheme (TSSS) [31] consists of two algorithms: Share and Reconstruct. Share scheme means a randomized algorithm that takes a message x in ${\mathbb{Z}}_L$ as the input and outputs a sequence a = $(a_1,a_2,\dots ,a_n)$ of $shares$. The reconstruct scheme means a deterministic algorithm that takes a collection of t or more $shares$ of a as the input, and outputs the message x. In a word, TSSS is that among n parties that hold secret shares, and at least t of them are able to restore the secret.

FALCON combines SecureNN and ABY3 [5] frameworks using 2-out-of-3 secret sharing scheme, which can tolerate at most corruption of one party. The RSS scheme can help three-party protocols be more efficient and allow more offline computing. $⟦x{⟧}^m$ denotes an RSS of x with module m. In fact, $⟦x{⟧}^m$ demonstrates that $(x_0,x_1)$ is held by $P_0$, $(x_1,x_2)$ is held by $P_1$, and $(x_2,x_0)$ is held by $P_2$. The paper assumes that there are two private inputs $\{x,y\}$, the process of converting the 3-out-of-3 secret sharing to the 2-out-of-3 RSS shows in Figure 2.

4.3. Basic Operations

The MPC-based secret sharing protocols do not need to reconstruct the shares in the calculations. To construct new protocols based on the RSS scheme, this paper will call the sub-protocol of the original work to implement the new protocol on a partial step. In this section, the paper describes the basic operations the algorithms needed.

Selectshare$\left({\mathsf{\Pi }}_{\mathsf{SS}}(P_1,P_2,P_3)\right)$: ${\mathsf{\Pi }}_{\mathsf{SS}}$ is a critical sub-routine for majority protocols. It takes two shares $⟦a{⟧}^L,⟦b{⟧}^L$ as input, for which $⟦b{⟧}^2$ is a select bit. The output c is either a or 0 depending on b. Set $⟦c{⟧}^L\leftarrow {\mathsf{\Pi }}_{\mathsf{SS}}$: Input form $\{⟦a{⟧}^L,⟦b{⟧}^L,⟦c{⟧}^L\}$, set $c=a$ if $b=0$; otherwise, set $c=0$ if $b=1$; the algorithm can abstract this formula as $⟦c{⟧}^L$ = (1 $-⟦b{⟧}^2$) $⟦a{⟧}^L$.

ReLU$\left({\mathsf{\Pi }}_{\mathsf{ReLU}}(P_1,P_2,P_3)\right)$: ReLU is a necessary component of protocol design and network construction, and it is a significant activation function used in neural networks; ${\mathsf{\Pi }}_{\mathsf{ReLU}}$ takes one share $⟦a{⟧}^L$ as input, the output $⟦b{⟧}^L$ is $⟦a{⟧}^L$ if $a>=0$ and 0, otherwise. Set $⟦b{⟧}^L\leftarrow {\mathsf{\Pi }}_{\mathsf{ReLU}}$: set $b=0$ if $a<0$ and set $b=a$ if $a>=0$

DReLU$\left({\mathsf{\Pi }}_{\mathsf{DReLU}}(P_1,P_2,P_3)\right)$: DReLU is a derivative of ReLu, for the reason that the paper uses ${\mathbb{Z}}_L$ to perform fixed point arithmetic. It notes that the number defines the positive and negative by the first bit, so it is known that the positive numbers are the first $2^{L-1}$ numbers; therefore, the most significant bits of positive numbers are 0 and negative numbers otherwise. The relation between DReLU and ReLU can be written as:

$$\mathsf{DReLU}\left(\mathsf{a}\right)=\mathsf{a}·\mathsf{ReLU}\left(\mathsf{a}\right)$$

(1)

Set $⟦b{⟧}^L\leftarrow {\mathsf{\Pi }}_{\mathsf{DReLU}}$: Set $b=1$ if $a>0$ and Set $b=0$ if $a<0$.

Pow$\left({\mathsf{\Pi }}_{\mathsf{Pow}}(P_1,P_2,P_3)\right)$: ${\mathsf{\Pi }}_{\mathsf{Pow}}$ is used for estimating the scaling factor to calculate the initial value of division and the rsqrt algorithm; it can always be combined with the truncation function to determine a suitable initial value of the denominator. It takes one share $⟦a{⟧}^L$ as an input and the output $⟦b{⟧}^L$, which is the bounding power of $⟦a{⟧}^L$. Set $⟦b{⟧}^L\leftarrow {\mathsf{\Pi }}_{\mathsf{Pow}}$: Input $⟦a{⟧}^L$ in ${\mathbb{Z}}_L$, then get $⟦b{⟧}^L$ in clear as output, where $a\in (2^b,2^{b+1})$

5. Algorithm Construct

In this section, the paper constructs three algorithms used in the work. The notation, security, basic operations, and secret sharing scheme have been elaborated in Section 4.

5.1. Condition Select

Previous work has implemented similar protocols for conditional selecting like Multiplexer and 1-of-2 select. Conditional selecting protocol functions as a secure if-else judgment statement so the algorithm implements this protocol in FALCON. The paper uses ${\mathsf{\Pi }}_{\mathsf{SS}}$ ($P_1,P_2,P_3$) to realize the function ${\mathsf{\Pi }}_{\mathsf{CS}}$, as Algorithm 1. The formula of ${\mathsf{\Pi }}_{\mathsf{CS}}$ can be described as $⟦d{⟧}^L=⟦(1-c)·a{⟧}^L+⟦c·b{⟧}^L$, which means that, when choosing bit $c=0$, outputs a and, when $c=1$, outputs b.

Algorithm 1 Condition Select ${\mathsf{\Pi }}_{\mathsf{CS}}(P_0,P_1,P_2)$:
Require: $P_0,P_1,P_2$ hold secret shares of $a,b$ and condition $\gamma$; Ensure: All parties get shares of the result c, which is chosen from the inputs by the condition $\gamma$
	1: Set ${\gamma }_0=1-\gamma$ 2: Compute $\tau \leftarrow {\mathsf{\Pi }}_{\mathsf{SS}}$ on inputs $\{a,\gamma ,\mathbf{Ø}\}$. 3: Compute ${\tau }^{\prime }\leftarrow {\mathsf{\Pi }}_{\mathsf{SS}}$ on inputs $\{b,{\gamma }_0,{\mathbf{Ø}}^{\prime }\}$. 4: Compute $c={\tau }^{\prime }+\tau$  5: return c;

5.2. Division

Our work use Goldschmidt’s [28,32] algorithm to present a new secret division protocol. The protocol assumed when the numerator and denominator both are secret. The Goldschmidt’s method offers quadratic convergence, and the multiplications in the algorithm iterations can be computed concurrently. We analyze the form of secret division as:

$$\frac{numerator}{denominator}=\frac{a}{b}$$

(2)

Both the numerator and denominator are secret shares. In the construction of division, the algorithm needs to transform the denominator b as:

$$\frac{a}{b}\to \frac{a_0}{b_0}=\frac{a_0}{1+y}$$

(3)

The transformation is to convert the formula to a Taylor series. Then, the algorithm can determine the value of $b_0$ to be scaled with $b_0\in (0,2)$ to ensure the right iteration conditions, which is more flexible than the previous work for initial approximation:

$$\frac{1}{1+y}=\sum _{n=0}^{\infty }{(-1)}^n{y}^n,y\in (-1,1)$$

(4)

We can naturally convert Equation (4) to the formula as follows:

$$\sum _{n=0}^{\infty }{(-1)}^n{y}^n=(1-y)(1+y^2+y^4+y^8+\cdots )=(1-y)(1+y^2)(1+y^4+y^8+\cdots )=(1-y)(1+y^2)(1+y^4)\cdots$$

(5)

The algorithm uses Equation (5) to obtain the approximation $a/b$ by multiplying it with the numerator and denominator at the same time. After multiplying, the denominator becomes 1 and then the algorithm obtains the quotient approximation. The paper has shown the iteration functions in Algorithm 2 with setting $m_0=1-y$ and using $m=2-b$ to update the iteration variate. In the ith iterations, the factor m is as Equations (6) and (7):

$$(1-y^2)=(1-y)(1+y)$$

(6)

$$(1-y^{2^i})=(1-y^{2^{i-1}})(1+y^{2^{i-1}})$$

(7)

The main hurdle for secure division is to determine an appropriate initial approximation. In order to make the division algorithm more adaptive in different computing scenarios, the paper uses global variables to control the LoopCounter $\kappa$ and initial factor $\eta$ to distinguish a different denominator range. All the settings depend on the public information and take effect when the program is precompiling. The paper scales the denominator range into three level intervals and set parameters, respectively: $\left(1\right)$$(0,512),(\kappa =1,\eta =0.0039625)$. $\left(2\right)$ $(0,1024),(\kappa =1,\eta =0.00198125)$, and the algorithm sets precise parameters to optimize computational speed and communication in the above range interval while maintaining accuracy; $\left(3\right)$ As for $(0,2^f),(\kappa =5,\eta =0.0625)$, our design principle is aiming for a larger effective divisor range and computational accuracy. Considering that the fix-pointed precision is $2^{-13}$, which is close to $1.2\times {10}^{-4}$, the algorithm tries to reduce the $1bit$ [24] error due to the calculation approaching this bound; The range of calculations is too large to be of practical significance. Based on parameters such as ℓ = 64 and the fixed-point precision, the maximum computation boundary is $2^f$. Hence, the selection of the initial value of the iteration needs to consider the size of the denominator on which the scene depends. Our algorithm is able to guarantee a reasonable deflation of each element of the required initial values vector, and with vectorisation processing can make the final output vector with a vector of scaling factors maintain the same size as the input vector, and the algorithm as shown in Algorithm 2.

Algorithm 2 Division ${\mathsf{\Pi }}_{\mathsf{Div}}(P_0,P_1,P_2)$:
Require: $P_0,P_1,P_2$ hold secret shares of $a,b$ in ${\mathbb{Z}}_L$ and Loop Counter $\kappa$, initial factor $\eta$ are   global settings to initialize dividend; Ensure:$P_0,P_1,P_2$ get shares of the result $a/b$ in ${\mathbb{Z}}_L$.
	1: Set $cond\leftarrow 2$ 2: Initialize $a_0\leftarrow a$, $b_0\leftarrow b$ and $\lambda \leftarrow 1$. 3: for$i=\{1,\dots ,\kappa \}$do 4:    Compute $\gamma \leftarrow {\mathsf{\Pi }}_{\mathsf{DReLu}}$ on inputs $\{cond-b_0\}$. 5:    Compute $b_0\leftarrow {\mathsf{\Pi }}_{\mathsf{CS}}$ on inputs $\{b_0,\eta b_0,\gamma \}$. 6:    Compute $\lambda \leftarrow {\mathsf{\Pi }}_{\mathsf{CS}}$ on inputs $\{\lambda ,\eta \lambda ,\gamma \}$. 7: end for 8: Compute $y\leftarrow b-1$ and $m\leftarrow 1-y$ 9: for  $i=\{1,\dots ,5\}$do 10:    Compute $a\leftarrow ma$, $b\leftarrow mb$ and $m\leftarrow 2-b$ 11: end for 12: return$\lambda a$;

Division is a necessary component in neural networks, such as the BN layer, Avgpool layer, etc. These layers determine the divisor based on an explicitly defined model structure, so truncation by the public can be used effectively to realize division if the divisor satisfies a power of 2. When division is applied to a general case above or the L1 Normalization, Min-Max Normalization in the neural network Layer, as well as some nonlinear activation functions sigmoid, tanh and some gradient descent methods, such as SGD, it should be applied with a secure division algorithm because the divisor is secret. ${\mathsf{\Pi }}_{\mathsf{Div}}(P_1,P_2,P_3)$ works with settings ℓ = 64 and $f=13$, and the paper proved the correctness of the division in secure inference using PriRepVGG in Section 6.5.

5.3. Avgpool

The paper implements Avgpool based on ${\mathsf{\Pi }}_{\mathsf{Div}}(P_1,P_2,P_3)$, as Algorithm 3, then uses it to construct the AdaptiveAvgpool layer, which can automatically calculate the kernel size of the average pool and the stride according to the output size and input size. The AdaptiveAvgpool layer can be further reduced in parameters in the network, so it is used before the final fully connected layer in RepVGG.

Algorithm 3 Avgpool ${\mathsf{\Pi }}_{\mathsf{Avgpool}}(P_0,P_1,P_2)$:
Require: $P_0,P_1,P_2$ hold shares of $a_1,a_2,\dots a_n$ in ${\mathbb{Z}}_L$, where n is the sharing form of filter size $\eta$; Ensure:$P_0,P_1,P_2$ obtain shares of the result $avg\leftarrow 1/\eta ·\sum a_i$.
	1: Initialize $sum\leftarrow a_1$ 2: for  $i=\{2,\dots ,\eta \}$do 3:    Compute $sum\leftarrow sum+a_i$. 4: end for 5: Compute $avg\leftarrow {\mathsf{\Pi }}_{\mathsf{Div}}$ on inputs $\{sum,n\}$ 6: return avg;

In neural networks, the AdaptiveAvgpool layer is popular and can reduce redundant information, enlarge the receptive field and prohibit overfitting. It is common now and will benefit FALCON to accommodate more neural networks.

6. Experiment Evaluation

The paper implements PriRepVGG, a secure inference framework built in C++ based on FALCON. All of the source code implementation of PriRepVGG will be open sourced in Github at https://github.com/Barry-ljf/PriRepVGG (accessed on 15 September 2022). The plaintext training machine configuration for this paper was performed on an IntelCore i7 processor with 16 GB of RAM and a GTX 1650 Ti GPU with a 4 GB graphics memory. The secure inference machine configuration for this paper is an Ubuntu server with 120 GB RAM and 18 cores with 72 threads. All the image dataset in this work comes from the functional diamond lab. The paper experiment with ℓ = 32 and $f=13$ for secure inference. The paper first performs data cleaning and feature processing on the dataset, and then trains the model in plaintext. After training, we extract the model to obtain all the weights and bias. Then, we transform the images and weights into shares. Finally, this work uses PriRepVGG to perform a series of secure inference experiments and record the training index of the plaintext models and compare the inference results of plaintext and ciphertext. Meanwhile, this work performed two experiments on division. First, we compare division of this paper with the original division method of the FALCON in terms of computational speed, range, and accuracy. Then, we compare the division with previous division methods based on different principles in terms of computation time and accuracy. Our division experiment environment is based on the FALCON and Rosetta and works in ℓ = 64 and $f=13$.

6.1. Division Experiment

In the process of using FALCON, it was found that the original division operation has a significant limitation for the denominator, which is mainly related to the design principle and algorithm ${\mathsf{\Pi }}_{\mathsf{Pow}}(P_1,P_2,P_3)$. In addition, mistakes always occur with vectorization processing with inputs: It is found that the division method in FALCON did not handle the results correctly when different values were taken in the vector. Thus, the work performed division experiments with Avgpool on the same denominator to ensure the origin division can normally work.

The paper compares the division in three different intervals of orders of magnitude (Range-I, Range-II, Range-III). They are $(0,512),(0,1024)$ and $(1024,2^f)$ respectively, as

Table 1. Comparison of division with the origin work in a different range.

Indicators	Range-I		Range-II		Range-III
	Ours	FALCON	Ours	FALCON	Ours	FALCON
Time(s)	0.12	0.22	0.16	0.3	0.37	0.26
Comm.(MB)	$4.32\times {10}^{-4}$	$5.08\times {10}^{-4}$	$4.32\times {10}^{-4}$	5.08 $\times {10}^{-4}$	9.32 $\times {10}^{-4}$	5.08 $\times {10}^{-4}$
Precision	${10}^{-5}$	${10}^{-5}$	${10}^{-4}$	-	${10}^{-3}$	-
Average Relative Error	2.8 $\times {10}^{-4}$	2.67 $\times {10}^{-4}$	2.67 $\times {10}^{-3}$	-	1.26 $\times {10}^{-1}$	-

. For each interval set of experiments, the algorithm took random values as input, set the divisor to be a positive floating point number to ensure that the original division method in FALCON can work, set the experimental data type length to unsigned int64, and the experiment recorded the computation time (every 20 times), the communication cost, the precision of the division, and the average relative error between the ciphertext and the plaintext division calculation. The experimental results outperform the original work in terms of efficiency and communication in the lower data range, with a speedup of nearly $2\times$. In addition, the algorithm improves the division operations being able to calculate in a larger range than the original division method—about a 4-order of magnitude increase. Then, the experiment compares the division respectively based on Goldschmidt’s method, the repeated subtraction method (SecureNN), and the Newton–Raphson iteration method (CRYPTEN) in ROSETTA, as

Table 2. Comparison of division in different methods.

Method	OPL(s)	OSL(s)	ARE	Precision
Ours	$0.116$	$1.76\times {10}^{-4}$	$1.92\times {10}^{-4}$	${10}^{-3}$
Snn	$0.697$	$2.33\times {10}^{-4}$	$1.84\times {10}^{-4}$	${10}^{-3}$
Newton	$0.216$	$2.63\times {10}^{-4}$	$2.49\times {10}^{-4}$	${10}^{-3}$

OPL: Only protocol elapsed; OSL: Only secure elapsed; ARE: Average relative error.

. The experiment results show that Goldschmidt-based implementation of the division achieves a runtime improvement of nearly $2\times$ compared to the Newton–Raphson iteration method, and about $5\times$ compared to the repeated subtraction method.

6.2. Network Construction

The paper reports the RepVGG-A0 architecture of RepVGG into FALCON. To ensure the limited memory is fully utilised, the work has redesigned the network to reduce the number of parameters in the fully connected layer by changing convolutional layers. The paper slightly adjusts the output feature map of the convolutional layer in the last stage from 1280 to 1024 in PriRepVGG. In addition, the paper implements AdaptiveAvgpool in FALCON using the Avgpool, which is constructed in Section 5.3. AdaptiveAvgpool is used as a downsampling layer before the fully connected layer. Then, the paper adds a Relu layer last in PriRepVGG to represent a negative class. The reconfigured PriRepVGG has an input resolution of $225\times 225$ and AlexNet has an input resolution of $227\times 227$. The number of layers in the AlexNet network are a total of 21 layers and 46 layers in PriRepVGG.

6.3. Model Training and Conversion

The dataset is images of typical defective features and some inclusions of perturbations that have been carefully selected by professional engineers. In addition, the paper performed data enhancement operations on all datasets, including random cropping, rotating, changing the chromaticity, etc. We trained the model with RepVGG and AlexNet, using a resolution of $225\times 225$ and $227\times 227$ as input, respectively. For AlexNet, this work uses BN layers in plaintext training to prevent overfitting and has deployed the BN layer in FALCON for AlexNet secure inference. For RepVGG, it needs to convert the training model with $3\times 3$, $1\times 1$ convolutional kernels and shortcut layers into an inference phase model with a plain structure, as shown in Figure 3. The transformed model is then used to perform plaintext inference on the validation set.The dataset was sampled randomly (Dataset-I) for validation and recorded the recall, precision, f1 score, and accuracy of the plaintext model obtained by RepVGG and AlexNet by performing the inference. In practical terms, the core situation that needs for attention is defective diamond substrates. Identifying diamond substrate pieces (positive class) as normal (negative class) would be costly in terms of manual cultivation and risk of compensation, so this work needs the recall score of the model to be stable and high. On the other hand, it is convenient to extract all defective diamond substrates for secondary checking when the model is used for model holders themselves, which would contribute to work efficiency. Like the result shown in

Table 3. Comparison of validation for RepVGG and AlexNet.

Indicators	Dataset-I		Dataset-II
	RepVGG	AlexNet	RepVGG	AlexNet
Recall	0.8281	0.7343	0.8438	0.5625
Precision	0.8412	0.8103	0.8307	0.782
F1-score	0.8346	0.7705	0.8372	0.6545
Accuracy	0.8359	0.7813	0.8359	0.7031

, the deployed inference model obtained performed relatively well, and this work also recorded the indexes for datasets with some perturbation samples (Dataset-II), which demonstrates that RepVGG’s model is more stable.

6.4. Weights Extraction and Sharing

We obtain the plaintext model from Section 6.3. Then, this work needs to extract all the weights and biases of the convolutional and fully connected layers from the plaintext model. After extracting all the weights, the model holder keeps the weights and the data holder keeps the image data in the local files; respectively; then, all data are shared as RSS to three party servers in PriRepVGG. We use $⟦x{⟧}^L$ to denote the privacy input for weight shares, and each party holds $x_0$, $x_1$, $x_2$, respectively, which satisfies $⟦x{⟧}^L=x_0+x_1+x_2$; then, all parties will share again to get a secret share about the next party’s RSS of $⟦x{⟧}^L$. At the same time, the data holder extracts its data about diamond substrate images and does the same thing as the model holder. After sharing, we obtain the RSS of $⟦y{⟧}^L$ and each party obtains $(y_0,y_1)$, $(y_1,y_2)$, $(y_2,y_0)$. The process about secret sharing has been shown in Figure 2.

6.5. Secure Inference

We use PriRepVGG for secure inference and simulate a 3-party scenario with three servers. We record the computation cost of secure inference with input resolution $225\times 225$ of PriRepVGG and $65\times 65$ of AlexNet, as

Table 4. Evaluation of secure inference in settings of batch size and ${\mathbb{Z}}_L$ for PriRepVGG and AlexNet.

Indicators	Batch Size = 32 ℓ = 32		Batch Size = 64 ℓ = 32		Batch Size = 128 ℓ = 32		Batch Size = 32 ℓ = 64
	Ours	AlexNet	Ours	AlexNet	Ours	AlexNet	Ours	AlexNet
Time (s)	5.89 $\times {10}^2$	5.95 $\times {10}^2$	1.289 $\times {10}^3$	1.269 $\times {10}^3$	2.58 $\times {10}^3$	4.83 $\times {10}^2$	7.82 $\times {10}^2$	7.61 $\times {10}^2$
Comm. (MB)	2.282 $\times {10}^3$	2.611 $\times {10}^3$	5.64 $\times {10}^3$	5.22 $\times {10}^3$	1.129 $\times {10}^4$	4.94 $\times {10}^3$	9.77 $\times {10}^3$	4.835 $\times {10}^3$
RAM(G/per node)	10	6.3	28	11.52	35	25	20	12

. Each party owns the private secret shares provided by the model holder and the data holder extracted in Section 6.4. Every single party is unable to reconstruct the origin secret in the security setting discussed in Section 4; thus, the non-data-owning party is unable to learn the data of others in the task of secure inference. The data holder will finally obtain the result of inference. We record the results of the secure inference to compute the recall, precision, f1 score and accuracy obtained by the model under ciphertext inference, and compare the result with the plaintext inference in Section 3.

As shown in Figure 4, the secure inference is stable, and the difference rate does not exceed 0.08. In addition, the accuracy of secure inference is flat compared to the accuracy in plaintext. We obtained the following detailed results: the setting of batch size of 128, and the average misjudged numbers of the plaintext model is 9. As shown in

Table 5. Record of the misjudgement for five random batch experiments.

Batch Size	I	II	III	IV	V
32	2	3	3	1	2
64	5	10	5	3	7
128	9	12	7	6	11
accuracy	0.0703125	0.09375	0.0546875	0.046875	0.0859375

the average error score rate for five random batch experiments is 0.0703.

We test AlexNet, VGG-16, and PriRepVGG to compare the performances of the three networks with input resolution $227\times 227$, $224\times 224$, and $225\times 225$. Figure 5 shows that the communication and time cost of PriRepVGG is 1–1.5 orders of magnitude lower than the VGG-16. The communication rounds of PriRepVGG is $3\times$ lower than the AlexNet shown in Figure 6. The evaluation shows that the specific plain inference architecture of PriRepVGG benefits the lightweight of parameters and computational cost reduction. In terms of the time cost for secure inference, the PriRepVGG network is under 10 to 20 s per image with the batch size growing.

7. Conclusions

Based on FALCON, This work presented a secure 3-party defect detection image-based framework PriRepVGG. We have successfully applied PriRepVGG to the task of detecting defection in industrial diamond substrates. Compared to plaintext, the secure inference with PriRepVGG performs stably, and the average error rate is less than 0.07. PriRepVGG is more lightweight then the realization of VGG in MPC. In addition, This work optimized the division algorithm of FALCON, which resulted in a $2\times$ computation speedup and a 4 order of magnitude increase in operational range over the original work. In the future, we would like to consider using PriRepVGG in more image-based secure inference scenarios with malicious settings and improve the compute and memory requirements of PriRepVGG.