A Blockchain-Based Fairness Guarantee Approach for Privacy-Preserving Collaborative Training in Computing Force Network

Abstract

The advent of the big data era has brought unprecedented data demands. The integration of computing resources with network resources in the computing force network enables the possibility of distributed collaborative training. However, unencrypted collaborative training is vulnerable to threats such as gradient inversion attacks and model theft. To address this issue, the data in collaborative training are usually protected by cryptographic methods. However, the semantic meaninglessness of encrypted data makes it difficult to prevent potential data poisoning attacks and free-riding attacks. In this paper, we propose a fairness guarantee approach for privacy-preserving collaborative training, employing blockchain technology to enable participants to share data and exclude potential violators from normal users. We utilize a cryptography-based secure aggregation method to prevent data leakage during blockchain transactions, and employ a contribution evaluation method for encrypted data to prevent data poisoning and free-riding attacks. Additionally, utilizing Shamir’s secret sharing for secret key negotiation within the group, the negotiated key is directly introduced as noise into the model, ensuring the encryption process is computationally lightweight. Decryption is efficiently achieved through the aggregation of encrypted models within the group, without incurring additional computational costs, thereby enhancing the computational efficiency of the encryption and decryption processes. Finally, the experimental results demonstrate the effectiveness and efficiency of our proposed approach.

1. Introduction

The advent of the big data era and the emergence of various data-driven technologies have brought unprecedented demands for data processing and analysis. With the integration of computing resources and network resources, the concept of the Computing Force Network (CFN) has emerged and offered possibilities for distributed collaborative training. Researchers have designed many attractive candidate technologies that can be implemented in a CFN, including the following:

• Network security: Researchers believe that active defense mechanisms can be implemented in a CFN through deep neural networks. This involves periodically adjusting the deployment locations of control nodes and mapping relationships between switch nodes and controller nodes to enhance the security objectives of the network system.
• Computational Awareness network: With the significant improvement in the accessibility and scheduling techniques of a CFN, dynamic allocation of computing resources becomes feasible. This allows for the allocation of idle resources to various tasks while reallocating resources to tasks that are better suited for them.
• Enhanced transmission technology: In a CFN, data are offloaded to the network edge, compressed, and then transmitted. This makes full use of computational resources across the network to meet the demands of massive data transmission. This involves coordinating the computing and spectral resources throughout the network to minimize system latency and deliver high-speed data services.

As illustrated in Figure 1, these technological characteristics establish the foundation for collaborative training, a novel application paradigm, by facilitating network and computational resource allocation, as well as ensuring secure data transmission. Specifically, users in a CFN can generate or process task-oriented data and subsequently provide the results to service providers for mutual benefit, necessitating that users possess valuable data and substantial data-processing capabilities [1]. This innovative model of data transaction has the potential to supplant the traditional method, where service providers directly collect data from users. Participants in this model may include both individual users and corporate entities in possession of data. In a CFN, identifying an authoritative aggregation center to facilitate the interaction among these participants poses a significant challenge. Consequently, this type of data transaction may adopt a decentralized approach.

Presently, numerous researchers are exploring the application of blockchain technology to facilitate decentralized and trusted transactions, akin to those in Bitcoin. However, the application of blockchain technology to data transactions presents several challenges [2,3,4]. To deliver and ensure the integrity of data, blockchain-based data sharing often involves storing users’ data on blocks [5], potentially leading to privacy breaches. While encryption methods safeguard data privacy, they also give rise to new challenges [6]. Owing to the semantic meaninglessness of encrypted data, malicious and dishonest users may perpetrate data poisoning and free-riding attacks [7] without detection and tracking. This situation undermines the core principles of equitable data transactions.

To achieve practical data transactions and guarantee fairness, three issues must be addressed. The first issue concerns the privacy of user data. Users must maintain adequate privacy when participating in data transactions in the CFN. If data privacy is violated in transactions, it will inevitably lead to negative long-term effects on participants’ positivity. The second issue to be addressed involves transaction fairness. If participants do not receive fair payment from the transaction, the degree of participation will decrease over the long-term operation of the system. The third issue pertains to feasibility. If user data do not contribute sufficiently to the collaborative training [8], or if the performance overhead of the data transaction system exceeds the CFN’s affordability [9], it will also make this new transaction mode difficult to implement.

In this paper, we propose a blockchain-based privacy-preserving collaborative training approach that guarantees fairness, enabling collaborative training and fair payment within the CFN. Initially, we discuss how attackers can be prevented from breaking the confidentiality of data and malicious attackers can be denied their malicious behavior through blockchain and secure aggregation methods. Based on this, the identity of the malicious attacker is accurately detected without breaking the confidentiality of normal users’ data through grouped aggregation evaluation methods.

The main contributions of the paper are as follows:

• We propose a blockchain-based approach for ensuring fairness, ingeniously integrating the anti-denial mechanism of distributed ledgers with the confidentiality assurance of secure aggregation and potential violator detection. This approach can effectively mitigate data poisoning and free-riding attacks in encrypted states.
• Our proposed method has the capability to identify poisoning attackers and free-riding attackers during the collaborative training process and exclude them from the collaborative training process. This approach reduces unfair behaviors such as malicious users reaping rewards without contributing data and computing resources and disrupting the effectiveness of collaborative training so that benefits for all participants are diminished. It ensures the benefits of legitimate users.
• We reduce the computational overhead of cryptographic methods by limiting the scale of negotiation during group formation using Shamir’s secret sharing. This approach circumvents high-computational multiplication calculations during the negotiation process, thereby accommodating the issue of limited computational capabilities of users in the CFN.
• We have developed a fine-grained potential violator detection algorithm that dynamically adjusts user groups, enabling the identification of potential violators in scenarios of dynamic user dropout during collaborative training.

In the remaining sections of this paper, we review existing research on data privacy, fairness guarantee, and usability optimization in collaborative training, and how our work performs better. Subsequently, we introduce our proposed approach and provide a comprehensive description of the fairness guarantee mechanism. Following this, security analysis and experimental evaluation are utilized to demonstrate the effectiveness and efficiency of the fairness guarantee approach. This paper concludes with a summary of the findings and implications.

2. Related Work

2.1. Data Privacy

In task-oriented data trading enabled by a CFN, addressing the privacy leakage issue is paramount [10,11,12,13]. Otherwise, this may lead to users’ reluctance to participate in collaborative training. Privacy protection in collaborative training is classified into two categories: data-level protection and information-level protection. Data-level privacy threats involve attackers directly accessing participants’ model parameters during collaborative training, with typical attack methods including model reconstruction attacks, among others. Information-level privacy threats entail attackers utilizing background knowledge and models to infer private information in the training data, with typical methods encompassing membership inference attacks, attribute inference attacks, and more.

To mitigate the aforementioned privacy threats at both the data and information levels in collaborative training, researchers have developed a variety of privacy-preserving techniques. To counter privacy threats at the data level, predominant approaches include homomorphic encryption, functional encryption, authenticated encryption, and secure multi-party computation. Bonawitz et al. [14] introduced a secure aggregation mechanism for collaborative training, employing an authenticated encryption scheme to encrypt client model parameters and utilizing a secret sharing method to deter the server from accessing specific clients’ model parameters. Consequently, So et al. [15] introduced a novel metric designed to measure privacy protection across multiple training rounds and developed a structured user selection strategy, aiming to ensure long-term privacy for individual participants across various training iterations. This strategy considers both the fairness of the selection process and the average number of participants per training round. This framework presents a trade-off between user selection and privacy preservation, thereby facilitating the optimization of model performance while ensuring an appropriate level of privacy. This approach effectively mitigates multi-round privacy leakage in federated learning and guarantees secure aggregation, thus ensuring the privacy of model parameters. To safeguard privacy at the information level, researchers typically introduce perturbations to the data or generalize the data, thereby preventing various types of inference attacks during collaborative training. Among these, Abadi et al. [16] proposed a differential privacy approach utilizing the stochastic gradient descent algorithm, which partitions the gradient and integrates noise in accordance with the differential privacy mechanism during the training process, thus thwarting various inference attacks. Yang et al. [17] proposed an innovative defense mechanism aimed at bolstering resilience against membership inference attacks in federated learning. This method involves personalizing the client’s local data distribution during training and inference stages, thereby generating an offset model. It strikes an effective balance between privacy protection and the preservation of model utility, effectively countering membership inference attacks while concurrently minimizing their impact on model accuracy.

However, data-centric privacy protection methods render traditional data quality verification approaches ineffective, as malicious actors can conceal poisoned models within encrypted data, evading quality checks targeted at individual participants. Some lazy participants may simply upload the initial models back to the server without any modifications, deceiving the rewards of collaborative training. These attacks significantly impede the fairness of collaborative training, prompting significant concern among researchers.

2.2. Fairness Guarantee

To address the previously mentioned challenges that undermine transaction fairness, researchers have devised various methods employing blockchain technology to ensure fairness. Initially, to guarantee data accuracy and ownership, Weng et al. [18] developed a data audit platform that utilizes the homomorphic Paillier encryption algorithm, complemented with a zero-knowledge proof method, certifying the legitimacy and integrity of each dataset. All audit information is recorded on the blockchain, forming a publicly searchable audit ledger, serving as a safeguard against malicious activities that undermine the system. Tian et al. [19] proposed a blockchain-based data auditing scheme aimed at establishing an effective data audit mechanism. This scheme introduces non-interactive designated decryptor function encryption and smart contract technology, achieving secure data auditing and providing indisputable arbitration mechanisms. Utilizing blockchain as a self-recording channel, it offers reliable evidence for resolving disputes between users and cloud storage service providers, thereby minimizing conflicts and ensuring undisputed arbitration. Concurrently, this scheme incorporates a set of reward and penalty measures to mitigate the malicious behavior of untrusted users and cloud storage service providers, thus ensuring the efficiency and security of the audit process. To address data quality disparities, Lu et al. [20] devised a training quality consensus scheme necessitating that participants honestly declare their data quality. An aggregator then assesses whether the quality of the aggregated result aligns with the participants’ claims, aiding in defending against malicious low-quality data submissions.

Researchers have proposed various fairness mechanisms to dynamically adjust the data sharing system in order to maintain fairness, including reputation mechanisms or contract theory, ensuring that participants with good data and credit can consistently receive more rewards. Zhou et al. [21] have implemented a reputation recording mechanism on the blockchain, enabling miners to evaluate participants’ data and subsequently reward higher data quality more. Furthermore, blockchain nodes are designed to prioritize the collection of these data. Jiao et al. [22] introduced a blockchain-based reputation mechanism that assesses participants’ historical contributions and behaviors. The mechanism assigns a reputation score to each participant, with those with higher scores being accorded more weight in the model training process, thereby incentivizing the contribution of high-quality data while restricting participants with lower scores, thus preventing the impact of malicious behavior. This approach effectively balances the need for user privacy protection with the imperative of improving data quality.

To ensure the long-term functionality of the system, guaranteeing the contribution evaluation and fair payment during collaboration is essential. Ma et al. [23] introduced a blockchain-based method to transparently assess participants’ contributions in federated learning. They utilized a novel group-based Shapley value calculation framework, compatible with secure aggregation, which enables a fair evaluation of each participant’s contribution while protecting data owners’ privacy. Implemented on the blockchain, this method ensures both the transparency and verifiability of the assessment process, as well as the effective evaluation of contributions. Chen et al. [24] developed a blockchain-based federated learning framework specifically designed to ensure fair transactions between task publishers and participants. The non-interactive designated decryptor function encryption scheme they adopted secures the privacy of training data, while the design of the blockchain ensures that participants receive corresponding rewards only after completing tasks and fulfilling requirements. Consequently, the framework strikes a balance between privacy protection, participant motivation, and the establishment of a fair payment mechanism. To address the contradiction between data auditing and the perceived irrelevance of encrypted data in the current blockchain-based framework, further research into the quality evaluation of encrypted data is imperative. Du et al. [25] proposed a blockchain-based method for assessing data quality. This method employs the Kullback–Leibler divergence to evaluate information loss between non-IID and IID data samples and uses the inverse of data quantity to simulate their marginal utility. Thus, the blockchain facilitates a transparent evaluation of data quality. Sun et al. [6] employed BCP homomorphic encryption and group aggregation auditing to evaluate the quality of encrypted data. This approach enables the identification of malicious users through iterative partitioning and group aggregation auditing without the need to decrypt individual user data.

However, existing fairness guarantee approaches often treat all participants as potential malicious attackers when auditing, resulting in significant wastage of auditing resources. Our proposed approach segregates ordinary users from those with potential malicious intent, thereby effectively enhancing the utilization of auditing resources.

2.3. Utility Optimization

To improve the utility of collaborative training in the CFN, the existing work can be categorized into two primary areas: enhancing user data availability, and optimizing system efficiency. Data availability is adjusted by balancing user privacy and data quality, with researchers incorporating game theory from economics and various game-theory-based incentive mechanisms to encourage high-quality data provision by users. Xiong et al. [26] proposed a game-theory-based incentive mechanism to achieve equilibrium between user privacy and data quality. They initially calculated users’ privacy levels with a personalized privacy metric algorithm, and then built a game model focusing on service quality and users’ personalized privacy. They analyzed Bayesian equilibrium to encourage users to submit high-quality data. Wang et al. [27] proposed a game-theory-based joint resource allocation and incentive mechanism. This mechanism is designed to address the resource allocation challenge within a blockchain-based federated learning context, wherein the model owner and the client are required to allocate computational resources in accordance with the provided rewards. It ensures fairness in reward distribution by applying the Shapley value and calculating rewards based on client contributions. This mechanism permits model owners and clients to independently determine the distribution of rewards and the allocation of computing resources, following their respective optimal strategies. Thereby, it incentivizes clients to submit high-quality data while ensuring system stability and sustainability. On the other hand, researchers have also been working on improving the efficiency of blockchain in collaborative data analysis. Qu et al. [28] proposed a federated learning scheme that combines blockchain technology with a distributed hash table to enable a data storage and exchange framework, thereby enhancing collaborative training efficiency.

However, existing utility optimization solutions for collaborative training predominantly focus on the training process itself, paying less attention to balanced solutions concerning privacy, fairness, and utility. We propose an encrypted data fairness guarantee method based on multi-party secure encryption. While leveraging multi-party secure computation to safeguard data privacy, our method also considers the use of noise addition as an alternative to homomorphic encryption, effectively reducing the computational and communication overhead incurred due to privacy protection and fairness assurance during the collaborative training process.

3. Overview of Our Proposed Approach

In this chapter, we provide an overview of our blockchain-based fairness guarantee approach for privacy-preserving collaborative training in Figure 2. Our proposed approach encompasses three roles: the service provider, the users and the blockchain.

• Service provider: The service application builders that release collaborative training tasks to the public categorize and rank the task participants (i.e., users) based on their situations. After each round of collaborative training, they examine the outcomes of groups and use this as a basis to determine if there are any issues with the users.
• Users: Physical devices with the ability to access data or an organization that collects and stores data. On the premise of ensuring data privacy, it trades data with service provider and benefits from it.
• Blockchain: decentralized distributed ledgers, which are a series of data blocks with interconnections generated using cryptography, are used to record local models uploaded by users and offer upload and download services to service providers and users.

The service provider and users transfer models via blockchain. Users are divided into general users and key users. General users are considered normal users by the service provider, while key users are considered users with a high probability of being potential violators by the service provider. The service provider conducts two rounds of basic grouping for the entire user group without distinguishing between user types. For individual key users, two additional general users are randomly selected to form a focus group. Each group consists of three users. The basic groups are used to satisfy basic audit for all users to determine whether they are key users and focus groups are used to implement fine-grained audit for key users to determine whether they are potential violators. In Figure 2, users with the same background color in a single grouping represent users in the same group.

Secure aggregation: A privacy concern in collaborative data analysis is the association between users and their respective data. Should an attacker discern that a specific data feature or model parameter is attributable to an identified participant, they could reconstruct training data or other attributes via a gradient inversion attack. To mitigate this issue, a prevalent strategy involves encrypting users’ data employing cryptographic aggregation methods like homomorphic encryption, secure multi-party computation protocols, and function encryption. Utilizing these methods, service providers can process encrypted data and only access aggregated data results and remain unaware of the specific details contributed by individual users. Encryption methods, while obstructing the association between users and their data, also conceal malicious behaviors such as data poisoning and free-riding attacks within the encrypted data and make it more challenging for service providers to assess the quality of user data and to identify potential violators. How service providers evaluate the contributions of users to learning tasks and how they detect potential violators have become significant challenges.

User detection: In collaborative training, users may engage in various malicious behaviors such as data poisoning and free-riding, while encryption hinders the association of these behaviors with their data and makes it harder for service providers to detect such malicious behaviors in an encrypted state. To facilitate the detection of users engaged in malicious behaviors like data poisoning and free-riding in an encrypted state, we employ a group detection method. This involves multiple unduplicated groupings of the entire user population and key users subjected to additional random groupings. We assess the contribution scores of users within each group by comparing the accuracy of the aggregated model for the groups with a threshold determined by the overall user situation. For users participating in a certain number of groups, we determine the presence of malicious behavior by obtaining a fine-grained assessment of their contributions through confidence intervals, thus achieving user detection in an encrypted state.

Efficiency optimization: Although CFNs integrate computing power with network resources, devices with limited computing capabilities still exist. Blockchain technology necessitates high-performance computing nodes. Although methods based on homomorphic encryption and secure multi-party computation (SMPC) can encrypt local models, the high computational overhead is burdensome for devices with limited computing capabilities. To reduce the computational demands of the encryption process and enhance the efficiency of model encryption and ciphertext aggregation, we have designed a group-based encryption method. This method involves grouping the user population, within which a secret key is negotiated using SMPC and then added to the local models for encryption. The service provider aggregates the encrypted models from each group and decrypts them to obtain the aggregated model of the group. The secret key negotiation process using SMPC is confined to within the groups, where the scale of users is not large, and neither the secret key negotiation nor the encryption process involves multiplication. Moreover, the normal model aggregation process can achieve decryption. This significantly reduces the computational overhead during the secret key negotiation and encryption/decryption processes, thereby improving efficiency.

4. Fine-Grained Potential Violator Detection Algorithm Based on Blockchain

In this chapter, we focus on the implementation of a blockchain-based fine-grained potential violator detection algorithm using cryptographic methods. This includes user grouping based on user types, intra-group encryption secret key negotiation based on Shamir’s secret sharing, encryption, aggregated decryption of local models within groups and fine-grained detection of potential violators. The overall process of the fine-grained potential violator detection algorithm is presented in Algorithm 1.

Algorithm 1: Fine-Grained Potential Violator Detection Algorithm

Input: user set $I$, key user set $KI$, number of training rounds $epochs$ Output: global model $M_{global}$, global potential violators set $P{I}_{global}$  1  Set the Potential Violators Set $PI$, and the New Key User Set $NKI$, and initialize them as emptyset and $KI$, respectively.  2  Set the initial model.  3  User grouping algorithm based on user types.  4  For $epoch$ in $\left[epochs\right]$  5    If $KI\ne NKI$ or $PI\ne \varnothing$  6      User grouping algorithm based on user type  7      $KI\leftarrow NKI$  8    End  9    For $i\in I$ 10      Distribute the initial model 11      Training a local model $m^i$ 12    End 13    For $g\in G$ 14      Intra-group encryption secret key negotiation based on Shamir’s Secret Sharing 15    End 16    For $i\in I$ 17      For $g\in G_a^i\cup G_p^i$ 18        Encryption algorithms for the local model of each user within a subgroup 19        Uploading encrypted models 20      End 21    End 22    For $g\in G$ 23      Aggregate decryption algorithms for local models within groups 24    End 25    $PI, NKI\leftarrow$ Fine-grained detection of potential violators 26    If $PI\ne \varnothing$ 27      $I\leftarrow I/PI$ 28      $P{I}_{global}\leftarrow P{I}_{global}\cup PI$ 29    End 30    $M_{global}\leftarrow$ Global model aggregation 31  End 32  Return $M_{global}$, $P{I}_{global}$

In Algorithm 1, Fine-Grained Potential Violator Detection Algorithm, the entire collaborative training process undergoes multiple rounds of training. The service provider conducts potential violator detection in each round and adjusts the key user set. Potential violators identified during the detection are removed from the collaborative training process. When changes occur in the user set or the key user set, the entire user population is regrouped and collaborative training continues.

In each round of collaborative training, the service provider groups the users based on a user-grouping algorithm based on user types and distributes the grouping information and the global model to be trained to each user. Then, each user trains their local model and negotiates encryption secret keys within their group using intra-group encryption secret key negotiation based on Shamir’s secret sharing. Subsequently, they encrypt their local models using the group-specific encryption algorithm and upload them to alliance chain. The service provider downloads cryptographic local models from alliance chain, uses aggregation decryption algorithm for local models within groups to obtain the aggregated model and conducts fine-grained detection of potential violators based on the aggregated models of each group. If potential violators are identified, they are excluded from the entire collaborative training process. Then, the server provider repeats global aggregation by group until the model converges.

4.1. User Grouping Based on User Types

The service provider categorizes users into three types: general users, key users, and potential violators. Potential violators are identified by the service provider as malicious participants based on the information the service provider holds and are denied participation in collaborative training. Key users are those whom the service provider needs to closely monitor in collaborative training to determine if they are potential violators. Depending on whether a user is considered a key user, the service provider adopts different grouping strategies for users participating in collaborative training, as detailed in Algorithm 2.

Algorithm 2: User Grouping Based on User Types
Input: all users set $I$, key user set $KI$, number of key user repetitions $KIN$ Output: basic grouping set $G_b$, focus group set $G_k$, set of active groups for each user $G_a^i, i\in I$, set of passive groups for each user $G_p^i, i\in I$  1  Initialize $G_b$, $G_k$, $G_a^i, i\in I$, $G_p^i, i\in I$ to the empty set
2    For $n$ in [2]	//Basic grouping for all users
3      For $\left(i_1,i_2,i_3\right)$ Unduplicated selection of a group of users from $I$  4      $g\leftarrow \left(i_1,i_2,i_3\right)$  5      $G_b\leftarrow G_b\cup g$  6      $G_a^{i_1}\leftarrow G_a^{i_1}\cup g, G_a^{i_2}\leftarrow G_a^{i_2}\cup g, G_a^{i_3}\leftarrow G_a^{i_3}\cup g$  7    End  8  End
9  For each $Ki\in KI$	//Focused grouping, targeting key users
10    For $n$ in $\left[KIN\right]$ 11      $Ki,i_2,i_3\in I$ 12      $g\leftarrow \left(Ki,i_2,i_3\right)$ 13      $G_k\leftarrow G_k\cup g$ 14      $G_a^{Ki}\leftarrow G_a^{Ki}\cup g, G_p^{i_2}\leftarrow G_p^{i_2}\cup g, G_p^{i_3}\leftarrow G_p^{i_3}\cup g$ 15    End 16  End 17  Return $G_b$, $G_k$, $G_a^i, i\in I$, $G_p^i, i\in I$

Algorithm 2, based on whether users are general or key users, employs different grouping strategies, with each group consisting of three users. To meet the basic contribution assessment needs, the service provider conducts two unduplicated randomized groupings of general and key users together. These groups are marked as basic groups by the service provider, and represented by $G_b$. Each user within these groups marks that group as active group, and $G_a^i$ represents the set of groups marked as active group by user $i$.

For the fine-grained assessment needs of key users, the service provider randomly selects two general users to form a group with a key user and repeats this $KNI$ times for a single key user. These groups are marked as focus group by the service provider, and represented by $G_k$. Each key user marks these groups as active group, while the selected general users mark these groups as passive group, and $G_p^i$ represents the set of groups marked as passive group by user $i$.

4.2. Intra-Group Secret Key Negotiation Based on Shamir’s Secret Sharing

Within each group, users utilize the Shamir’s secret sharing algorithm for intra-group secret key negotiation. By leveraging the features of Shamir’s secret sharing, the process can be further categorized into generation of secret and secret shares, homomorphic addition, secret reconstruction and secret key computation, as specified in Algorithm 3.

Algorithm 3: Intra-Group Secret Key Negotiation Based on Shamir’s Secret Sharing
Input: user set $I_g$ of Group $g$ Output: secret key $s{k}_g^i$ of user $i$ within group $g, i\in I_g$
1  For $i\in I_g$	//Secret and secret share generation
2    Present a random number $ran{d}_g^i$  3    $s_g^{ij}\leftarrow sharing\left(ran{d}_g^i,\left|I_g\right|,\left|I_g\right|\right), j\in I_g$  4    Directly transmit secret share $s_g^{ij}$ to user $j$ within the group  5  End
6  For $i\in I_g$	//homomorphic addition
7    $s_g^i\leftarrow add\left(s_g^{ji},j\in I_g\right)$  8    Directly distribute secret share $s_g^i$ to other users within the group  9  End
10  For $i\in I_g$	//Recovering secret and calculating key
11    $s_g\leftarrow reconstruction\left(s_g^j, j\in I_g\right)$ 12    $s{k}_g^i\leftarrow ran{d}_g^i-s_g/\left|Ig\right|$ 13  End 14  return $s{k}_g^i, i\in I_g$

Algorithm 3 is achieved using secure multi-party computation based on Shamir’s secret sharing and ensures the security of the secret key. Among Algorithm 3, the $sharing()$ function represents the secret key generation function based on Shamir’s secret sharing, which divides a secret $s$ into $n$ secret shares, and the secret $s$ can be reconstructed when the number of secret shares is no less than $t$. The $add()$ function represents the addition of secret shares and its output is equivalent to a secret share obtained from using $sharing()$ on the sum of two secrets $s_1$ and $s_2$ because of the additive homomorphism property of Shamir’s secret sharing. The $reconstruction()$ function represents the secret reconstruction function based on Shamir’s secret sharing, which can reconstruct the secret $s$ when given no less than $t$ secret shares.

Within group $g$, each user independently presents a random number $ran{d}_g^i$, and computes the sum of the random numbers $s_g$ within group $g$ through secure multi-party computation based on Shamir’s secret sharing scheme. After obtaining $s_g$, each user in the group divides it equally and subtracts their portion from their own random number to derive their local model’s encryption secret key $s{k}_g^i$ for group $g$.

This method uses the additive homomorphism property of Shamir’s secret sharing to perform summation. Each user in group $g$, as a participant, only knows their own random number $ran{d}_g^i$ and the sum $s_g$ of all random numbers within the group, without knowledge of other users’ random numbers. Since each user’s encryption secret key within the group is determined by both their own random number $ran{d}_g^i$ and the group sum $s_g$, the secret key $s{k}_g^i$ remains unknown to other users within the group.

Based on the numerical computation process of the secret key, we can derive the following property:

$${{\displaystyle \sum }}_{ i\in I_g}s{k}_g^i={{\displaystyle \sum }}_{ i\in I_g}ran{d}_g^i-s_g/\left|Ig\right|={{\displaystyle \sum }}_{ i\in I_g}ran{d}_g^i-s_g=0,$$

(1)

4.3. Encryption and Aggregated Decryption of Local Models within Groups

The secret key negotiated within the group is utilized as noise added to the users’ local models to derive encrypted models. According to the property of the secret key as demonstrated by Equation (1), decryption is achieved through the aggregation of encrypted models within the group, as detailed in Algorithms 4 and 5.

Algorithm 4: Encryption of Each User’s Local Model within the Group

Input: local model $m^i$ of user $i$, secret key $s{k}_g^i$ of user $i$ within group $g$ Output: cryptographic model $m_g^i$ for user $i$ within group $g$  1  $m_g^i\leftarrow m^i+s{k}_g^i$  2  Return $m_g^i$

Algorithm 4 implements the encryption of each user’s local model within the group. During the encryption process of local models within group $g$, each user adds the previously negotiated encryption secret key $s{k}_g^i$ as noise to their local model $m^i$, and obtains the encrypted model $m_g^i$ for user $i$ within group $g$. After the implementation of Algorithm 4, each user uploads the encrypted models to the alliance chain.

Algorithm 5: Aggregate Decryption of Local Models in Groups

Input: cryptographic model $m_g^i$ of each user within group $g$, $i\in I_g$ Output: aggregated decryption Model $m_g$ of Group $g$  1  $m_g\leftarrow agg\left(m_g^i, i\in I_g\right)$  2  Return $m_g$

Algorithm 5 implements the aggregated decryption of local models within the group. $agg()$ represents the aggregation function for the local models and employs an average aggregation method, i.e., $agg\left(m^i, i\in I\right)=\frac{{{\displaystyle \sum }}_{ i\in I}{m}^i}{\left|I\right|}$. During the aggregated decryption process of local models within group $g$, the encrypted models $m_g^i$ of all users within group $g$ are aggregated according to $agg()$ to obtain the aggregated encrypted model $m_g$. Based on the property derived in Equation (1), we can deduce the following:

$$m_g=\frac{{{\displaystyle \sum }}_{ i\in I_g}{m}_g^i}{\left|Ig\right|}=\frac{{{\displaystyle \sum }}_{i\in I_g}{m}^i}{\left|Ig\right|},$$

(2)

thus, through the aggregation of encrypted models within the group, the aggregation of unencrypted local models within the group can be obtained synchronously, achieving the decryption of the encrypted models.

4.4. Fine-Grained Detection of Potential Violators

The service provider utilizes the recording property of the blockchain and the multiple groupings of users in each round of collaborative training to conduct multiple group detections. This approach reduces the impact of randomness during the model training and grouping processes in indirect user detection, enabling fine-grained detection of potential violators, as detailed in Algorithm 6.

Algorithm 6: Fine-Grained Detection of Potential Violators
Input: all user set $I$, key user set $KI$, basic grouping set $G_b$, focus group set $G_k$, set of active groups for each user $G_a^i, i\in I$, set of passive groups for each user $G_p^i, i\in I$, aggregation model for each group $m_g,g\in G$, key user score bound $boun{d}_{score}$, potential violator confidence bound $boun{d}_{conf}$ Output: set of potential violators $PI$, new key user set $NKI$  1  Initialize the set of potential violators $PI$, the new key user set $NKI$ is the empty set  2  Set the average accuracy of the basic group aggregation model to $\tau$ and initialize it to 0  3  Set the contribution score $scor{e}_i$ for each user $i$ and initialize it to 0 $,i\in I$  4  Set Confidence Interval Bounds $con{f}_{up}^i,i\in KI$ and $con{f}_{lo}^i, i\in KI$
5  For $g\in G_b$	//Calculate mean grouping accuracy
6    $\tau \leftarrow \tau +acc\left(m_g\right)$  7  End  8  $\tau \leftarrow \tau /\left|G_b\right|$
9  For $i \in I$	//Calculate the contribution score for each user
10    For $g \in G_a^i$ 11      If $acc\left(m_g\right)>\tau$ 12        $scor{e}_i\leftarrow scor{e}_i+1$ 13    End 14    $scor{e}_i\leftarrow scor{e}_i/\left|G_a^i\right|$ 15  End
16  For $i \in KI$	//Calculate confidence intervals for key users
17    $scor{e}_{mean}\leftarrow \raisebox{1ex}{${{\displaystyle \sum }}_{g\in G_a^i}scor{e}_g$}\!\left/ \!\raisebox{-1ex}{$\left|G_a^i\right|$}\right.$ 18    $con{f}_{lo}^i\leftarrow scor{e}_{mean}-\frac{S}{\sqrt{\left|G_a^i\right|}}{t}_{0.025}\left(\left|G_a^i\right|-1\right)$ 19    $con{f}_{up}^i\leftarrow scor{e}_{mean}+\frac{S}{\sqrt{\left|G_a^i\right|}}{t}_{0.025}\left(\left|G_a^i\right|-1\right)$ 20  End
21  For $i \in I$	//Determine if a user is a key user
22    If $scor{e}_i<\left(\tau -boun{d}_{score}\right)$ 23      $NKI\leftarrow NKI\cup i$ 24  End
25  For $i\in NKI$	//Determine if a user is a potential violator
26    If $i\in KI$ and $\left(con{f}_{up}^i-con{f}_{lo}^i\right)<boun{d}_{conf}$ 27      $PI\leftarrow PI\cup i$ 28      $NKI\leftarrow NKI/i$ 29  End 30  Return $PI$,$NKI$

Algorithm 6 is based on user grouping information and the group aggregated model’s accuracy. The service provider calculates the mean accuracy rate $\tau$ of the groups based on basic groupings as the standard for subsequent scoring. Each user, based on their active group $G_a^i$, calculates a contribution score $scor{e}_i$. If the accuracy of a certain group exceeds $\tau$, that group scores 1 point; otherwise, it scores 0 points. The average of these scores is then taken as the contribution score $scor{e}_i$ for user $i$. For key users, the confidence interval’s upper bound $con{f}_{up}^i$ and lower bound $con{f}_{lo}^i$ are additionally calculated based on the scoring of their active groups $G_a^i$ with a 95% confidence level for a more detailed contribution assessment, where $S=\raisebox{1ex}{${{\displaystyle \sum }}_{g\in G_a^i}{\left(scor{e}_g-scor{e}_{mean}\right)}^2$}\!\left/ \!\raisebox{-1ex}{$\left|G_a^i\right|$}\right.$ and $scor{e}_g$ represents the score of group $g$.

After the contribution assessment, the service provider determines whether to add a user $i$ to the new key user set $NKI$ based on whether the user’s contribution score $scor{e}_i$ falls below a certain threshold $\left(\tau -boun{d}_{score}\right)$. Within the new key user set $NKI$, users $i$ who are also in the key user set $KI$ are further selected. Whether to move a user to the potential violator set $PI$ is judged based on whether the more finely grained contribution score confidence interval $(con{f}_{lo}^i,con{f}_{up}^i)$ narrows to a smaller range $boun{d}_{conf}$.

5. Security Analysis and Experiment

5.1. Security and Privacy Analysis

(1)

Privacy Preservation for Local Model

Users encrypt their local models using the encryption keys of their respective groups to obtain encrypted models, which are then uploaded to the blockchain. The service provider downloads the encrypted models from the blockchain for potential violator detection and model aggregation. Throughout this process, all other members of the blockchain have access to these encrypted models. Therefore, ensuring that users’ encrypted models do not leak sensitive information after being obtained by the service provider and other users becomes the focal point of our research.

We employ secure multi-party computation based on Shamir’s secret sharing [29] to negotiate keys under the assumption of honesty and curiosity. In a group comprising three users, let us assume user $I_i\left(i=1,2,3\right)$ holds a secret $s_i\left(i=1,2,3\right)$. The secret sharing scheme divides the secret $s_i$ into three shares $s_{ij}\left(j=1,2,3\right)$, where three shares are required to reconstruct the secret $s_i$. Throughout the negotiation process, user $I_i$ only knows the secret $s_i$ they hold, one share of the secret sent by each of the other two users $s_{ji}\left(j=1,2,3;j\ne i\right)$, and the negotiation result ${\displaystyle \sum }_{i=1,2,3}{s}_i$. They cannot deduce the secrets $s_j\left(j=1,2,3;j\ne i\right)$ held by the other two users based on the secure multi-party computation process. Therefore, under the assumption of honesty and curiosity, users cannot infer the secrets held by other users through the secure multi-party computation process.

In the secure multi-party computation process, the secret $s_i$ is a random number generated by user $I_i$, denoted as $ran{d}_i$. If other users cannot know the secret $s_i$, then the key $s{k}^i=ran{d}_i-{\displaystyle \sum }_{j=1,2,3}ran{d}_j=s_i-{\displaystyle \sum }_{j=1,2,3}{s}_j$ also remains unknown. Therefore, the process of encrypting local models is secure in terms of key confidentiality.

(2)

Potential Violator Detection

The encryption of local models by users not only ensures the protection of their own privacy, guards against malicious attackers attempting to infer sensitive information and compromise privacy interests, but also introduces a challenge. Specifically, it deprives the direct ability of the service provider to conduct quality assessments on the local models provided by users. This opens up more covert channels for violators to engage in malicious behavior, such as data poisoning and free-riding attacks.

To address this issue, we leverage the non-repudiation property of blockchain to facilitate the detection of user behavior. By recording the outputs of each user on the blockchain and synchronizing this information across the entire peer-to-peer (P2P) network, we enable the service provider to indirectly assess the quality of user’s local models. This is achieved through multiple rounds of group aggregation of user outputs recorded in the blockchain. Such an approach allows the service provider to evaluate the presence of data poisoning attacks and free-riding attacks, thereby implementing a system for the detection of potential violators.

5.2. Experiment

In this section, we will conduct experimental evaluations on the blockchain-based fine-grained potential violator detection algorithm to demonstrate its effectiveness and performance in a blockchain network. This entails ensuring that (i) the algorithm can execute within the constraints of limited computing resources, and (ii) it can effectively detect potential violators.

• Experimental setup: We employed a total of eight CPU servers, each equipped with a Hygon C86 7159 16-core processor, which is produced by Hygon Information Technology Co., Ltd. in Beijing, China, and two GPU servers; each GPU server was outfitted with eight NVIDIA TESLA T4 GPUs, which are produced by NVIDIA in City of Santa Clara, CA, USA, and 16 GB of memory. These servers were utilized to conduct training for data owners.
• Dataset: We conducted our tests using the MNIST and CIFAR-10 datasets. The MNIST dataset consists of binary images of handwritten digits and was curated by the National Institute of Standards and Technology in the United States. MNIST is widely employed in machine learning for training and testing purposes and serves as a benchmark in the field. On the other hand, CIFAR-10 is a color image dataset and represents objects more akin to everyday entities. In contrast to MNIST, CIFAR-10 presents real-world objects because it introduces significant noise and variations in object proportions and features. This realistic complexity poses substantial challenges for recognition tasks and makes CIFAR-10 a valuable resource for assessing model performance under more demanding conditions.

Q1: 

Can our approach accurately detect data poisoning and free-riding attacks? Furthermore, can it pinpoint the identity of malicious users?

To validate the feasibility of our potential violator detection algorithm, we employ a Convolutional Neural Network (CNN) model for image classification tasks on the MNIST and CIFAR-10 datasets, subsequently assessing its accuracy. Specifically, the CNN architecture for the MNIST dataset comprised two convolutional layers, two pooling layers, and two fully connected layers. On the CIFAR-10 dataset, the CNN model featured eight convolutional layers, four pooling layers, and three fully connected layers. The aggregation algorithm for collaborative training is FedAvg, with a learning rate of 0.01 for each round. The batch size is set as B = 64, and the maximum communication round (epochs) is set as E = 50. In this experiment, potential violators will not be excluded from the collaborative training process.

In each learning round, nine client machines perform local training, and the resulting models are aggregated by the service provider. We evaluated the collaborative training accuracy in the absence of malicious users, involving the accuracies of global and each group models in the MNIST and CIFAR-10 datasets. As shown in Figure 3, the results indicate that, under sufficient training, the accuracy of the group models closely approaches that of the global model as baseline, where no malicious user is present. After 50 iterations, the global model achieved an accuracy of 98.63% on the MNIST dataset and 80.93% on the CIFAR-10 dataset.

In our simulated experiments on data poisoning attacks, we randomly designate a device as the poison attacker (PAC). This chosen device uploads encrypted local models containing negative gradient information acquired during the training process, simulating a data poisoning attack. We assess the impact of poisoning attacks through group aggregation detection, recording the influence of PAC attacks on the accuracy of global model and group models.

As shown in Figure 4a,b, the simulated data poisoning actions by the poison attacker (PAC) exhibit fluctuations in the accuracy of the global model on both the MNIST and CIFAR-10 datasets. These fluctuations hinder convergence and result in a significantly lower global model accuracy compared to the baseline accuracy. The impact of PAC is further evident in the aggregation model accuracies of the two groups influenced by PAC. These group models not only have lower accuracies compared to the global model but also show no signs of convergence. This implies that PAC receives lower scores in our user detection and is identified as a potential violator, leading to its exclusion from the collaborative training process. These results substantiate the effectiveness of our user detection algorithm in resisting users’ data poisoning attack and accurately pinpointing attacker within the collaborative training framework.

In our simulated experiments on free-riding attacks, we randomly designate a device as the free-riding attacker (FAC). This device does not actively participate in regular local training and only uploads zero-gradient parameters to blockchain in each iteration to simulate a free-riding attack. Group aggregation detection is employed to assess free-riding behavior, and we record the impact of FAC’s on the accuracies of global model and group models.

As shown in Figure 4c,d, in the MNIST dataset, the accuracies of the two groups influenced by FAC are lower than those of the global model and other groups during the early iterations with significant accuracy fluctuations. Although the accuracies of the FAC-influenced groups gradually increase and approach the accuracy of the global model in later iterations, the stability of the model accuracy is comparatively poorer and the magnitude of accuracy fluctuations in these groups is greater than that observed in other groups. In the CIFAR-10 dataset, during the early iterations, the model accuracies of the FAC-influenced groups are not significantly different from those of the global model and other groups. However, with continued iterations, it becomes evident that the accuracies of these two groups are consistently lower than those of the global model and other groups. Free-riding attacks initiated by malicious users may converge with increasing iteration counts, indicating that detecting free-riding attacks is more challenging compared to data poisoning attacks. Nevertheless, through extended observations, abnormal behaviors and the underlying FAC can still be identified. This result demonstrates the effectiveness of our user detection algorithm in resisting user’s free-riding attacks and accurately pinpointing attackers.

To validate the scalability of our proposed method and examine its ability to identify potential violators in the presence of multiple malicious users, we conducted an assessment with nine participants and without excluding potential violators. We performed four basic groupings per round with epoch = 50 and analyzed the changes in user scores and confidence intervals, as depicted in Figure 5.

In Figure 5, we observe that as the number of malicious users increases, the difference in scores between general users and poisoning attackers, as well as free-riding attackers, decreases, and the overlapping area of the confidence intervals for scores becomes larger. However, even with the presence of three poisoning attackers and three free-riding attackers, i.e., when two-thirds of the participants are malicious users, the scores and confidence intervals for general users remain significantly higher than those for malicious users. This indicates that in the collaborative training process, malicious users can be identified as potential violators and excluded from the collaborative training process, demonstrating good scalability.

Q2: 

Is our proposed method resistant to malicious attacks?

To validate the impact of our user detection algorithm on the accuracy of collaborative training, we compared the accuracy variations among three categories of collaborative training scenarios: collaborative training with benign users (no malicious behavior), collaborative training with malicious attacks and no user detection, and collaborative training with malicious attacks and user detection.

From Figure 6, it is evident that on the MNIST dataset, the user detection algorithm enhances the stability of collaborative training processes that include malicious users engaging in poisoning attacks. This improvement allows the global model to reach a more stable state with a higher accuracy. For collaborative training involving free-riding users, the user detection algorithm also contributes to an increased model accuracy, aligning it closely with the accuracy of collaborative training with malicious users. On the CIFAR-10 dataset, the use of our user detection algorithm provides the accuracy and convergence speed of the global model in collaborative training scenarios with malicious users. This indicates that our proposed user detection algorithm is effective in identifying potential violators during the collaborative training process, enhancing both the convergence speed and final model accuracy of the entire collaborative training process.

Q3: 

Does our proposed approach effectively improve performance under ciphertext potential violator detection?

In our user detection algorithm, secure multi-party computation based on Shamir’s secret sharing and multiple group localization plays a crucial role. In this section, we assess the feasibility of secure multi-party computation based on Shamir’s secret sharing and multiple rounds of grouping localization by examining the overhead of encryption and decryption, as well as the communication overhead for users under differential numbers of groups. We employ the BCP algorithm for encrypting and decrypting model parameters [6] as a baseline.

To validate the effectiveness of secure multi-party computation based on Shamir’s secret sharing, we evaluated its overhead. Since the model decryption process aligns with the normal model aggregation process, we have omitted the decryption process and only calculated the overhead incurred by key negotiation and computation during encryption, along with the necessary communication.

The overhead is evaluated based on the number of encryption and decryption iterations. As shown in

Table 1. The overhead of encryption and decryption time(s).

Times	5	10	15	20
BCP_Encavg [6]	24.30	43.59	72.62	96.35
BCP_Dec [6]	45.95	89.03	132.11	175.19
Our_Encavg	1.265	2.535	3.796	5.060
Our_Dec	-	-	-	-

, the experimental results demonstrate that the BCP algorithm requires 96.35 s for encryption and 175.19 s for decryption in 20 iterations. In contrast, our proposed method incurs no additional overhead during the decryption process, and the overhead for 20 encryption iterations is only 5.06 s. This is significantly lower than the overhead associated with the BCP algorithm.

This is attributed to the fact that our method only has complex computations in the secret key negotiation phase based on Shamir’s secret sharing. The computation in the model encryption phase is straightforward and results in an overall lower computational load. Additionally, the user detection algorithm constrains the scale of key negotiation within each group. This contributes to a lower overhead of communication during the secret key negotiation process. As a result, our proposed method incurs a significantly lower overhead compared to the BCP algorithm.

To validate the effectiveness of multiple rounds of grouping localization, we evaluated its overhead of communication by calculating the overhead of communication incurred through uploading group models via the blockchain and necessary communication during the encryption process.

The overhead of communication was assessed based on the number of groups. Experimental results, as shown in

Table 2. The overhead of communication in uploading models (MB).

Times	5	10	15	20
BCP [6]	42.55	85.11	127.67	170.23
Our	10.48	20.97	31.45	41.94

, indicate that the encryption algorithm based on BCP requires the transmission of 170.23 MB of data during 20 times of group model uploads, whereas our proposed method only requires 41.94 MB, which is lower than the overhead of communication of the BCP. This is because our proposed grouping encryption method does not expand the messages during the encryption process, resulting in the same size for both plaintext and ciphertext. However, in the BCP algorithm, only one group model needs to be uploaded per round of training, whereas our proposed method requires uploading multiple groups. Therefore, in practice, if there are too many groups in one training round, it can lead to significant overhead of communication.

Furthermore, our proposed method requires only one blockchain compared to the BCP. All content uploaded to the blockchain is encrypted, and no key-related content is uploaded, thus avoiding the risk of a single node simultaneously being part of two blockchains, where it could potentially access both keys and ciphertexts and decrypt them to obtain the original local models if the entire blockchain content is readable by blockchain nodes.

6. Conclusions

Over the past decade, communication technologies have experienced significant breakthroughs, driving the expansion of extensive data applications. The advent of CFNs has rendered multi-party data collaborative training an irreversible trend. However, traditional approaches to multi-party data collaborative analysis necessitate a trusted entity for data fusion. While blockchain-based multi-peer endorsement technology offers a novel solution to this challenge, the issue of undetectable data poisoning and free-riding attacks that emerge post the encryption of data on the blockchain still requires resolution. Consequently, we propose a blockchain-based fairness guarantee approach for privacy-preserving collaborative training in a CFN. Our proposed approach employs cryptography-based secure aggregation to prevent malicious attackers from obtaining private information in the training data through gradient inversion attacks. It also identifies potential violators through group aggregation evaluation of accuracy, effectively mitigating encrypted data poisoning and free-riding attacks. Additionally, in our use of Shamir’s secret sharing for secret key negotiation within the group, we limit the scale of the negotiating group and avoid high computational multiplication calculations, thereby enhancing the computational efficiency of encryption and decryption. Finally, we validate our proposed approach on two publicly available datasets, and our experimental results affirm its effectiveness and efficiency.

In our forthcoming research endeavors, our objective is to expand the scope of encrypted data fairness guarantee approaches in collaborative training beyond poisoning attacks and free-riding attacks to comprehensively explore various deep neural network threats, including backdoor attacks. Concurrently, we are committed to guaranteeing the precision and sensitivity of malicious user behavior detection. In addition to conventional metrics such as model accuracy, we will delve into multi-dimensional assessment techniques. This comprehensive approach will enable us to more effectively identify potential violators within complex and diverse environments, thus guaranteeing the security and resilience of collaborative training systems.