LFL-COBC: Lightweight Federated Learning on Blockchain-Based Device Contribution Allocation ^†

Abstract

In the distributed cyber-physical systems (CPSs) within the industrial domain, the volume of data produced by interconnected devices is escalating at an unprecedented pace, presenting novel opportunities to enhance service quality through data sharing. Nevertheless, data privacy protection emerges as a significant challenge for data providers in wireless networks. This paper puts forward a solution integrating blockchain and lightweight federated learning, designated as LFL-COBC, which aims to tackle the issues related to data privacy and device performance optimization. We initially analyze multiple dimensions influencing the performance of computing devices, such as mining capacity, data quality, computational efficiency and local device deviation, which are crucial for augmenting user engagement. Based on these dimensions, we deduce a set of cooperation strategies for selecting the optimal committee members and rewarding the contributions of node devices equitably, thereby stimulating cooperation between users and servers. To intelligently and automatically detect device anomalies and alleviate the operational burden, a convolutional neural network (CNN) model is employed. Additionally, to address the escalating cost of customer participation and the potential data explosion issue, a near-optimal model pruning algorithm is designed. This algorithm can make the model obtained from the training of node equipment lightweight, thereby reducing the load of federated learning and the blockchain, as well as enhancing the overall efficiency of the system. The efficacy of our approach is demonstrated through numerical experiments on the HDFS and BGL public data sets. Experimental results indicate that the LFL-COBC scheme can effectively safeguard data privacy and optimize device performance concurrently, providing an effective solution for device anomaly detection in CPSs.

1. Introduction

With the introduction of a large number of intelligent and unmanned equipment, and with the program structure of equipment being too complex—making the maintenance and overhaul of industrial equipment more and more complex—artificial intelligence real-time anomaly monitoring for distributed industrial equipment is the future development trend [1]. However, nowadays, many industries have put forward higher requirements for the ability of industrial equipment to provide services. With the value brought by data, there are serious concerns about the protection of device privacy [2,3].

At present, multi-directional privacy computing, full homomorphic encryption and other technologies are often used to protect device data privacy [4], but they will may cause huge computing and resource consumption whilst also not supporting real-time overhaul and maintenance of current industrial distributed equipment. In order to maintain an efficient and safe working state of computing devices and to ensure the operation of devices’ data, federated learning coordinates multiple-IoT devices to protect privacy and automate networks. Many neural networks are over-parameterized [5,6], enabling the compression of each layer or the entire network [7]. Some compression methods achieve more efficient calculations with pruning parameters [8], training quantization [9], or blockchain data storage technology [10,11,12]. To mitigate the computation and resource consumption resulting from these techniques, we devise a lightweight privacy protection mechanism, enabling it to support real-time overhaul and maintenance of industrial distributed devices. Therefore, prior to the server aggregating the models, the models of each node device need to undergo lightweight processing. Currently, there exists a correlation among different computing devices within the industrial Internet ecosystem. Taking into account the correlation among different computing devices in the industrial Internet ecosystem, we incorporate blockchain technology to enhance the security and stability of the system. Once the data of computing devices are leaked or subject to deliberate attacks, the intervention of the blockchain can guarantee the security and stability of the entire industrial architecture [13].

In order to balance the overhead and security of the system, Lu et al. [10] designed a reward mechanism based on the amount of node data but ignored the impact of data quality on the effectiveness of client-side model training. Hyesung Kim et al. [14] analyzed the end-to-end delay model of the system and combined the factors that have a significant impact on the efficiency of each client to carry out the study, paving the way for us to design the blockchain reward mechanism. According to the characteristics and requirements of the industrial Internet, the system should follow the principles of ensuring the safety of devices in complex environments and promoting the fair and equal participation of multiple organizations [15,16]. We design an incentive mechanism based on device mining, data collection, computing power, and local and global dissimilarity to improve the work enthusiasm of client devices and promote the sharing of distributed data among multiple untrusted parties. In order to improve the efficiency of federated learning, we design a reward mechanism based on mining ability, node data volume, and work efficiency. At the same time, we consider the impact of data quality on the effectiveness of client-side model training, which ensures the accuracy and reliability of model training. The convergence of the blockchain and federated learning enables data sharing and privacy protection in industrial infrastructure, facilitating progress in the field while ensuring data security for clients.

Based on the blockchain-federated learning architecture proposed by [17], we extend and make the following contributions:

• This paper presents a privacy protection scheme for the industrial environment that integrates the blockchain and federated learning (LFL-COBC), mitigating the aggregation of local models by the central server through pruning the models trained by node devices.
• In order to enhance data privacy and alleviate the burden of the blockchain in the industrial Internet environment, a lightweight federated learning model for device anomaly detection is proposed. We make global and local adjustments to adapt to the local data set. A global model trained using such imbalanced data is challenging to generalize well on the data of each customer.
• We further update the blockchain incentive mechanism to achieve distributed multi-party data sharing, reduce the disparity between local and global, and we verify data calculations, thereby enhancing the fairness and effectiveness of each node’s equipment operation.

2. Paper Organization

The rest of this article is organized as follows. Firstly, the proposed global scheme overview for computer devices in industry and detailed structural design flow is discussed in Section 3. We then present the implementation of the blockchain and federated learning algorithms in Section 4. Section 5 elaborates on the proposed approach for detecting anomalies in industrial equipment and conducting lightweight operations. Next, we present the experimental results, which are described in Section 6. Finally, this article is concluded in Section 7.

3. Lightweight Federated Learning on Blockchain-Based Device Contribution Allocation

3.1. System Model

Traditional machine learning aggregates data from different devices to a central server, which may lead to data leakage and privacy infringement. Users are unwilling to share data, which leads to the problem of an isolated data island [18]. FL has been studied by many researchers in order to alleviate these problems. However, the application of FL technology to devices faces many challenges. For example, devices upload local gradients or models to a central station, which will impose resource overhead and security risks. It is essential to integrate the blockchain and federated Learning to protect data privacy.

Blockchain technology together with decentralized federal learning architecture can solve many problems. The blockchain is a kind of decentralized, trusted database. It can use digital encryption and timestamp methods to implement point-to-point transactions without mutual trust. Our proposed global architecture is shown in Figure 1. The proposed system consists of two modules: a blockchain module and a federated learning module. The blockchain establishes secure connections among all the end-user computing devices through its encrypted records, which are maintained by the entities equipped with computing and storage resources.There are two types of transactions in our approved blockchain: data sharing transactions and data computation transactions [15]. To ensure fairness, we evaluate the client based on their mining and working abilities. Additionally, to minimize resource waste and alleviate the load on the blockchain, we employ compression and lightweight techniques to share the model results from each node device.

In this paper, we adopt a classic blockchain system architecture and blockchain system, which are divided into four levels: the cloud layer, the smart consensus layer, the edge layer, the local layer. The model of federated learning is stored in the edge layer. The cloud layer and the edge layer support the operation of the system, training, and update the model together, and the smart consensus layer can support the data isolation and privacy protection of each distributed device in the whole system.

The cloud encompasses a cloud server possessing potent computing, communication, and storage capabilities, and the task publisher is an organization or enterprise with FL tasks. The task publisher transmits the task information, encompassing the task publisher’s identity, task description, and model accuracy requirements to the cloud server. The cloud server supervises the task training process and conveys the global model to the task publisher upon meeting the accuracy requirements.

The smart consensus layer contains all the information of the blockchain, and the blockchain layer is based on the POW-POD-Deviation-Quatity (PODQ) consensus for recording the global model and edge model. In this layer, each edge server corresponds to a consensus node or leader node, and each cloud server corresponds to a unique monitoring node. In this scenario, edge servers and consensus nodes are referred to as edge nodes. Each consensus node stores their edge model and verifies edge models from other consensus nodes. The leader node manages data communication and transfers between nodes, and the monitoring node generates and broadcasts blockchain information containing task information and aggregates local models to generate the global model. The blockchain supports off-chain storage of private data and on-chain storage of hashes, suitable for storing user data and ensuring the trustworthiness and availability of the model, improving system performance.

The edge layer mainly realizes the rapid consensus and calculation of transactions and data by highly scattered nodes in the whole network. The main element of this layer is the edge node, which has fixed computing and communication capabilities. The edge nodes are deployed close to the terminals to reduce the communication pressure on the cloud. There can be overlap between edge nodes. At each epoch, edge nodes will aggregate from locally uploaded model aggregations to generate edge models.

The local layer consists of identically distributed local devices with limited computing and communication capabilities. Assemble the device nodes. At each epoch, each local device trains a local model using its own data and uploads it to the relevant edge node.

3.2. The Constituents of the Overall Architecture

The proposed LFL-COBC architecture is shown in Figure 2. The system consists of two modules: a blockchain module and a federated learning module. Blockchain technology combined with federated learning architecture can highly solve the problems currently faced by distributed industrial equipment. Participants can train their models locally and upload the model parameters to a central server for aggregation. However, due to concerns about data privacy, blockchain technology can provide a secure way to share and transmit data at this point, allowing participants to train models together without exposing raw data [19,20].

The overall system architecture of LFL-COBC consists of the following three important components:

• Local devices: The local industrial equipment trains the model based on the collected data and transmits the trained model to the edge cloud.
• Edge cloud: The edge cloud is capable of uploading the model from the local device to the blockchain, selecting the committee through an incentive consensus mechanism, specifying the highest score for summarizing the local device’s model, and updating the global model.
• Blockchain: The blockchain is a trusted database that establishes a secure connection between terminal computing devices through its encrypted records. It can use digital encryption and timestamping methods to realize peer-to-peer trustless transactions [19].

In summary, the implementation of the proposed blockchain-based lightweight federated learning method is given in Algorithm 1. Let $N_{iter}$ denote the number of iterations in the federated learning process and $N_{epoch}$ be the number of iterations in the local training process. $M_{init}$ represents the initialization model, $N_{client}$ represents the number of client users, and $M_{N_{iter}}^i$ represents the model under client i for a federated learning iteration $N_{iter}$.

Algorithm 1 Proposed blockchain-based lightweight federated learning approach

Require:  model structure, $N_{iter}$, $N_{epoch}$, $M_{init}$, $N_{client}$, reward Ensure:  Device fault detection model ${\left\{M_{N_{iter}}^i\right\}}_{i=1}^{N_{client}}$ 1: Build the blockchain, initialize the fault diagnostic model as $M_0$, and set the reward score for each client to $rewar{d}_i$ 2: for $j=1\to N_{iter}$do 3:     Selection of $N_{cmt}$ committee members based on each client reward score 4:     for each non-committee client i do 5:         Download latest global model from the blockchain $M_j^i=M_{j-1}$ 6:         for $k=1\to N_{epoch}$ do 7:            The client model trains and updates the model to $M_j^i$ 8:            The model is compressed to $M_j^i$ 9:         end for 10:        Send the trained model to the committee members 11:     end for 12:     Calculate the score of each non-committee member with Equation (5) 13:     The miner computes the global model with Equation (9) 14:     The blockchain is expanded with updated global model $M_j$ 15: end for

The architecture design flow of the proposed scheme in the industrial Internet environment is shown in Figure 3. The overall system architecture workflow based on the proposed lightweight blockchain-based federated learning is as follows:

• Node registration. All computing devices need to be registered on the blockchain before joining the network. This can be achieved by a decentralized registration mechanism that uses smart contracts to verify the identity of devices and issue unique identifiers.
• The excitation of the node. With the help of the PODQ incentive policy, each node device is scored regularly, and then the members of the committee are elected according to the scores of each node device. According to the scores of nodes, rewards are automatically allocated through smart contracts to encourage nodes to actively participate in network maintenance and data processing.
• Block generation and verification. Each node device mines on the blockchain to find available blocks, and the node device mines blocks through the PODQ mechanism. After block generation, it needs to be verified by other nodes in the network. Byzantine Fault Tolerance (BFT) can be implemented to ensure that more than 2/3 of the nodes reach a consensus on a new block. Once a block is verified enough, it will be added to the blockchain, ensuring the immutability and security of the data.
• Model upload. After scoring each node device based on its mining ability, the quality of its local owned data, and the efficiency of its work, the node device uses its local data to train the model or participates in the federated learning process to cooperate with other nodes to train a shared model. Each node can choose to upload its own model or aggregate the models of other nodes based on its score and computing power. The uploaded model needs to be verified by the network to ensure the validity and security of the model. The model verification process can be implemented through smart contracts. The model that passes the verification will be added to the blockchain for use by other nodes or for further training.

3.3. Attack Model

The types of attacks that can be encountered in blockchain-federated learning architectures include tampering and impersonation. In this attack model, the attacker’s goal is to undermine the results and credibility of federated learning by tampering or falsifying data, and they may try to inject wrong data into the federated learning model to cause the model to produce wrong outputs [2,18]. The attacker may be a malicious data provider, participant, or other malicious entity in the network.

In the attack model of tampering and impersonation, the attacker can take several paths to carry out the attack.

• Data tampering: An attacker could manipulate certain samples or features within the data set or might inject counterfeit data samples into the federated learning model to disrupt the training and prediction process of the model.
• Malicious model update: An attacker may modify model parameters or updates to produce erroneous model output results. They can do this by injecting malicious code into the model updates.
• Collusion attack: The attacker may collude with other participants to increase the success rate of the attack by jointly tampering or falsifying the data. This type of attack may make affect the results of federated learning more.

4. Blockchain-Enabled Federated Learning Approach for Industrial Equipment Inspection

The blockchain and federated learning are two popular technology areas that have a wide range of applications in different application scenarios. The blockchain is a distributed ledger technology that enables secure sharing and transmission of data. Federated learning is a machine learning technology that enables model training and updating through model aggregation without exposing original data.

4.1. Data Structure Design for Blockchain

The blockchain is a storage method for electronic data. The data are obtained in the form of blocks, which are linked together to provide immutability to their internal data. Once a block is linked to the chain, its internal data can no longer be changed. Once a block is added to the chain, the data inside are publicly visible to anyone. This technology can be used to record almost any information we can think of (e.g., property rights, identities, balances, medical records, etc.) without the risk of tampering with the records. Current practices require the existence of a decentralized industrial Internet system that allows for direct peer-to-peer monetary transactions between trading parties. In the transaction process between nodes and the blockchain, processing of data is involved, so it is especially important to design a reasonable blockchain data structure.

Based on the incentive mechanism and the demand of federated learning for data, we can reason out the blockchain block data structure, as shown in Figure 4.

The block header uses the hash of the previous block to maintain the chain structure, and the Merkle tree root is used to summarize the transaction information in the block. At the same time, nodes can use the Merkle tree to quickly verify transactions. At the same time, proof, base model, accuracy, and quantity store the node mining computing power, the final trained model, the model accuracy, and the data quality owned by the node, which are all prepared for the reward and punishment of each node device and the aggregation model later. Since each industrial equipment node is dynamically changing, it will transmit the updated model to the blockchain for a data update. Afterwards, it is necessary to create a data structure about data transactions under the block body, as shown in Figure 4.

Block bodies mainly hold information regarding hundreds or thousands of transactions. Once a transaction is sent to the blockchain network, it is packaged into the block. In the transaction information of this block, the type determines whether the update model information of the obtained node is acquired through a global download or a self-update.

4.2. Incentive: Certificate of Training Quality and Data Management (PODQ)

Transforming data sharing issues into model sharing brings significant benefits in terms of data security. By sharing only the model without disclosing the original data, the privacy of the data owner can be effectively protected. However, existing incentive mechanisms like Proof of Work (POW) either impose high computational and communication resource costs or lack a comprehensive understanding of node allocation contributions [4,16]. To tackle this challenge, we propose the Training Quality Data Proof (PODQ) protocol for federated learning authorization. PODQ integrates data model training and the quantity of data processed into the consensus process, thereby optimizing the utilization of computing resources across nodes.

POW: The incentive mechanism of Proof of Work (POW) in the blockchain is primarily designed for public blockchains. In POW, nodes engage in a competition to secure accounting rights by solving complex mathematical puzzles, known as hashing algorithms. The ability to successfully find the correct numerical solution and generate blocks is a tangible demonstration of a node’s computational power. Nodes with a higher computational power have a greater likelihood of solving these numerical values and gaining accounting rights [4].

$$P_i={\displaystyle \frac{{\phi }_i}{{\sum }_{j=1}^N{\phi }_j}}$$

(1)

where i represents each participating node, N is the total number of nodes, and ${\phi }_i$ represents the arithmetic power of node i.

Training quality and data volume-based incentive: The committee node selection process involves a subset of all participants, and it aims to strike a balance between costs, security, and fairness. To achieve a consensus on data sharing, we have introduced additional factors, such as proof of training workload and data capability. The selection of committee leaders is based on the quality, volume of data, and mining ability of the training model. Since each committee node trains a local data model, it becomes essential to validate and measure the quality of these models during the consensus process. The performance of the trained local model is quantified using prediction accuracy. By incorporating these metrics, we can effectively assess and compare the performance of local models.

$$MAE\left(m_i\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n|y_i-f\left(x_i\right)|$$

(2)

where $f\left(x_i\right)$ is the prediction value of model $m_i$ and $y_i$ is the real value of the records. The lower the MAE of model $m_i$, the higher the accuracy of $m_i$.

After model training, we obtain a trained global data model M and a local model $m_i$ for each committee node. The incentive process is performed by the committee. As a proof of training work, $D_j$ verifies all transactions it receives by calculating the MAE defined in Equation (3). The MAE for $D_j$, $MA{E}^u\left(D_j\right)$ is calculated as follows:

$$MA{E}^u\left(D_j\right)=\gamma +MAE\left(m_j\right)$$

(3)

where $MAE\left(m_j\right)$ is the MAE of the locally trained model $m_j$, and $\gamma$ is the weight parameter denoting the contribution of $D_j$ to the global model, which is determined by the training data size of $P_j$ and other participants, $\gamma =\left|d_j\right|/{\sum }_i\left|d_i\right|$.

Local device deviation: In the context of the distributed industrial Internet, the issue of collaborative learning with streaming data entails multiple nodes collecting data incrementally. Since data are scarce at the onset of data collection, collaboration through data sharing can furnish nodes with machine learning models trained on shared data. Moreover, nodes with higher contributions (e.g., richer nodes with superior local data) can obtain better incentives. However, this can widen the “gap” between richer and less-resourced nodes, possibly impeding the participation of less-resourced nodes, which is undesirable. Subsequently, the score calculation for each node device also needs to incorporate the gap between local devices and global devices $D{F}_i$ to provide equal and comprehensive incentives for maintenance equipment.

$$D{F}_i=\left||f_i\left(w\right)-g_t|\right|$$

(4)

$f_i\left(w\right)$ represents the deviation gap between the model trained using the i-th local device and the $g_t$ global model.

PODQ: By combining the POW mechanism with the training data and the quality, the formula is as follows:

$$S_j=\gamma +MAE\left(m_j\right)+P_j+DF\left(m_j\right)$$

(5)

where $S_j$ denotes the score obtained by the j-th node through the reward mechanism, and $P_j$ represents the mining computational arithmetic provided by node j on the blockchain. $DF\left(m_j\right)$ is the dissimilarity between the local and global models.

When the incentive process starts, the committee node with the highest score at that time will be elected as the committee leader through PODQ-based voting. The leader is responsible for driving the consensus process among the participating nodes, and the committee nodes audit the blocks by verifying the model transaction traces in the same way that a verification node verifies the amount of Bitcoin transactions. The leader broadcasts the task of generating a new block to all members of the committee for approval, and if the block containing all transactions is approved by each committee node, the leader sends the block data signed with their signature to all nodes. Then, the records are stored in the blockchain, which are tamper-proof.

4.3. Federated Learning

In the multi-party environment of the industrial Internet, the distribution of data sets is fragmented. Moreover, based on the principle that the data do not move but the model moves, data are available but not visible [3], the local devices are trained based on the data sets, and the model parameters are uploaded to the central server for aggregation. After aggregation, new model parameters are sent to each device [10]. In this process, the blockchain can remove the trusted central authority. Therefore, it can solve the trust problem of the central server in federated learning and prevent single points of failure in federated learning [20,21]. To change the tradition and pursue a superior global model, we strike a balance between the local model and the global model—making global and local adjustments to adapt to the local data set.

In traditional federated learning, we need to solve the following problem across N clients: find a global model w that minimizes the loss function $f\left(w\right)$ for each client.

$$min\{f\left(w\right):={\displaystyle \frac{1}{N}}\sum _{i=1}^N{f}_i\left(w\right)\}$$

(6)

In LFL-COBC, add an $l_2$ regular term to the loss function:

$$f_i\left(w\right)=f\left({\theta }_i\right)+{\displaystyle \frac{\lambda }{2}}\left||{\theta }_i-w|\right|$$

(7)

where ${\theta }_i$ represents the personalization model of client i, $\lambda$ lambda is used as a hyperparameter to control the degree of personalization, and the larger $\lambda$ is, the lower the personalization program will be.

In order to improve the communication efficiency, when the gradient is uploaded to the central server, the method employs the PODQ mechanism to judge the rewards provided by each node. The detailed process of FL is shown below.

Step 1: Global model initialization. All industrial devices are registered on the blockchain to participate in federated learning. When each industrial device completes the local modeling task, the edge cloud aggregates the local models and sends the initialized global model $g_0$ to the blockchain.

$$g_0\to D_r$$

(8)

where $g_0$ denotes the initial global model, and $D_r$ denotes the r-th industrial equipment.

Step 2: Local model training. The industrial equipment uses local data to train a local model $g_t\to g_k$ by using $g_t$ and receive an updated local model $L_{k+1}$ with a trace optimized via the value of the loss function.

Step 3: Aggregate the model. According to the PODQ reward mechanism, the node with the highest reputation is selected to aggregate the blockchain model, and the elected node aggregates the local model through global training to obtain the globally updated model $g_{t+1}$. Then, the server broadcasts the new model to all devices.

$$g_{t+1}=g_t+\lambda \sum _{i=0}^m(L_i^{t+1}-g_i^t)$$

(9)

$g_{t+1}$ denotes the global model parameters updated in round t + 1, and $L_i^{t+1}$ denotes the model of the i-th client after the local update in round t + 1. $g_i^t$ denotes the model of the i-th client after the t-th round of global update. $\lambda$ denotes the reward obtained by each industrial equipment based on the PODQ mechanism.

5. Real-Time State Anomaly Detection of Industrial Equipment

The increase in scale and complexity of modern systems makes manual detection infeasible, and in the current industrial Internet environment, intelligent automatic detection of equipment anomalies is an urgent need in order to reduce the manual workload [6]. The log sequence is constructed, and the TF-IDF method is used to construct the feature vector, and then the CNN model is used to perform device anomaly detection [22] on the extracted features. At the same time, in order to reduce the communication overhead and resource allocation burden of the blockchain, the model rows need to be compressed to form a smaller network model before the model is uploaded by the computing device.

5.1. Industrial Equipment Anomaly Detection

With the rapid growth in the scale and complexity of modern systems, traditional manual detection approaches have been unable to fulfill the requirements of the current industrial Internet environment. In this context, intelligent and automatic detection of equipment anomalies has emerged as an urgent necessity, which can effectively alleviate the manual workload and enhance the detection efficiency and accuracy.

To achieve this objective, a diverse range of advanced technologies and approaches can be employed. Firstly, the working log of the computing device offers a detailed account of the events generated by the system. Nevertheless, traditional anomaly detection methods based on keywords or regular expressions do not yield satisfactory performance in sequential pattern-based anomaly detection. As modern systems expand in size and increase in complexity, manual detection becomes unfeasible. By constructing log sequences and utilizing the TF-IDF method to construct feature vectors, crucial information can be effectively extracted, laying a foundation for subsequent anomaly detection. The TF-IDF method can measure the relative significance of terms in a document and reduce the influence of common yet unimportant terms through an inverse document frequency, thereby enabling a more accurate identification of key terms in a document. This approach has many applications in document classification, keyword extraction, and recommender systems.

Next, a convolutional neural network (CNN) model was utilized to detect anomalies in the device based on the extracted features, which could further enhance the detection accuracy. The CNN model possesses distinct advantages in extracting abnormal waveform features, can handle nonlinear relationships, and has a flexible network structure design, which is suitable for various business scenarios and improves the universality and accuracy of the model.

5.2. Lightweight Model Pruning

Over-parameterization primarily implies that during the training phase, a significant number of differential solutions [23] in mathematics are necessary to capture minute variations in the data. Once the iterative training is accomplished, the network model does not require such a large number of parameters during inference. The pruning algorithm is founded on the theoretical basis of over-parameterization [24]. The core concept of the pruning algorithm is to decrease the quantity of parameters and computations in the network model while striving to ensure that the performance of the model remains unaffected. In the AI framework, the main effect of pruning is essentially to make the end-side model smaller, enabling small IoT devices such as tablets, phones, watches, headphones, etc., to effortlessly utilize the AI model [2,25]. From a macro perspective, pruning techniques are mainly categorized into two classic pruning algorithms, namely Drop Out and Drop Connect. In this paper, the drop-connect approach is employed for pruning, as shown in Figure 5.

• Drop-Out: On the left of Figure 5, the output of the neuron is randomly set to zero.
• Drop-Connect: In the process of training the neural network model, the right figure of Figure 5, the solid line is the input neuron under the retention, and the dotted line with red x is the redundant input neuron. Instead of randomly changing the output of the hidden layer node to 0, it changes the weight of each input connected to it in the node to 0, and the weight of the input neuron under the retention to 1.

Due to the limited carrying capacity of the blockchain on resource capacity, it is necessary to utilize the redundancy of neural network parameters and network structure to streamline the model. This enables us to obtain a model with fewer parameters and a more streamlined structure without affecting task completion. In a deep learning network model that extends from the convolutional layer to the fully connected layer has a large number of redundant parameters, with a large number of neurons’ activation value tending to 0. Moreover, the removal of these neurons can show the same expression ability of the model, with pruning algorithms being based on the theoretical basis of over-parameterization. In this paper, the connection between some neurons is randomly set to zero, which makes the weight connection matrix become sparse [11]. There are three main parts: training, pruning, and fine-tuning. Figure 6 and Algorithm 2 show model pruning, with the steps being as follows:

• Pre-training. The network model is first trained to construct a baseline model for the pruning algorithm to obtain the original model trained on the specific underlying task.
• Pruning. The magnitude of the weight values is ranked, and connections below a preset pruning threshold or ratio are removed to obtain the pruned network.
• Fine-tuning. Fine-tune the pruned network to recover lost performance, and then continue with the second step, alternating in this order until the termination condition is satisfied, e.g., the accuracy drops within a certain range.

Algorithm 2 Model pruning algorithm

Input: Trained model $model$, pruning threshold $threshold$ Output: Pruned model $pruned\_model$ for each layer $layer$ in $model.layers$ do     if $layer$ is of type $Conv2D$ then         $weights\leftarrow layer.get\_weights\left(\right)\left[0\right]$         $weight\_abs\leftarrow \mathrm{abs}\left(weights\right)$         $threshold\_weights\leftarrow weight\_abs>threshold$         for $i\leftarrow 0$ to $weights.size-1$ do            if $threshold\_weights\left[i\right]==\mathrm{False}$ then                $weights\left[i\right]\leftarrow 0$            end if         end for     end if end for $layer.set\_weights\left(\right[weights,layer.get\_weights\left(\right)[1:]\left]\right)$ Return: $model$

6. Experiment and Evaluation

In this section, we investigate model aggregation for device clustering and the impact of large models’ lightweight treatment on device log anomaly detection. We then compare our scheme with other schemes on homebrew FL learning models and real data sets.

6.1. Data Set

• HDFS [26]: HDFS (Hadoop Distributed File System) is a distributed file system running on general-purpose hardware to support the Hadoop ecosystem, a big data processing framework. There are three groups of HDFS logs: HDFS v1, HDFS v2, and HDFS v3. HDFS v1 was generated using a benchmark workload in a 203-node HDFS and manually marked via manual rules to identify anomalies. In addition, HDFS v1 provides specific anomaly-type information while also allowing us to study duplicate problem identification. HDFS v2 is collected by aggregating logs from an HDFS cluster in our laboratory environment, which includes a name node and 32 data nodes HDFS v3 is an open data set from trace-oriented monitoring, which was collected by detecting HDFS systems using MTracer 2.1 in a real IaaS environment.
• BGL [26]: BGL is an open log data set gathered by the BlueGene/L supercomputer system, which possesses 131,072 processors and 32,768 GB of memory. It is located at Lawrence Livermore National Laboratory (LLNL), Livermore, California [27]. The log encompasses both alert and non-alert messages, identified through alert category labels. In the first column of the log, a `-’ indicates non-alert messages, whereas others signify alert messages. The label information is conducive to studies on alert detection and prediction.

6.2. Security Analysis

In this experiment, we conduct security analysis from three aspects: data privacy, network attacks, and malicious participants.

• Data tampering: Malicious participants may attempt to obtain sensitive data of other participants and violate the privacy rights of users. In this paper, we adopt federated learning techniques, which can perform calculations without exposing data, having the ability to protect data privacy and ensure the accuracy of experimental results.
• Malicious model update: Attackers may attempt to perform attacks on communication channels to steal data or tamper with fake data transmissions. In this experiment, we use blockchain technology. Moreover, the data uploaded to the blockchain can only be stored in the blockchain when more than 2/3 of the clients pass the verification test, which ensures the integrity and security of the data in the transmission process.
• Collusion attack: There may be malicious participants in blockchain-federated learning experiments who try to corrupt the results of the experiment or obtain sensitive information of other participants. However, we use an incentive mechanism based on PODQ for the election of the committee to ensure that only legitimate participants with the highest rewards can participate in the experiment and ensure the reliability of the experiment.

6.3. The Performance of the Proposed Scheme

In this subsection, we investigate the impact of the proposed scheme on the model under different operations of model lightening and aggregation.

6.3.1. Comparison and Evaluation of Incentive Mechanisms

We use three non-interfering virtual machine devices to conduct experiments, and we explore the impact of the three incentives of PODQ, POW and POD on the training of models at each node, as shown in Figure 7 and Figure 8. With the increase in training batches, under the incentive mechanism we designed (PODQ), the training accuracy of the three devices is almost the best or close to that of the other incentives, and the loss rate is not only the lowest but is also maintained in the downward trend continuously. In contrast, the results of the training of multi-models of PODQ under the overall system architecture are superior to those of POW and POD.

6.3.2. Model Pruning Evaluation

In this subsection, we compare the model device working time and efficiency before and after model pruning.

• Comparison of aggregation times before and after model pruning. We compared the changes in the aggregation time of the models before and after pruning. As can be seen in Figure 9, we found that the aggregation time of the models after pruning is much shorter. At the same time, the size of the models after pruning was relatively reduced by 87%, which improves the efficiency and resource utilization of the overall system architecture to a certain extent.
• Comparison of node device work efficiency before and after model pruning. In order to investigate the effect of model pruning on the model training results of node devices, we compared the model training results of the virtual machines before and after model pruning, as shown in Figure 10. Model lightweighting had almost no effect on the level of model accuracy loss; however, the loss rate of the node devices was lower after model pruning. These data results suggest that the original model is over-parameterized and lightweight model operations are essential.

7. Conclusions and Discussion

In this paper, we discuss the emerging issues of LFL-COBC on incentives and resource maximization. Furthermore, we investigate the impact of updating the incentives of multidimensional clients and model lightweighting on the overall system architecture. We also derive the collaborative interactions among individual nodes by deducing their equilibrium strategies based on the rewards received by node devices. We further propose a near-optimal pruning algorithm to minimize the loss of model accuracy while minimizing model size, thus further improving the efficiency of individual nodes and resource utilization. In a future study, we will explore how to perform model aggregation in the blockchain for de-decentralization to avoid privacy and security issues that arise when model aggregation is performed on a designated client. Additionally, we will design a practical blockchain-federated learning system based on our analysis results.