EDSCVD: Enhanced Dual-Channel Smart Contract Vulnerability Detection Method

Abstract

Ensuring the absence of vulnerabilities or flaws in smart contracts before their deployment is crucial for the smooth progress of subsequent work. Existing detection methods heavily rely on expert rules, resulting in low robustness and accuracy. Therefore, we propose EDSCVD, an enhanced deep learning vulnerability detection model based on dual-channel networks. Firstly, the contract fragments are preprocessed by BERT into the required word embeddings. Next, we utilized adversarial training FGM to the word embeddings to generate perturbations, thereby producing symmetric adversarial samples and enhancing the robustness of the model. Then, the dual-channel model combining BiLSTM and CNN is utilized for feature training to obtain more comprehensive and symmetric information on temporal and local contract features.Finally, the combined output features are passed through a classifier to classify and detect contract vulnerabilities. Experimental results show that our EDSCVD exhibits excellent detection performance in the detection of classical reentrancy vulnerabilities, timestamp dependencies, and integer overflow vulnerabilities.

1. Introduction

Nakamoto [1] attracted a lot of attention when he first introduced the concept of blockchain [2,3] through the Bitcoin white paper in 2008. In recent years, the rapid development of virtual currencies and blockchain technology has had a significant impact on society and its financial value. American computer scientist Nick Sabo [4] proposed the concept of smart contracts [5,6] as early as 1994, but since computers were not powerful enough at the time, smart contracts were not widely used for a long time. As blockchain technology becomes more popular, smart contracts are getting new life. Now they are starting to be used in many different fields like finance [7,8], supply chain management [9,10,11], Internet of Things [12,13,14], and healthcare [15,16,17]. For instance, in September, 2023, Huawei and Beijing Bank showed, for the first time, the application of smart contracts in IoT scenarios. By equipping vehicles with OBUs (On Board Units), which communicate with the cloud control platform, real-time traffic information is provided. The OBUs are embedded with digital RMB IoT application terminals, which support digital RMB payments and enable multi-party intelligent account distribution through smart contracts in the Internet of Vehicles scenarios. These contracts can automatically execute the rules defined in the contract, thereby reducing trust costs, accelerating transaction processing speed, and improving overall transparency and security.

Currently, tens of millions of smart contracts have been deployed on blockchain platforms such as Ether [5], and the number is still growing at an explosive rate. Due to the characteristics of blockchain technology, such as irreversibility and tampering, the security of smart contracts has become particularly important. However, since the development of blockchain technology, the deployment of smart contracts has led to a series of major security vulnerabilities and attacks. A prime example of this is the famous DAO incident [18] in June 2016, in which hackers stole USD 60 millionworth of Ethereum by exploiting a reentrancy vulnerability. This incident occurred due to an undiscovered vulnerability in the deployment of smart contracts. It led hackers to steal the balance of the contract account by using repeated calls to the function to infinitely replicate transfers. Moreover, there was another hack involving a Parity multi-signature wallet [19] smart contract in 2017. This time, a hacker found a way to use something called a delegatecall bug to steal lots of Ethereum coins. Over 150,000 eth coins were taken, which at the time was worth around USD 30 million dollars. Then, there was also an integer overflow vulnerability in the contract of BEC (Beauty Chain) [20] in 2018. Attackers exploited this vulnerability to replicate BEC tokens indefinitely, ultimately leading to a plunge in the price of BEC tokens to zero. There are also some attacks, such as King of the Ether Throne [21], using a denial-of-service vulnerability to cheat tokens. These security incidents not only bring huge economic losses to the users of blockchain platforms but also undermine the foundation of people’s trust in smart contracts. The security of smart contracts has become a hot topic of current research.

Currently, deep learning technologies are gradually utilized to address smart contract security issues, achieving significant progress especially in the field of smart contract vulnerability detection. For instance, Qian et al. [22] utilized BiLSTM and an attention mechanism to detect smart contract vulnerability. Zhuang et al. [23] employed graph neural networks to construct contract graphs, facilitating subsequent vulnerability detection. Tang et al. [24] proposed the “Lightning Cat” solution, which comprises three deep learning models. These methods primarily consider global temporal features and contract graph features when extracting contract features, with inadequate consideration of local contract features. This may lead to insufficient robustness and generalization capabilities of the models, and the detection accuracy may need to be improved.

In this work, we propose an enhanced dual-channel smart contract vulnerability detection method (EDSCVD), which is mainly designed to improve the accuracy of smart contract vulnerability detection. It can help better solve the smart contract security problem. Firstly, the pre-training model BERT can better improve the performance and adaptability for smart contract source code analysis so as to enhance the model’s understanding of source code. Secondly, the adversarial training FGM is utilized to generate adversarial samples by disturbing the word embeddings, thereby enhancing the robustness and generalization of the model. Then, the word embeddings output from upstream are input to a dual-channel network composed of a CNN and a BiLSTM in parallel, combined with the multi-head attention mechanism, so as to capture the important contextual sentences in the smart contract source code. Finally, the detection and classification are performed through the full connected layer to determine whether there is some kind of common vulnerability in smart contracts, such as reentrancy, timestamp dependency, and integer overflow.

The main contribution of this paper can be summarized as follows:

• Pre-training enhancement: We use BERT to replace the traditional Word2vec pre-trained model to improve the performance and adaptability of the pre-processing of contract source code.
• Symmetry sample enhancement: We utilized the adversarial training method FGM (Fast Gradient Method) in the field of smart contract vulnerability detection. By utilizing FGM on word embeddings to generate symmetric adversarial samples, the model can better resist the attack of adversarial samples and enhance robustness and defense capabilities.
• Feature extraction enhancement: We propose a dual-channel model that optimizes the traditional downstream BiLSTM network model by adding CNN in parallel, which cannot only extract the local features of the contract source code but also retain its temporal information.
• We conducted experimental comparisons of EDSCVD with three traditional methods and two deep learning methods on the self-built SYSCD dataset. The experimental results show that our method with three enhancements exhibits outstanding detection performance in detecting reentrancy vulnerabilities, timestamp dependencies, and integer overflow vulnerabilities, with F1 scores of 96.06%, 95.97%, and 79.44%, respectively. These results surpass those achieved by existing methods.

The rest of this paper is organized as follows: Section 2 introduces related work. Section 3 describes the EDSCVD vulnerability detection method, including pre-training enhancement, symmetric sample enhancement, feature extraction enhancement, and vulnerability detection. Section 4 presents the experimental results. Section 5 provides future outlook and suggestions. Section 6 concludes the paper.

2. Related Work

In this section, we first review common types of smart contract security vulnerabilities, then introduce four traditional vulnerability detection methods, and finally elaborate on the current integration of smart contract vulnerability detection with deep learning networks.

2.1. Smart Contract Security Vulnerabilities

With the development of blockchain technology, more and more smart contracts are being deployed in the platform, which also means more types of vulnerabilities are being discovered. There are many known smart contract vulnerability types, and we have chosen three common types of vulnerabilities to study in this paper, which are reentrancy vulnerability [25], Timestamp Dependency vulnerability [26], and Integer Overflow vulnerability [27].

2.1.1. Reentrancy

Reentrancy vulnerability is one of the most common vulnerabilities in smart contracts, which mainly stems from the security issues when smart contracts handle external contract calls. The danger of a reentrancy vulnerability is that an attacker can bypass the intended logic and control flow of a contract through repeated execution of a malicious contract. It leads to incorrect state changes and loss of funds. In 2016, a serious reentrancy vulnerability incident occurred in The DAO [18], an Ethereum smart contract platform, which ultimately led to a hard fork of the Ethereum network [28]. Figure 1 shows the source code of the reentrancy vulnerability, which first calls the external contract msg.sender and then updates the called party’s state variable credit. the reentrancy vulnerability exploits this way of updating the state variable after the external call, which allows the malicious external contract to avoid the pre-call checking and enter into the function many times.

2.1.2. Timestamp Dependency

In blockchain, each block contains a timestamp that is used to record the generation time of the block. The timestamps of miner confirmations are used as conditional standards or to implement time constraints, and ordinary attackers are usually unable to break the timestamps to create a vulnerability. Timestamps are initially random. If miners can modify the timestamps of blocks to deceive the condition judgments in contracts, this may lead to unauthorized operations or the bypassing of certain restrictions. In this way, a malicious miner can win the payoff or reward locked in that smart contract. Figure 2 shows a code snippet of a lottery trivia game where a timestamp-dependent vulnerability occurs. Line 5 indicates the need to turn in 1 Ether [29] first to gain access to the contract balance. If the miner maliciously adjusts block.timestamp in line 6 to fulfill the condition, they will receive the Ether reward in the contract.

2.1.3. Integer Overflow

Integer overflow is a common type of vulnerability in smart contracts. When the value of an integer variable in a contract exceeds its maximum representable range, it will lead to data overflow or underflow. The calculation of integers in smart contracts is often closely related to the transaction of a user’s digital assets, and the occurrence of an integer overflow vulnerability can have serious results. For example, the uint 256 type represents a 256-bit unsigned integer in the range 0 to $2^{256}-1$, and overflow occurs when the result of the operation exceeds the maximum value that can be represented by the data type. An overflow in an unsigned integer causes the result to go back to 0, restarting the count from the maximum value. Figure 3 shows the issue source code for the integer overflow vulnerability. Uint256 in line 4 of the code represents a 256-bit unsigned integer, and when the passed-in _value is too large, the value of the amount exceeds the data range of uint256, and thus, an overflow occurs [30].

2.2. Traditional Methods

Symbolic execution [31] statically analyzes each execution path of a smart contract program by symbolizing the program. It computes for each path the conditions that need to be satisfied to execute on that path, to verify that the smart contract is secure. It can achieve better detection coverage, but the problems of spatial explosion and execution path explosion are still the current problems faced by the method. Oyente [32] is one of the earliest tools for static detection of smart contract vulnerabilities. It conducts security checks on contracts through the symbolic execution of bytecodes based on the control flow graph of the contract. Mythril [33] is a symbolic execution engine that analyzes and detects various types of contracts based on EVM bytecode. Securify [34] is also a symbolic execution method. It analyzes the dependency graph of a contract as well as extracts precise semantic information from the code to check the security of the contract.

Formal verification [35] mainly relies on mathematical reasoning and logical rules to check whether contracts satisfy pre-defined security properties set by programmers, ensuring that vulnerabilities or errors do not occur during contract execution. This method can be used to infer complex contract vulnerabilities, but its low degree of automation is not conducive to large-scale detection. Common methods of formal verification include theorem proving, model checking, and deductive verification. ZEUS [36] is a smart contract detection method based on formal verification. It converts Solidity source code into a low-level virtual machine language and then utilizes abstract interpretation, symbolic model checking, and constraint statements to quickly verify the security of contracts. Additionally, VerX [37] and the KEVM framework [38] are both contract detection tools based on formal verification.

The fuzzy testing method [39] is a dynamic detection technique that discovers security vulnerabilities in smart contracts by generating a large amount of input data and monitoring the contract’s execution process. This method can cover more code paths and has the opportunity to trigger hidden vulnerabilities, but it requires a lot of time and computational resources to achieve sufficient coverage. ContractFuzzer [40] is one of the early research solutions for smart contract fuzzy testing. It modifies the Geth client based on the Go language version of Ethereum and conducts offline vulnerability detection by recording the instruction logs during the execution of smart contracts. ILF [41] is a neural network-based smart contract fuzzy tester. It utilizes neural networks to generate call sequences, aiding in the feature learning of the neural network model for effective vulnerability detection.

Intermediate representation (IR) refers to the conversion of smart contract source code into an intermediate representation specifically designed for vulnerability analysis. It provides a more abstract and structured form of representation, making the analysis and detection of vulnerabilities in smart contracts more convenient and effective. However, this method relies excessively on expert rules, and there is still room for improvement in terms of its universality and scalability. SmartCheck [42] is a tool that transforms Solidity source code into an intermediate representation for contract vulnerability detection. It utilizes lexical and syntax analysis methods to analyze smart contracts. Slither [43] is a static analysis framework that takes Solidity source code as input and converts it into an intermediate representation called SlithIR for vulnerability detection.

2.3. The Integration and Application of Deep Learning Networks

As the difficulty of smart contract vulnerability detection continues to rise, researchers have started to utilize deep learning techniques gradually for smart contract vulnerability detection. In recent years, there have been increasingly successful applications of deep learning in the field of program security [44,45]. For novel and complex types of security vulnerabilities, deep learning methods exhibit excellent scalability and adaptability, which can effectively enhance detection efficiency. Qian et al. [22] applied the BiLSTM model and attention mechanism to smart contract vulnerability detection, aiming to capture the control flow and semantic information within the contracts to identify vulnerabilities. Zhuang et al. [23] proposed a graph neural network-based method called GNNs, which constructs a contract graph to represent the syntax and semantic structure of smart contract functions. They further introduced a novel Dense Residual Graph Convolutional Network (DR-GCN) and a temporal message propagation network (TMP) for learning vulnerability detection from normalized graphs. Zhang et al. [46] presented the hybrid model SPCBIG-EC, which combines a serial-parallel convolutional bidirectional gated recurrent neural network model and an ensemble classifier. This model extracts features from combinations of multiple variables while preserving temporal and positional information. The ensemble classifier is then used for weighted classification detection. Zhen et al. [47] presented a smart contract vulnerability detection method called DA-GNN, which is based on a dual-attention graph neural network. This method involves transforming the opcode sequence of a contract into semantic features and a relationship feature matrix among nodes. By employing a dual-attention mechanism, the embedding representations of nodes and graph features are updated. Finally, classification and detection are performed. Tang et al. [24] proposed a solution named Lightning Cat, which comprises three deep learning-based models. This solution extracts vulnerability features by obtaining code function fragments containing vulnerabilities and uses the CodeBERT pre-trained model for data preprocessing to improve the semantic analysis ability of the model.

Table 1. Related research applications of deep learning techniques.

Methods	Years	Advantages	Disadvantages	Literature
BiLSTM-att	2020	simple structure, easy to understand	poor feature extraction, poor generalizability	[22]
GNNs	2020	novelty, automation	poor feature extraction, hard understand	[23]
SPCBIG-EC	2022	novelty, rich feature extraction	model complexity, low deployment efficiency	[46]
DA-GNN	2024	novelty, good generalization	model complexity, hard understand	[47]
Lightning Cat	2023	automation, easy to understand	low deployment efficiency, limited scalability	[24]

Source: Own elaboration.

systematically summarizes the above relevant research.

These methods have improved detection effectiveness at various levels, but they still suffer from issues of low robustness and generalization ability. To address this, we propose EDSCVD. Adversarial samples are generated through the sample enhancement module to help the model enhance its robustness. A dual-channel module is employed for enhanced feature extraction, aiding in increasing the model’s generalization ability. Finally, through experimental verification, EDSCVD effectively enhances the detection effect of smart contract vulnerabilities.

3. Our Method

In view of the existing problems in training and detecting models, we propose a new smart contract vulnerability detection method called the EDSCVD model based on a dual-channel network in combination with adversarial training FGM. Its framework is shown in Figure 4, where the input data for training and vulnerability detection is Solidity source code. Firstly, data preprocessing involves building a dataset and extracting code snippets that may contain vulnerabilities from contract source code. We use the pre-trained model BERT to replace the traditional models such as Word2vec and Fasttext, converting these segments into word embeddings required by downstream networks. Secondly, adversarial training is performed using the Fast Gradient Method (FGM) to perturb the word embeddings, thereby improving the stability and generalization ability of the model. Then, during the training phase, the basic BiLSTM is optimized by fusing CNN to form a dual-channel network structure. The pre-trained word embeddings are input into the dual-channel model for training and feature learning, and a multi-head attention mechanism is used to better explore the vulnerability information in smart contracts. Finally, in the vulnerability detection phase, based on the learning process of the upstream network, our model utilizes a fully connected network to obtain the final smart contract vulnerability detection results.

3.1. Pre-Training Enhancement

3.1.1. Smart Contract Fragment Representation

The contract data used in this study are sourced from real-world smart contracts transacted on the Ethereum platform, which includes a significant number of non-essential and blank statements. Due to the length limitations of our network model inputs, it is necessary to reduce irrelevant statements in the contract code to enable more precise detection of the contract code in the dataset. We slice the contract code in the dataset and represent smart contracts with contract fragments with key information. This paper primarily focuses on three common types of smart contract vulnerabilities, and

Table 2. Vulnerability characteristics of different vulnerabilities.

Vulnerability	Vulnerability Characteristics
Reentrancy	call.value(), fallback()
Timestamp Dependency	block.number, now, or block.timestamp
Integer Overflow	integer variables

Source: Own elaboration.

presents the key information representation of the discussed contract vulnerabilities in this paper.

Taking the reentrancy vulnerability as an example, Figure 5 shows the representation of smart contract source code using contract code segments. By performing slicing on the contract, irrelevant statements and comments unrelated to contract vulnerabilities were removed from the source code. Subsequently, each word or symbol in the contract was separated by delimiters to identify different types. Finally, the custom identifiers for function names in each code segment were replaced with FUNI, FUN2, and FUN3, while variable custom identifiers were replaced with VAR1, VAR2, and VAR3. This approach allows us to preserve the semantic information in the contract with the minimum data length without affecting the model’s vulnerability detection performance. It effectively eliminates unnecessary code and functions in the contract while retaining critical syntax and semantic information, reducing code length, and enhancing the conciseness and comprehensibility of the contract.

3.1.2. BERT

The pre-trained model [48] BERT is a context-based model that utilizes contextual information to better understand text. Compared with Word2vec and FastText, BERT adopts a bidirectional encoder structure that can consider both the left and right sides of the context at the same time, thus capturing more comprehensive semantic relations and dependencies. BERT adopts the Transformer architecture, which incorporates a self-attention mechanism. This allows it to process each word in the input sequence in parallel, thereby improving computational efficiency. In contrast, Word2vec and FastText have relatively simpler model structures and may exhibit more limited performance when handling complex tasks. Currently, BERT is a promising pre-trained model in Solidity source code detection applications. Figure 6 shows the structure of the BERT pre-trained model.

The processed contract fragments go to BERT in the form of tokens, which are then generated in the form of word embeddings. BERT tokenizes the input sequence based on the maximum input length specified by the model. For instance, given an input sentence $S=\{w_1,w_2,w_3,\dots w_n\}$, BERT adds a special token [CLS] at the beginning of the sentence and a separator token [SEP] between two sentences. At the same time, the input content is divided into three embeddings: Token embedding, Segment embedding, and Position embedding. These three embeddings are combined to form the input for BERT. Subsequently, the bidirectional Transformer is employed to extract features, resulting in word embeddings that capture rich semantic relationships. Through training the BERT model, we aim to enhance its applicability in the field of smart contracts, enabling more efficient and accurate batch detection of vulnerabilities.

3.2. Symmetric Sample Enhancement

Adversarial training is a training method that introduces noise by perturbing samples without significantly altering the distribution of the original samples. This allows the model to disregard such perturbations, thereby improving the model’s robustness and generalization in downstream tasks. FGM (Fast Gradient Method) is a commonly used adversarial training method in the field of NLP, initially proposed by Ian J. Goodfellow et al. in 2015 [50]. It can be used to generate symmetric adversarial samples, which can deceive the model and lead to wrong outputs after adding adversarial perturbations. By generating adversarial samples and incorporating them into the training data, the model can learn from a broader range of edge cases and exceptional scenarios, thereby enhancing its ability to identify unknown contract vulnerabilities or malicious contract vulnerabilities.

FGM mainly perturbs the word embeddings. The following Figure 7 depicts the disturbance term r, which is added to the word embeddings. Here, $V\in R^{(K+1)\ast D}$ represent the word embeddings generated by the pre-trained model, with K denoting the number of words in the vocabulary. Each $v_k$ corresponds to the i-th word embedding. It is important to note that the (K + 1)-th word embedding is utilized as the embedding for the end of sequence (eos) marker denoted as $v_{eos}$.

To prevent pathological perturbation solutions, the word embeddings are normalized by replacing the word embedding $v_k$ with the normalized word embedding ${\overline{v}}_k$, defined as shown in Equations (1)–(3).

$${\overline{v}}_k={\displaystyle \frac{v_k-E\left(v\right)}{\sqrt{Var\left(v\right)}}}$$

(1)

$$E\left(v\right)={\sum }_{j=1}^K{f}_j{v}_j$$

(2)

$$Var\left(v\right)={\sum }_{j=1}^K{f}_j{(v_j-E\left(v\right))}^2$$

(3)

where $f_i$ is the frequency of the i-th word, calculated during training. There are various ways to introduce perturbations to word embeddings. For instance, directly adding random noise to the word embeddings is one approach, but such random perturbations have limited adversarial strength. Therefore, the approach chosen in this study involves perturbing the word embeddings in the direction of gradient ascent, thereby increasing the target loss. The perturbation term that maximizes the loss is determined while keeping the parameters fixed. In Equation (4), x represents the input of the model and represents the model’s parameters. When FGM is applied to a specific model, the adversarial training incorporates the following formula into the cost function.

$$-logp\left(y\mid x+r_{adv};\theta \right)\mathrm{where}{r}_{adv}=\underset{r,\parallel r\parallel \le ϵ}{argmin}logp(y\mid x+r;\widehat{\theta })$$

(4)

where r is the perturbation term and $\widehat{\theta }$ is a constant set for the current parameters of a classifier. The above formula requires solving for an optimal disturbance term after each parameter update, which is computationally inefficient. Using the gradient of the scale as the update direction of the disturbance term allows us to quickly obtain better adversarial samples. In neural network models, the value of r is often difficult to minimize accurately, so a linearizing $logp\left(y\right|x;\widehat{\theta })$ centered around x is utilized to approximate this value. Ultimately, linear approximation and L2 norm constraints are employed to generate the adversarial perturbation $r_{adv}$, as calculated in Equation (5).

$$r_{adv}=-\in {g\left|\right|g\left|\right|}_2\mathrm{where}g={\nabla }_{x}logp\left(y\right|x;\widehat{\theta })$$

(5)

This perturbation can easily be computed using backpropagation in neural networks. The detailed procedure of the adversarial training method, FGM, is shown in Algorithm 1. The pre-trained word embeddings x pre-trained by BERT serve as the input, while the updated model parameters after adversarial training with added perturbations serve as the output. Firstly, the forward loss of x is calculated, and gradients are obtained through backpropagation without updating the gradients (line 1). Then, the perturbation value $r_{adv}$ is computed based on Equation (8) using the gradients of the embeddings. It is added to x to obtain the adversarial sample $x_1$ (lines 2–3). The forward loss of the adversarial sample is computed, and gradients are accumulated onto the original gradients through backpropagation (lines 4–8). The embeddings are restored to their original state (line 9). Finally, the model parameters are updated based on the gradients of the adversarial sample (line 10). Through this process, we introduce perturbations into the word embeddings, enhance adversarial sample training, and improve the model’s performance.

Algorithm 1 Adversarial Training Methods FGM

Input: BERT Pre-trained word embeddings x. Output: Model parameters updated after training with adversarial samples.   1: Calculate the forward loss of x, backpropagate to get the gradient of x;   2: Based on the gradient of the embedding matrix, we calculate $r_{adv}$. Equation(5);   3: Calculate the confrontation sample $x_1,x_1=x+r_{adv}$;   4: if  $x_1=x+r_{adv}$ then   5:     Calculate the forward loss of $x_1$;   6:     backpropagate to get the $gradien{t}_1$ of $x_1$;   7:     gradient = gradient + $gradien{t}_1$; //gradient accumulation   8: end if   9: Restore embedding was (1); 10: Using the gradient of step (4) ($x_1$), the model parameters are updated; 11: return x;

3.3. Feature Extraction Enhancement

3.3.1. Dual-Channel Network Architecture

After being processed by the BERT pre-trained model, the data are outputted in the form of word embeddings. These word embeddings are then fed into our optimized dual-channel network module. We have added a CNN to the BiLSTM to form a parallel symmetric dual-channel structure. The CNN module is responsible for extracting local features from the smart contract source code data, while the BiLSTM module is responsible for extracting the overall temporal information in the source code. This structure not only extracts the key features of the data but also retains the overall temporal features. The specific structure of the dual-channel network is shown in Figure 8.

Within the CNN module, the input word embedding sequence x consists of n entries, with each entry represented by an m-dimensional dense vector. Hence, the input x is represented as a feature map with dimensions $m\ast n$. The convolutional layer performs feature learning by sliding filters. For the input x: $x_1,x_2,\dots x_i$, a vector $c\in R^{wd}$ is formed by concatenating w entries $(x_{i-w+1},\dots x_i)$, where w represents the filter width and $0<i<s+w$. The embedding of $x_i$ is padded with zeros for $i<1$ or $i>n$. We generate the representation p for the w-gram $x_{i-w+1},\dots x_i$ using convolution weights $W\in R^{d\ast wd}$, where bias $b\in R^m$, as shown in Equation (6).

$$p_i=tanh(W\ast c_i+b)$$

(6)

In the pooling layer, the final output $D_1=max(p_1,p_2,\dots ,p_m)$ by maxpooling.

The BiLSTM module is responsible for capturing temporal information in the input sequence, with the combination of forward and backward LSTM units forming the BiLSTM. Within each LSTM unit, there are gate mechanisms, including an input gate $i_t$, an output gate $o_t$, a forget gate $f_t$, and a cell state $c_t$. These gates allow for selective memory and forgetting of information while the current hidden state $h_t$ is outputted. The structure of an LSTM unit is shown in Figure 9. Similarly, the word embeddings x processed by BERT are input into the BiLSTM module for feature learning, and finally $D_2$ is generated.

After being processed by the CNN module, the data are ultimately outputted as $D_1$, while the data processed by the BiLSTM module are ultimately outputted as $D_2$. The outputs from both channels are concatenated together to form the final output of the dual-channel module, denoted as $D=concat(D_1,D_2)$. To provide a more detailed description of the dual-channel network module, Algorithm 2 presents the specific process of the dual-channel network in the form of code.

Algorithm 2 Dual-Channel Network Architecture

Input: BERT Pre-trained word embeddings x. Output: Feature Representation with Smart Contract Contextual Semantics.   1: Define the class of the dual-channel network;   2:    Define the constructor $\_\_init\_\_$;   3:       Initialize $self.bilstm$;   4:       Initialize $self.cnn$;   5:    Define function forward propagation;   6:       Define outputs;   7:       //BiLSTM   8:       Execute self.bilstm(outputs), assigning the result to bilstm_output;   9:       //CNN 10:       Execute self.cnn(outputs), assigning the result to cnn_output, Equation(6); 11:    Execute mix_output = bilstm_output + cnn_output; 12: END

Compared to an individual network model, the dual-channel approach offers the advantage of leveraging both the strengths of CNN for feature extraction and BiLSTM for capturing sequential and sequence-related information in the data. This allows for the preservation of both local feature information and global sequence-related features to the greatest extent possible.

3.3.2. Multi-Head Attention Mechanisms

In the multi-head attention mechanism, as shown in Figure 10, the input sequence is first linearly transformed into multiple sets of queries, keys, and values. Each set of queries, keys, and values is used to calculate attention weights. The outputs from multiple attention heads are concatenated together and then integrated through linear transformations to obtain the final output representation. The computation formula for single-head attention is shown in Equation (7).

$$Attention(Q,K,V)=softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(7)

In the above equation, $d_k$ represents the dimensionality of the embeddings, ensuring that the computed values are appropriately usable. The subsequent Equations (8)–(10) represent the calculation of the multi-head attention mechanism.

$$MultiHead(Q,K,V)=Concat(hea{d}_1,\dots hea{d}_n)W^O$$

(8)

$$hea{d}_i=Attention(Q{W}_i^Q,K{W}_i^K,V{W}_i^V)$$

(9)

$$MAT=Norm(X+MultiHead\ast W^O)$$

(10)

$$V_c=Maxpool\left(MAT\right)$$

(11)

In the above equations, $W_i^Q\in R^{d_{model}\ast d_k},W_i^K\in R^{d_{model}\ast d_k},W_i^V\in R^{d_{model}\ast d_k}$, $W_i^O\in R^{d_{model}\ast d_k}$, $d_{model}$ represents the output dimensionality of all sub-layers and the embedding layer in the entire Transformer model, h denotes the number of heads, and $d_k=d_v=d_{model}/h$. The values of $d_k$ and $d_v$ can vary during the execution phase of the model. $W^Q,W^K$, and $W^V$ are trainable weight matrices, while X represents the embedding matrix composed of word vectors $x_t$ outputted by the upstream network. When the n head vectors are combined, the word vector representation $V_c$ is obtained after residual addition, normalization, and max pooling operations.

3.4. Final Detection

$V_c$ is the feature representation processed by the multi-head attention mechanism, capturing context statements in the contract source code that hold significant weights. Subsequently, these feature representations are fed into a fully connected layer, where they are linearly combined with the model’s weights and nonlinearly mapped through an activation function. The scores for each vulnerability type are then computed. We employ the softmax function to normalize these scores, yielding probabilistic results for each vulnerability type. The specific calculation is shown in Equation (12).

$${\widehat{Y}}_c=softmax\left(FC\left(V_c\right)\right)$$

(12)

Lastly, the cross-entropy loss function is employed to measure the discrepancy between the actual vulnerability probability $Y_c$ and the predicted probability ${\widehat{Y}}_c$. This loss function optimizes the model by taking a weighted sum of the predictions for each vulnerability type, enabling more accurate vulnerability predictions in the contract. The formulation of this loss function is shown in Equation (13).

$$H(Y_c,{\widehat{Y}}_c)=-{\sum }_c{Y}_{c}log{\widehat{Y}}_c$$

(13)

In this context, ${\widehat{Y}}_c$ represents the predicted probabilities obtained by our approach, while $Y_c$ denotes the actual vulnerability probabilities. Through our method, we can detect the probabilistic results for each vulnerability type and use them to determine the presence of vulnerabilities in the contract.

4. Experimental Analysis

4.1. Datasets

We utilized web crawlers to obtain real and objective smart contract source code from the Ethereum official website to construct our dataset. The obtained dataset is labeled by two tools, Oyente and Smartcheck, and further manual verification of labeling for contracts that are inconsistently labeled by the tools. To augment the contract data, we also incorporated a portion of the SmartWild dataset collected by Durieux et al. [33] and a portion of the RSC dataset collected by QIAN et al. [22]. With the above two parts, we filtered out 18,188 smart contracts to form our SYSCD dataset. Among them, there are 6170 vulnerability samples, including 1253 reentrancy vulnerabilities, 2907 timestamp dependency vulnerabilities, and 2110 integer overflow vulnerabilities. There are no vulnerability samples 12,018. A smart contract may contain multiple vulnerability categories. We randomly selected 80% of the dataset for training and reserved the remaining 20% for testing.

4.2. Experimental Settings

Parameter Settings: The learning rate was set to 0.00002. The dropout was adjusted to 0.2. The optimizer was AdamW, and the model was trained for 60 iterations.

Experimental Environment: The CPU was i5-13500HX. The GPU was NVIDIA RTX 4050. The running memory was 16GB, and the disk capacity was 1.5TB. The experiment was executed on the Windows 11 platform, with the deep learning framework PyTorch 1.9 and Python version 3.7.

4.3. Evaluation Metrics

We employed widely used evaluation metrics that are widely used in the field of machine learning, such as F1 score, accuracy, recall, and precision, to provide a reasonable assessment of the detection results produced by our proposed model. The computation of these evaluation metrics is based on four key values: True Positives ($TP$): the detection result has vulnerabilities, and the actual label also has vulnerabilities. False Positives ($FP$): the detection result has vulnerabilities but the actual label does not have vulnerabilities. False Negatives ($FN$): the detection result does not have vulnerabilities but the actual label has vulnerabilities. True Negatives ($TN$): the detection result does not have vulnerabilities and the actual label also does not have vulnerabilities. The calculations for F1 score, accuracy, recall, and precision are shown in Equations (14)–(17).

$$Accuracy={\displaystyle \frac{TP+TN}{TP+FP+FN+TN}}$$

(14)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(15)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(16)

$$F1-Score={\displaystyle \frac{2\ast Precision\ast Recall}{Precision+Recall}}$$

(17)

4.4. Comparative Experiments

We experimentally compared our method with three traditional methods and two deep learning methods, totaling five methods on the SYSCD dataset. The experimental results are shown in

Table 3. Comparative experimental performance evaluation results.

Vulnerability	Methods	Performance Index
		Pre (%)	Rec (%)	F1 (%)	Acc (%)
Reentrancy	Oyente	63.16	58.90	51.81	55.14
	SmartCheck	54.21	41.05	55.71	47.27
	Mythril	51.58	42.25	37.04	39.48
	BiLSTM-Att	91.05	87.84	89.04	88.76
	SPCBIG-EC	92.63	89.19	91.66	90.05
	EDSCVD	96.85	96.05	96.05	96.05
Timestamp Dependency	Oyente	59.75	45.99	34.43	39.38
	SmartCheck	58.09	46.97	31.96	38.04
	Mythril	53.94	40.27	31.09	35.09
	BiLSTM-Att	92.32	96.92	88.00	92.24
	SPCBIG-EC	93.15	95.74	90.72	93.16
	EDSCVD	96.06	98.39	93.34	95.97
Integer Overflow	Oyente	56.63	35.48	49.11	41.20
	SmartCheck	44.75	46.88	31.25	37.50
	Mythril	49.72	58.33	34.65	43.47
	BiLSTM-Att	83.43	82.35	66.67	73.68
	SPCBIG-EC	83.98	84.62	67.69	75.21
	EDSCVD	87.85	92.39	69.67	79.44

Source: Own elaboration.

.

Traditional methods:

Oyente [32]: It is one of the earliest detection tools for smart contract vulnerability that utilizes symbolic execution methods. By analyzing and detecting potential vulnerabilities in the bytecode representation of smart contracts, it aims to identify possible vulnerabilities within the contracts.

SmartCheck [42]: It is a static analysis method for smart contract vulnerability detection based on an intermediate representation. Its purpose is to identify potential contract vulnerabilities and security risks. Integrated with various smart contract development platforms and blockchain platforms, it can provide developers with detailed detection reports and recommendations for remediation.

Mythril [33]: This is a static code analysis-based approach that utilizes symbolic execution techniques for smart contract vulnerability detection. It is compatible with various smart contract development platforms and integration tools and aims to discover potential vulnerabilities by simulating different transaction paths.

Deep Learning Methods:

BiLSTM-Att [22]: It is a deep learning method published by Qian et al. It combines bidirectional long short-term memory networks and attention mechanisms to capture crucial semantic information and control flow dependencies in smart contracts, thus enhancing the accuracy of contract vulnerability detection.

SPCBIG-EC [46]: It is a serial hybrid model published by Zhang et al. This model achieves excellent performance in smart contract vulnerability detection by combining convolutional neural networks and bidirectional gated recurrent neural networks in a sequential manner, along with an integrated classifier.

4.4.1. Comparison of Reentrancy Vulnerability Detection

The proposed EDSCVD method was compared with traditional approaches (Oyente, SmartCheck, and Mythril), followed by a performance comparison with two deep learning methods. The quantitative results from Table 2 reveal that among the traditional methods for vulnerability detection, Oyente, based on symbolic execution, exhibits relatively better performance in detecting reentrancy vulnerabilities, with an F1 score of 55.14%, surpassing SmartCheck and Mythril. However, our EDSCVD method significantly outperforms the traditional approaches in terms of detection effectiveness.

Subsequently, we proceeded to compare the deep learning methods, where SPCBIG-EC demonstrated superior detection performance compared to BiLSTM-Att, achieving an F1 score of 90.41%. SPCBIG-EC possibly enhances detection performance further by combining sequential convolutional neural networks and incorporating an integrated classifier. Our method consistently outperforms BiLSTM-Att and SPCBIG-EC in terms of detection effectiveness. This performance disparity can be attributed primarily to the superior analysis and comprehension capabilities of our pre-trained model, BERT, in relation to source code. Additionally, the inclusion of the FGM adversarial training method contributes to improved model stability and generalization to a certain extent.

4.4.2. Comparison of Timestamp Dependency Vulnerability Detection

We conducted a detection comparison with traditional methods (Oyente, SmartCheck, and Mythril). Among these traditional methods, Oyente achieves the highest detection accuracy of 59.75% for timestamp dependency vulnerabilities. It is noteworthy that Oyente’s F1 score reaches a maximum of only 39.38%. Visualizing the results in Figure 10, it is evident that our proposed method significantly outperforms the traditional approaches in terms of detection performance. This indicates that traditional methods may struggle to effectively detect timestamp dependency vulnerabilities.

Subsequently, we conducted a detection comparison using deep learning-based methods, where both of these methods, along with our EDSCVD approach, achieved F1 scores of over 90%. Our EDSCVD method demonstrated outstanding performance across all metrics, achieving an accuracy of 96.06%, precision of 98.68%, recall of 93.34%, and an F1 score of 95.97%. This indicates that our method is capable of effectively detecting vulnerabilities in smart contracts, thereby enhancing the overall security of the blockchain credit system.

4.4.3. Comparison of Integer Overflow Vulnerability Detection

In the detection of integer overflow vulnerabilities, Mythril emerges as the most effective among the traditional methods, as shown in

Table 4. Ablation experimental performance evaluation results.

Vulnerability	Methods	Performance Index
		Pre (%)	Rec (%)	F1 (%)	Acc (%)
Reentrancy	BERT	94.21	95.77	89.47	92.53
	BBMA	94.74	94.60	92.11	93.34
	BDCMA	95.26	93.51	94.74	94.21
	EDSCVD	96.85	96.05	96.05	96.05
Timestamp Dependency	BERT	93.98	98.19	89.26	93.74
	BBMA	93.78	98.18	89.26	93.51
	BDCMA	94.81	98.62	90.91	94.62
	EDSCVD	96.06	98.39	93.34	95.97
Integer Overflow	BERT	84.25	89.16	60.66	72.20
	BBMA	85.36	87.10	66.40	75.35
	BDCMA	86.74	89.36	68.85	77.77
	EDSCVD	87.85	92.39	69.67	79.44

Source: Own elaboration.

, with an F1 score of 43.47%. The relatively higher F1 score further indicates that Mythril achieves a good balance between precision and recall. Mythril exhibits a detection precision of 58.33%, which is significantly higher than the precision values of Oyente and SmartCheck. While Oyente attains the highest detection accuracy of 56.63%, its F1 score is lower than that of Mythril.

In the context of deep learning-based methods, both approaches exhibit relatively strong detection performance. However, when compared to our EDSCVD method, EDSCVD outperforms BiLSTM-Att and SPCBIG-EC by 5.76% and 4.23% in terms of F1 score, respectively. This further confirms the effectiveness of incorporating the BERT pre-trained model and the FGM adversarial training method, which allows us to focus more on the contract’s feature representation during training, thereby improving overall detection performance.

During the process of the model training, each Epoch requires training our complete SYSCD dataset. We set the experiment to 60 iterations, and the relevant parameter curves of the iterative training process are shown in Figure 11. We can see that when trained from 30 to 36 iterations, the four evaluation metrics are all in a good convergence state. In the iterative training of the Timestamp Dependency vulnerability, the model converges at 24 iterations and then gradually begins to decline after 36 iterations. In the training iterations of the other two types of vulnerabilities, the model converges around 30 iterations and gradually declines after 42 iterations. Taking everything into consideration, we choose the average value from 30 to 36 iterations as the final experimental result.

4.5. Ablation Experiments

To assess the impact of each module in the EDSCVD method on detection performance, this section compares the detection performance of three models to validate the effectiveness of our approach. The experimental results are shown in Table 4.

BERT: In the task of vulnerability detection, the pre-trained model of BERT serves as an initial model, which is fine-tuned and utilized with fully connected layers for classification, thereby aiding in the detection of vulnerabilities in smart contracts. Here, we use the BERT model as the baseline model for ablation experiments.

BBMA (BERT–BiLSTM–Multi head Attention): It is an augmented model based on the BERT baseline model with the introduction of the BiLSTM model and multi-head attention mechanism. BiLSTM is introduced to better capture sequence information in smart contracts. The multi-head attention mechanism further enhances the representational capacity of the model and the flexibility of the attention mechanism. This extended structure further improves the performance of smart contract vulnerability detection.

BDCMA(BERT–Dual Channel–Multi head Attention): It adds CNN to BBMA so that it forms a parallel symmetric dual-channel structure with BiLSTM. The introduction of BiLSTM is known to better capture sequential information within smart contracts, while the inclusion of the CNN module enables the extraction of local features from the source code data, thereby better preserving the key contract features.

In the detection of reentrancy vulnerabilities, all models achieved an F1 score of over 90%. After adding BiLSTM and multi-head attention mechanism, the F1 of the BBMA model is improved by about 1% to 93.34%. Furthermore, with the addition of the dual-channel module, the detection effect of BDCMA is further improved, and the F1 value is 94.21%. Finally, by adding the adversarial training method FGM, our EDSCVD method achieves better results, in which the F1 reaches 96.85%. The experimental results are sufficient to show that the method with the addition of the FGM adversarial algorithm has positive significance for vulnerability detection in smart contracts.

In the detection of timestamp dependency and integer overflow vulnerabilities, the F1 of the BDCMA model without the addition of the adversarial training FGM module is only 94.62% and 77.77%, and the F1 of BBMA has values of 93.51% and 75.35% for these two vulnerabilities. It is worth mentioning that our method EDSCVD has better detection performance than all the other three models, with F1 values of 95.97% and 79.44%. EDSCVD is an improvement of 2.23% and 7.24%, respectively, compared to the baseline model BERT. This difference reaffirms that the inclusion of the dual-channel module and the adversarial training FGM enhances the detection performance of the model. It also further improves the robustness and generalization ability of the model.

A comprehensive analysis of the performance and effectiveness of the proposed EDSCVD method in this paper was conducted. Experimental comparisons with other approaches revealed significant improvements in accuracy and F1 score for vulnerability detection achieved by our EDSCVD method. These findings provide evidence of the effectiveness and superiority of our model in the field of smart contract vulnerability detection.

5. Future Prospects and Suggestions

Although our method has good vulnerability detection, it still has shortcomings. Firstly, our method is limited by the input size of the pre-trained model, especially for a very small fraction of overly long smart contracts, so it lacks rigor. Secondly, the coverage of the types of detected vulnerabilities is not comprehensive enough, and the ability to detect multiple types of vulnerabilities is poor. Finally, there is a lack of applying more advanced graph neural networks or large models.

The application of smart contracts is developing in a more complicated direction. We propose the following recommendations for addressing smart contract security issues and the above shortcomings:

1.

Build authoritative datasets. Future work can be dedicated to collecting more data on unknown vulnerabilities to build a comprehensive and unified form of dataset. This can be accomplished by expanding on existing datasets to help better train and evaluate deep learning models.

2.

Establish a community platform. In the future, communication and sharing can be carried out through the smart contract vulnerability detection community platform. This will encourage researchers to share vulnerability samples, attack scenarios and detection experience. We can better discover and understand unknown vulnerabilities.

3.

Explore advanced methods. We can utilize more advanced graph neural networks or large models for vulnerability detection. These models can process longer input sequences but also possess stronger representation learning capabilities, allowing for a deeper capture of semantic information in contract codes.

4.

Research vulnerability remediation strategies. Currently, relevant research is still mainly focused on the detection of contract vulnerabilities. In the future, security research can be conducted on vulnerability contracts to help relevant researchers understand the logic behind contract vulnerabilities.

6. Conclusions

In this paper, a detection method based on EDSCVD is proposed for addressing the security vulnerabilities present in smart contracts within the blockchain ecosystem. Our EDSCVD method utilizes the adversarial training FGM to improve the model’s robustness and resilience against attacks. Furthermore, the integration of a symmetric dual-channel network consisting of a CNN and a BiLSTM enhances the feature extraction capability of the source code, leading to improved accuracy in vulnerability classification. The paper concludes with experiments to compare the vulnerability detection model proposed in this paper with traditional methods and other deep learning detection methods. The experimental results show that our method achieves F1 scores of 96.05%, 93.34%, and 69.67%, respectively, in detecting reentrancy vulnerabilities, timestamp dependencies, and integer overflows. Compared to traditional methods, our F1 scores are improved by over 40.43%, 58.91%, and 20.56%, respectively. When compared with other deep learning models, our F1 scores are also enhanced by more than 4.39%, 2.62%, and 1.98%, respectively. It has notable advantages in the smart contract vulnerability detection task. Our work has had a positive impact on the work of relevant academicians, practitioners, and managers. It not only helps scholars and practitioners improve the effectiveness and accuracy of smart contract vulnerability detection but also helps managers maintain the ecological security of smart contract platforms. We all work together to promote the secure development and wide application of smart contracts and lay a solid foundation for the wide application of smart contracts.