A Study on Double-Headed Entities and Relations Prediction Framework for Joint Triple Extraction

Abstract

Relational triple extraction, a fundamental procedure in natural language processing knowledge graph construction, assumes a crucial and irreplaceable role in the domain of academic research related to information extraction. In this paper, we propose a Double-Headed Entities and Relations Prediction (DERP) framework, which divides the entity recognition process into two stages: head entity recognition and tail entity recognition, using the obtained head and tail entities as inputs. By utilizing the corresponding relation and the corresponding entity, the DERP framework further incorporates a triple prediction module to improve the accuracy and completeness of the joint relation triple extraction. We conducted experiments on two English datasets, NYT and WebNLG, and two Chinese datasets, DuIE2.0 and CMeIE-V2, and compared the English dataset experimental results with those derived from ten baseline models. The experimental results demonstrate the effectiveness of our proposed DERP framework for triple extraction.

1. Introduction

With the development of natural language processing and knowledge graphs, data storage and presentation methods for structured text have become more mature, but there are still many unsolved problems in the processing of unstructured and semi-structured text [1]. Extracting triple groups is crucial in natural language processing and knowledge graph construction. In constructing knowledge graphs, unstructured texts usually extract entities and form correspondences by forming a (head entity, relation, tail entity) triple.

Existing triple extraction methods mainly include two major kinds, pipeline extraction methods and joint extraction methods. Traditional pipeline extraction methods divide knowledge extraction into two subtasks [2]: named entity recognition and relation extraction. However, this approach ignores potential information interactions between entities and relations, leading to incorrect relation extractions or failure to recognize entity relations. Many previous experiments have demonstrated that a joint learning approach greatly improves the effectiveness of entity and relation extraction due to the consideration of the information interactions between the two subtasks, so most of the current research for the task of entity and relation extraction adopts the joint learning approach.

In recent scholarship, there has been a notable surge in research attention directed toward the intricacies of overlapping triples, as shown in Figure 1. This phenomenon is exemplified in sentences wherein there is the potential presence of both entity pair overlap (EPO) triples and single entity overlap (SEO) triples. This burgeoning area of inquiry underscores the escalating interest in dissecting and comprehending the complexities inherent to overlapping triples in textual data.

Previous research has revealed several shortcomings in the extraction of multiple relationships (overlapping triples) within the same entity. For example, the NovelTagging method uses a joint decoding of sequence annotations to treat entity and relation extraction as a sequence annotation problem [3]; however, this method only assigns a single label to each token, rendering it incapable of handling overlapping triples in the data. In contrast, the CasRel framework models relations as functions that map subject to object [4], successfully overcoming the issue of poor handling of overlapping triples by previous models. Nevertheless, the CasRel framework suffers from the disadvantage of incorrectly identifying the head entity, leading to failure in identifying the relation and the tail entity. An overview of the CasRel framework structure is shown in Figure 2.

In this study, a head entity recognition module is used to predict the triple related to the head entity and a tail entity recognition module is added to predict the triple related to the tail entity. Combining the information from the two modules results in a triple of higher accuracy. Experimental results show that the performance of the framework is improved by combining the BERT encoder. This work contributes as follows:

• A double-headed entities and relations prediction framework for joint triple extraction based on the BERT encoder is proposed. The named entity recognition task is decomposed into head entity recognition and tail entity recognition.
• To ensure recognition accuracy, a triple prediction module, which gives different weights to the triple derived from the head entity recognition and the triple extracted from the tail entity recognition, is set up to improve the accuracy of triple extraction.
• To validate the method, experiments were conducted on two English public datasets, NYT and WebNLG, and two Chinese datasets, DuIE2.0 and CMeIE-V2, and the proposed framework was compared with ten baselines.

2. Related Work

In recent years, many methods have been proposed to accomplish knowledge extraction that can be categorized into pipeline extraction methods and joint learning methods based on the learning process.

2.1. Pipeline Extraction Methods

Usually, pipeline extraction methods consist of the entity recognition stage and the relation extraction stage, where the output of the previous stage becomes the input of the next stage. This approach has the advantage that a specific model can handle a responding task, but it may also lead to errors accumulating in each stage.

The primary objective of named entity recognition (NER) is to identify and classify named entities within textual content, such as people, places, time, purpose, etc., with specific meanings. It is mainly responsible for automatically extracting the basic element entities in the knowledge graph from the unstructured and semi-structured. In order to uphold the quality of the knowledge graph, it is imperative to ascertain the precision and comprehensiveness of the entities extracted therein. Li et al. proposed a meta-learning method, integrating distributed systems with a meta-learning approach to extract relations among Chinese entities [5]. Through the utilization of machine learning and neural network methodologies, particularly leveraging the attention mechanism within the domain of natural language processing, Li et al. proposed a combination of conditional random fields (CRF) and bidirectional long short-term memory (BILSTM) for extracting information in a mathematical language [6]. Luo et al. introduced a neural network model, known as the attention-based bidirectional long short-term memory with a conditional random field layer (Att-BiLSTM-CRF), for document-level chemical entity recognition [7]. Li et al. advocated the utilization of distinct layers, specifically long short-term memory (LSTM) for text feature extraction and conditional random field (CRF) for label prediction decoding [8]. Ren proposed a method to enhance entity recognition by transforming text into a vector representation combining contextual and global features through a pretrained model and a graph neural network GCN [9].

Relation extraction refers to extracting relations between connecting basic element entities from the unstructured and semi-structured. The mesh structure of the knowledge graph is similar to the structure of the brain for storing knowledge. Neurons represent entities and record basic information, and the process of extracting relations activates some of the neurons (entities) and adds them to the brain structure (knowledge graph), using relations to connect the entities to the whole knowledge graph. Zeng et al. conducted an analysis of the pivotal role played by the order of relation extraction and employed reinforcement learning techniques to ameliorate the efficiency of relation extraction [10]. Han et al. proposed a one pass model based on BERT, capable of predicting entity relations by processing the text in a single pass [11]. Chen et al. utilized a neutralized feature engineering approach for entity relation extraction, namely, enhancing neural networks with manually designed features [12]. Yuan et al. proposed a relation-aware attention network to construct relation-specific sentence representations [13]. Wan et al. proposed a span-based multi-modal attention network (SMAN) for joint entity and relation extraction [14].

2.2. Joint Learning Methods

In pipeline learning methods of relation extraction, the intrinsic connection between entities and relations is often overlooked, and the federated model is an excellent solution to this problem. Huang et al. suggested using soft label embedding as an effective means to facilitate information exchange between entity recognition and relation extraction [15]. Wei et al. proposed a novel cascade binary tagging framework (CASREL), which models relations as functions that map subjects to objects [4]. Liu et al. introduced an attention-driven integrated model, primarily comprising an entity extraction module and a relation detection module, as a means to effectively confront the prevailing challenges [16]. Yu et al. decomposed the comprehensive extraction task into two mutually interconnected subtasks: one subtask handles the head entities, and the other subtask deals with the tail entities related to the head entities and their respective relations [17]. Guo et al. introduced an integrated model for the extraction of entities and relations pertaining to concepts within the realm of cybersecurity (CyberRel) [18], and they adopted a perspective wherein the triple is conceived as a sequence of entity relations. Subsequently, Lv et al. constructed the joint extraction of entity mentions and relations model, which was based on the bidirectional long short-term memory and maximum entropy Markov model (Bi-MEMM) [19]. Zheng et al. introduced an integrated framework for extracting relational triples, underpinned by the principles of potential relation and global correspondence (PRGC) [20]. Li et al. proposed a relation-aware embedding mechanism (RA) for relation extraction, with attention mechanisms being used to merge relational tags into sentence embeddings, which are used to distinguish the importance of relational tags for each word [21]. Huang et al. proposed a novel translation-based unified framework, which is used to solve redundant predictions, overlapping triples, and relation connections problems [22]. Liu et al. presented a model referred to as the bidirectional encoder representation from transformers–multiple convolutional neural network (BERT–MCNN), which has demonstrated a high level of accuracy and stability [23].

3. CasRel Framework

The goal of triple extraction is to identify all possible triples (head entity, relation, tail entity) in a sentence, which may contain some overlapping and shared entities. The structure of the CasRel framework is shown in Figure 2. The CasRel framework presents a fresh perspective on the task of triple extraction. It introduces a novel cascade binary tagging framework, known as CasRel, that effectively addresses the complex challenge of managing overlapping relations by systematically establishing subject–object mappings within sentences [4]. This framework consists of a set of functions that identify entities and their related relations in an entity tagger and relation-specific object taggers. By employing the CasRel framework, the issue of sharing the same entity in multiple triples is addressed effectively, providing multiple related relations and corresponding entities for each entity. However, in the CasRel framework, if the subject tagger does not recognize an entity, the associated triad will be missed.

To solve the triple extraction omission that occurs in the CasRel framework, we propose an improved DERP framework based on the CasRel framework. Which improves the entity recognition accuracy by adding a tail entity recognition module in the entity tagger, and adding a triple prediction module after relation-specific object taggers. This framework will combine head entities, tail entities and relations to make predictions and comes up with a more accurate triple.

4. The DERP Framework

Entity recognition and relation extraction are the design priorities for triple extraction. The primary objective of this DERP framework is to ascertain the complete set of potential triples within a given sentence, acknowledging the potential existence of entities with overlapping attributes in some instances.

The ultimate prediction of the (head entity, relation, tail entity) triple is achieved through the recognition and forecasting of the acquired triples within the triple prediction layer. The DERP framework is shown in Figure 3.

In the DERP framework, we model relations as functions that map topics to objects. We optimize the previously commonly used learning relation classifiers $f\left(E_1,E_2\right)\to R$, to learning relation-specific taggers $f_R\left(E_1\right)\to E_2$. Each tagger will identify entities that may exist under a specific relationship, or entities that may not be returned. If the entity is not returned, it indicates that there is no triple in the current entity and relation.

When dealing with overlapping triples, the DERP framework uses an entity tagger for entity recognition and allows multiple relationship representations in relation-specific entity taggers. Within relation-specific entity taggers, multiple relationships and their corresponding entities can be obtained. By using the DERP framework, different types of data structures, including EPO triples and SEO triples, can be effectively handled.

We used an entity tagger to identify head entities at the very beginning of the research on framework development and used the identified head entities to find related relations and tail entities. During the experiments, it was found that if there is a head entity in the entity tagger that is missing, this triple will be missed in the final triple prediction, especially in the case of overlapping triples where a head entity corresponds to more than one related tail entity. There are also cases where some of the tail entities related to this head entity are missed when performing the triple extraction; in this case, we can better find these missing tail entities by adding a tail entity recognition module to the entity tagger. So, two matching entities and accurate relations between entities are achieved by adding a tail entity recognition module to the entity tagger, and by looking up the corresponding relation and another matching entity in the relation-specific entity taggers.

During the experiment, by learning and improving the previous model, we added the tail entity recognition module. If the probability of recognizing the correct triple by the head entity module only is $P\left(Head\right)$ and the probability of recognizing the correct triple by the tail entity module only is $P\left(Tail\right)$, we will increase the probability of finally recognizing the correct triple by combining the two entity modules with the following probability equation:

$$\begin{gathered} P\left(Triple\right)=P\left(Head\right){{\displaystyle \cup }}^{ }P\left(Tail\right) \\ =P\left(Head\right)+P\left(Tail\right)-P\left(Head{{\displaystyle \cap }}^{ }Tail\right) \end{gathered}$$

(1)

where $P\left(Triple\right)$ is the probability of obtaining the correct triple, $P\left(Head\right)$ is the probability of obtaining the correct triple by only using a single head entity recognition module, $P\left(Tail\right)$ is the probability of obtaining the correct triple by only using a single tail entity recognition module, and $P\left(Head{{\displaystyle \cap }}^{ }Tail\right)$ is the probability of duplicate triples obtained by the head entity recognition module and tail entity recognition module.

4.1. BERT Encoder

BERT mainly consists of N layers of transformer block. A BERT encoder extracts sentence feature information from sentence S and inputs the feature information into the entity tagger.

$$h_0=O_{hot}{W}_n+W_p$$

(2)

$$S_{ri}=BERT\left(ri\right)$$

(3)

where $O_{hot}$ is the one-hot vector matrix indexed in the input sentence, $W_n$ is the word embedding matrix, $W_p$ is the positional embedding matrix, $p$ in $W_p$ denotes the positional index in the input sequence, and $S_{ri}$ is the i-th relation type embedding.

4.2. Entity Tagger

Compared with the CasRel framework, the entity recognition is divided into head entity recognition and tail entity recognition in the entity tagger, which reduces the situation of missing triples due to the omission of the first stage of entity recognition, and also improves the accuracy of the extraction of overlapping triples [24].

The BERT encoded sentence is entered in the entity tagger to extract head and tail entities by the binary method.

Within the entity tagger, the identification of entity positions within sentences encoded by the BERT encoder is achieved. In this module, two binary classifiers are designed to check for the start and end positions of entity words. By setting specific thresholds, if the probability surpasses the designated threshold, the token is marked as 1; otherwise, it is marked as 0. The following is specific to the head entity recognizer and tail entity recognizer:

$$p_i^{\mathrm{HE}\_\mathrm{start}}=sigmoid\left(W_{start}^{HE}{x}_i^{HE}+b_{start}^{HE}\right)$$

(4)

$$p_i^{\mathrm{HE}\_\mathrm{end}}=sigmoid\left(W_{end}^{HE}{x}_i^{HE}+b_{end}^{HE}\right)$$

(5)

$$p_i^{\mathrm{TE}\_\mathrm{start}}=sigmoid\left(W_{start}^{TE}{x}_i^{TE}+b_{start}^{TE}\right)$$

(6)

$$p_i^{\mathrm{TE}\_\mathrm{end}}=sigmoid\left(W_{end}^{TE}{x}_i^{TE}+b_{end}^{TE}\right)$$

(7)

where $p_i^{\mathrm{HE}\_\mathrm{start}}$, $p_i^{\mathrm{HE}\_\mathrm{end}}$, $p_i^{\mathrm{TE}\_\mathrm{start}}$, and $p_i^{\mathrm{TE}\_\mathrm{end}}$ are the probability of the marker position being predicted to be the start and end positions of the head entity and the tail entity, $x_i$ denotes the i-th marker in sentence S, $W_{start}^{HE}$, $W_{end}^{HE}$, $W_{start}^{TE}$, and $W_{end}^{TE}$ denote the training weights of the head entities and tail entities, and $b_{start}^{HE}$, $b_{end}^{HE}$, $b_{start}^{TE}$, and $b_{end}^{TE}$ denote the bias of the head entities and tail entities. In the use of the model, we need to keep the dimensions of the start binary classifier and the end binary classifier the same.

The entity recognition module uses the following likelihood function to recognize the range of sentences that have been encoded by the encoder:

$$p_{\theta }\left(E_{Head}|X_{HE}\right)={{\displaystyle \prod }}_{t\in \left\{H\mathrm{E}\_\mathrm{start},\mathrm{HE}\_\mathrm{end}\right\}}{{\displaystyle \prod }}_{i=1}^L{\left(p_i^t\right)}^{I\left\{y_i^t=1\right\}}{\left(1-p_i^t\right)}^{I\left\{y_i^t=0\right\}}$$

(8)

$$p_{\theta }\left(E_{Tail}|X_{TE}\right)={{\displaystyle \prod }}_{t\in \left\{\mathrm{TE}\_\mathrm{start},\mathrm{TE}\_\mathrm{end}\right\}}{{\displaystyle \prod }}_{i=1}^L{\left(p_i^t\right)}^{I\left\{y_i^t=1\right\}}{\left(1-p_i^t\right)}^{I\left\{y_i^t=0\right\}}$$

(9)

where L is the length of the sentence, $I\left\{z\right\}=1$ if $z$ is true and 0 otherwise, $y_i^{HE\_start}$, $y_i^{HE\_end}$, $y_i^{TE\_start}$, and $y_i^{TE\_end}$ are the i-th tag in the sequence that marks the start position and the end position.

4.3. Relation-Specific Entity Taggers

In the relation-specific entity taggers, an entity tagger is assigned to each relation word. The relation terms are used to correspond to the head entity or tail entity extracted in the previous layer to extract the entity in satisfying the relations. The calculations are shown below:

$$p_i^{\mathrm{HR}\_\mathrm{start}}=sigmoid\left(W_{start}^{HR}\left(x_i^{HE}+v_E^k\right)+b_{start}^{HR}\right)$$

(10)

$$p_i^{\mathrm{HR}\_\mathrm{end}}=sigmoid\left(W_{end}^{HR}\left(x_i^{HE}+v_E^k\right)+b_{end}^{HR}\right)$$

(11)

$$p_i^{\mathrm{TR}\_\mathrm{start}}=sigmoid\left(W_{start}^{TR}\left(x_i^{TE}+v_E^k\right)+b_{start}^{TR}\right)$$

(12)

$$p_i^{\mathrm{TR}\_\mathrm{end}}=sigmoid\left(W_{end}^{TR}\left(x_i^{TE}+v_E^k\right)+b_{end}^{TR}\right)$$

(13)

where $p_i^{\mathrm{HR}\_\mathrm{start}}$, $p_i^{\mathrm{HR}\_\mathrm{end}}$, $p_i^{\mathrm{TR}\_\mathrm{start}}$, and $p_i^{\mathrm{TR}\_\mathrm{end}}$ are the probabilities that the head entity and the tail entity at the labeled position are predicted to be the entity start position and end position, $v_E^k$ is the relation-specific entity tagger’s vector of coded representations of the kth subject detected in the module, $W_{start}^{HE}$, $W_{end}^{HE}$, $W_{start}^{TE}$, and $W_{end}^{TE}$ denote the training weights of the head entities and tail entities, and $b_{start}^{HE}$, $b_{end}^{HE}$, $b_{start}^{TE}$, and $b_{end}^{TE}$ indicate deviations of head entities and tail entities.

Relation-specific entity taggers use the following likelihood function to identify the range of sentences that the encoder has encoded:

$$p_{\theta }\left(E_{Tail}|E_{Head},X_{HE}\right)={{\displaystyle \prod }}_{t\in \left\{H\mathrm{E}\_\mathrm{start},\mathrm{HE}\_\mathrm{end}\right\}}{{\displaystyle \prod }}_{i=1}^L{\left(p_i^t\right)}^{I\left\{y_i^t=1\right\}}{\left(1-p_i^t\right)}^{I\left\{y_i^t=0\right\}}$$

(14)

$$p_{\theta }\left(E_{Head}|E_{Tail},X_{TE}\right)={{\displaystyle \prod }}_{t\in \left\{\mathrm{TE}\_\mathrm{start},\mathrm{TE}\_\mathrm{end}\right\}}{{\displaystyle \prod }}_{i=1}^L{\left(p_i^t\right)}^{I\left\{y_i^t=1\right\}}{\left(1-p_i^t\right)}^{I\left\{y_i^t=0\right\}}$$

(15)

where $L$ is the length of the sentence, $I\left\{z\right\}=1$ if $z$ is true and 0 otherwise, and $y_i^{HE\_start}$, $y_i^{HE\_end}$, $y_i^{TE\_start}$, and $y_i^{TE\_end}$ are the i-th tags in the sequence that marks the start position and the end position.

4.4. Triple Prediction

The relation-specific entity taggers identify the head entity, tail entity, and the corresponding relations and use the method of entity relation prediction to match the head entities and tail entities identified in the entity tagger using the following method:

$$f_{HE\_start}=\{\begin{array}{c}1, p_i^{\mathrm{HR}\_\mathrm{start}}\ge {\lambda }_1\\ 0, p_i^{\mathrm{HR}\_\mathrm{start}}<{\lambda }_1\end{array}$$

(16)

$$f_{HE\_end}=\{\begin{array}{c}1, p_i^{\mathrm{HR}\_\mathrm{end}}\ge {\lambda }_2\\ 0, p_i^{\mathrm{HR}\_\mathrm{end}}<{\lambda }_2\end{array}$$

(17)

$$f_{TE\_start}=\{\begin{array}{c}1, p_i^{\mathrm{TR}\_\mathrm{start}}\ge {\lambda }_3\\ 0, p_i^{\mathrm{TR}\_\mathrm{start}}<{\lambda }_3\end{array}$$

(18)

$$f_{TE\_end}=\{\begin{array}{c}1, p_i^{\mathrm{TR}\_\mathrm{end}}\ge {\lambda }_4\\ 0, p_i^{\mathrm{TR}\_\mathrm{end}}<{\lambda }_4\end{array}$$

(19)

When $f_{HE\_start}$, $f_{HE\_end}$, $f_{TE\_start}$, and $f_{TE\_end}$ equal to 1, the head entity or tail entity corresponding to the entity extracted in entity tagger and the corresponding relation is obtained, and if the value is equal to 0, the triple is excluded. ${\lambda }_1$, ${\lambda }_2$, ${\lambda }_3$, and ${\lambda }_4$ are the set thresholds.

$$g_{HE\_TE}=\left\{HeadEntity, Relation,TailEntity\right\}$$

(20)

$$g_{TE\_HE}=\left\{HeadEntity,Relation,TailEntity\right\}$$

(21)

$$g=g_{HE\_TE}{{\displaystyle \cup }}^{ }{g}_{TE\_HE}$$

(22)

where $g_{HE\_TE}$ represents the triplets of the tail entity and the relation between entities obtained based on the head entity, $g_{TE\_HE}$ represents the triplets of the head entity and the relation between entities obtained based on the tail entity, and $g$ denotes the final predicted triplets.

4.5. Loss Function

The training loss is defined as below:

$$\begin{gathered} \mathcal{L}=\sum _{j=1}^{|D|}[\sum _{E\in T_j}\log p_{\theta }\left(E_{Head}\right|X_j^{HE})+\sum _{E\in T_j}\log p_{\theta }\left(E_{Tail}\right|X_j^{TE}) \\ +{\displaystyle \sum }_{r\in T_j\mid E}\log p_{{\varphi }_r}\left(E_{Tail}\mid E_{Head},X_j^{HE}\right) \\ +{\displaystyle \sum }_{r\in T_j\mid E}\log p_{{\varphi }_r}\left(E_{Head}\mid E_{Tail},X_j^{TE}\right) \\ +{\displaystyle \sum }_{r\in R\setminus T_j\mid E}\log p_{{\varphi }_r}\left(E_{Tail}{}_{\varnothing }\mid E_{Head},x_j\right) \\ +{\displaystyle \sum }_{r\in R\setminus T_j\mid E}\log p_{{\varphi }_r}\left(E_{Head}{}_{\varnothing }\mid E_{Tail},x_j\right)] \end{gathered}$$

(23)

where parameters $\theta =\left\{\theta ,{\left\{{\varnothing }_r\right\}}_{r\in R}\right\}$, $p_{\theta }\left(E_{Head}\mid X_{HE}\right)$, and $p_{\theta }\left(E_{Tail}\mid X_{TE}\right)$ are defined in Equations (7) and (8), and $p_{{\varphi }_r}\left(E_{Tail}\mid E_{Head},X_{HE}\right)$ and $p_{{\varphi }_r}\left(E_{Head}\mid E_{Tail},X_{TE}\right)$ are defined in Equations (13) and (14).

5. Experiments

The effectiveness of the proposed framework is validated with experiments. The datasets and evaluation metrics are first introduced, and then the model names are compared with different baseline models.

5.1. Experiment Setup and Experiment Description

As most of the previous studies conducted experiments using English datasets, this study conducted experiments using two publicly English available datasets, NYT [25] and WebNLG [26], and compared the results of the experiments with 10 baseline models. Due to the specificity of the Chinese language, the complexity and difficulty of Chinese triple extraction is considerably greater than that of English relations [27]. We used two Chinese datasets, DuIE2.0 [28] and CMeIE-V2 [29]. DuIE2.0 is the most comprehensive Chinese relational extraction dataset in the industry [30]. CMeIE-V2 is a Chinese medical information extraction dataset, specifically designed for pediatrics and covering more than a hundred common diseases.

This model performs head entity recognition and tail entity recognition in the entity recognition part and performs the corresponding triple extraction based on the experimental results. In the experiments, the head entity recognition model and the tail entity recognition model are used individually for comparison experiments to verify the reliability and validity of the experiments. The schematic diagram of the head entity recognition module and the tail entity recognition module is shown in Figure 4.

The DERP framework is implemented using TensorFlow. In the BERT encoder section, the framework is implemented on English datasets using the cased_L-12_H-768_A-12 model and on Chinese datasets using the RoBERTa model. Dropout is applied to word embeddings and hidden states with a rate of 0.1. Network weights are optimized with Adam. The learning rate is set as 1 × 10⁻⁵. The max length of the input sentence is set to 100. The batch size is set as 6. We use 100 epochs and choose the model with the best performance on the validation set to output results on the test set.

In our experimental procedures, for the sake of maintaining consistency with prior research, an extracted triple is deemed accurate if the head entity, the relation, and the tail entity are each validated as correct. The study reports standard metrics, including micro-precision (Prec.), recall (Rec.), and F1 score (f1), in line with the established baselines.

5.2. Baseline

To evaluate the performance of the DERP Framework, it is compared with ten baseline models: NovelTagging [3], CopyRE [31], GraphRel [32], ETL-Span [17], CopyMTL [33], CasRel [4], TPLinker [34], RSAN [13], CGT [35], and RIFRE [36].

Unless otherwise noted, the results of these baseline models were taken from the original papers.

5.3. Results

Table 1. Precision (%), recall (%) and F1 score (%) of the compared models on the NYT and WebNLG databases. * marks results quoted directly from the original papers.

Model	NYT			WebNLG
	Prec.	Rec.	f1	Prec.	Rec.	f1
NovelTagging* [3]	61.5	41.4	49.5	-	-	-
CopyRE* [31]	61.0	56.6	58.7	37.7	36.4	37.1
GraphRel* [32]	63.9	60.0	61.9	44.7	41.1	42.9
ETL-Span* [17]	53.8	65.1	59.0	84.3	82.9	83.1
CopyMTL* [33]	75.7	68.7	72.0	58.0	54.9	56.4
CasRel* [4]	89.7	89.5	89.6	93.4	90.1	91.8
TPLinker* [34]	91.3	92.5	91.9	91.8	92.0	91.9
RSAN* [13]	85.7	83.6	84.6	80.5	83.8	82.1
CGT* [35]	94.7	84.2	89.1	92.9	75.6	83.4
RIFRE* [36]	93.6	90.5	92.0	93.3	92.0	92.6
DERP	92.05	89.94	90.98	92.82	92.90	92.86
DERP_HeadEntity	91.12	90.47	90.80	92.10	92.18	92.28
DERP_TailEntity	92.03	72.49	81.10	93.42	86.70	90.35

shows the results of our model relative to other baselines extracted from entities and relations on both datasets. On the WebNLG dataset, DERP outperformed all baselines in both recall and F1 score, and on the NYT dataset, DERP achieved the second highest F1 score. These results directly validate the utility of the proposed DERP framework.

Table 2. Precision (%), recall (%) and F1 score (%) of the compared models on the DuIE2.0 and CMeIE-V2 databases. * marks results of reproduced experiments.

Model	DuIE2.0			CMeIE-V2
	Prec.	Rec.	f1	Prec.	Rec.	f1
CasRel*	69.56	65.54	67.49	47.56	42.56	44.91
DERP	71.06	65.35	68.09	47.51	46.11	46.80
DERP_HeadEntity	70.38	65.80	68.01	47.27	45.15	46.19
DERP_TailEntity	73.97	53.50	62.09	49.10	43.01	45.85

shows the experimental results of DERP on the DuIE2.0 and CMeIE-V2 datasets, which shows an improvement over CasRel in terms of F1 score results. The F1 score of DERP_HeadEntity is also higher than CasRel when experiments are conducted using DERP_HeadEntity.

We conducted experiments on CasRel under the same experimental conditions as the DERP framework. On the NYT dataset, CasRel* scored precision 88.87%, recall 90.34%, and F1 score 89.60%; on the WebNLG dataset, CasRel* scored precision 91.92%, recall 91.39%, and F1 score 91.65%. Compared with the replicated CasRel* framework, DERP has 1.38 percent improvement in F1 score on the NYT dataset, 1.21 percent improvement in F1 score on the WebNLG dataset, 0.6 percent improvement in F1 score on the DuIE2.0 dataset, and 1.98 percent improvement in F1 score on the CMeIE-V2 dataset. On the four datasets of NYT, WebNLG, DuIE2.0, and CMeIE-V2, in the experiments using head entity recognition and tail entity recognition alone for triple prediction, DERP_HeadEntity has higher precision, recall and F1 score than the original CasRel model in the experiments. In the DERP tail entity experiment, the features of the tail entity are not as easy to recognize as the features of the head entity, resulting in weaker F1 experimental results than DERP_HeadEntity on the four datasets.

Table 1 also presents that in the experiments on the two English datasets, with the existing models compared, a significant gap in processing performance between the models is found, which proves that DERP performs better in dealing with redundant entities and overlapping triples. In the comparison experiments on four datasets, NYT, WebNLG, DuIE2.0, and CMeIE-V2, it is demonstrated that dividing entity recognition into head entity recognition and tail entity recognition, as in the DERP framework, can effectively improve the accuracy of entity recognition, and can produce more accurate results in relation extraction and triple prediction.

6. Conclusions

In this study, a double-headed entities and relations prediction framework for joint triple extraction is proposed. The entity recognition part is decomposed into head entity recognition and tail entity recognition. Specifically, relation prediction and tail entity recognition are executed for the head entities, and in parallel, relation prediction and head entity recognition are performed for the tail entities. In addition, a triple prediction module is designed to solve the entity overlapping problem in previous joint triple extractions. We systematically conducted experiments across four distinct datasets and compared them with ten baseline models. By proceeding with joint triple extraction, a good foundation is constructed for subsequent natural language processing or knowledge graph construction efforts. The results of these rigorous investigations substantiate that the conceptual framework introduced in this paper exhibits certain improvements when juxtaposed with prior models.

In the DERP framework, we have only improved the case of missing triple extraction, and in future work, we will conduct research on the case of error in triple extraction. We will also conduct research on Chinese text triple extraction to study the special characteristics of Chinese text triple extraction and improve the accuracy and effectiveness of Chinese text triple extraction.