SLNER: Chinese Few-Shot Named Entity Recognition with Enhanced Span and Label Semantics

Abstract

Few-shot named entity recognition requires sufficient prior knowledge to transfer valuable knowledge to the target domain with only a few labeled examples. Existing Chinese few-shot named entity recognition methods suffer from inadequate prior knowledge and limitations in feature representation. In this paper, we utilize enhanced Span and Label semantic representations for Chinese few-shot Named Entity Recognition (SLNER) to address the problem. Specifically, SLNER utilizes two encoders. One encoder is used to encode the text and its spans, and we employ the biaffine attention mechanism and self-attention to obtain enhanced span representations. This approach fully leverages the internal composition of entity mentions, leading to more accurate feature representations. The other encoder encodes the full label names to obtain label representations. Label names are broad representations of specific entity categories and share similar semantic meanings with entities. This similarity allows label names to offer valuable prior knowledge in few-shot scenarios. Finally, our model learns to match span representations with label representations. We conducted extensive experiments on three sampling benchmark Chinese datasets and a self-built food safety risk domain dataset. The experimental results show that our model outperforms the F1 scores of 0.20–6.57% of previous state-of-the-art methods in few-shot settings.

1. Introduction

Named entity recognition (NER) aims to identify specific, meaningful entities from text, such as LOCATION and ORGANIZATION, and classify them into predefined categories, as shown in Figure 1. NER is an essential prerequisite task for many natural language processing task, such as information extraction [1], question-answering systems [2], and machine translation [3].

In recent years, neural-network-based techniques have been widely applied in NER [4,5]. However, neural networks are data-driven machine learning methods, and the quantity of training data often limits their performance. Unfortunately, annotated data used for training are often scarce and expensive, especially in specific domains (e.g., the food safety risk domain). Therefore, there has been widespread interest in a challenging yet practical research field: few-shot NER.

One of the challenges of few-shot NER is how to accurately incorporate prior knowledge to effectively classify unseen entity types when confronted with a few examples. Recently, similarity-based methods such as prototype networks have been extensively studied and achieved great success for few-shot learning [6,7,8]. The core idea is to classify input examples from a new domain based on the similarity between their representations and those of each class in the support set. However, this approach experiences a significant drop in performance in a few-shot setting due to the limited representativeness of the data. The prompt-based approach [9,10], by manually or automatically adding prompt words to sentences, guides the model to learn more quickly and accurately while reducing the gap between pretraining and fine tuning. This approach has shown remarkable performance in few-shot learning. However, these methods do not directly leverage the rich prior knowledge contained in label semantics.

In addition, Chinese NER is more challenging than English NER due to the relatively ambiguous entity mentions in Chinese, which limits feature representation and affects the accuracy of NER. Zhang and Yang [11] addressed this issue by using lattice LSTM to represent the entity in sentences and by incorporating the potential lexical information into a character-based LSTM-CRF model. While this character-based representation effectively solves the segmentation error problem, it requires the introduction of a complex external lexicon. Some more recent attempts have switched to span-based feature representations for Chinese NER [12,13], explicitly utilizing span-level information to address token-wise label dependency and better handle nested entities. However, these span-based feature representations only perform simple concatenation of the start and end positions of the span without fully exploiting the internal information of the span, which limits the feature representation of the named entity. For example, when the named entity “伊河谷食品科技有限公司 (Yihegu Food Technology Limited Company, Urumqi, China)” has the internal information “科技有限公司 (Technology Limited Company)”, it becomes easier to classify it as an ORGANIZATION entity.

In response to these challenges, we propose a model called SLNER with enhanced span and label semantic representations to tackle the challenges of Chinese few-shot NER. Specifically, SLNER utilizes two encoders. One encoder is used to encode the text and its spans. This module captures the head and tail information of spans using the biaffine attention mechanism and incorporates self-attention to capture the internal information of spans. Ultimately, these representations are fused to obtain enhanced span representations. This approach fully utilizes the internal composition of entity mentions to achieve more accurate feature representations. In contrast to traditional span-based methods that concatenate the start and end positions of entity mentions, our approach fully exploits the information of each token within entity mentions, providing sufficient and essential clues for entity recognition.

The other encoder is used to encode full label names. Label names are highly generalized specific entity categories and exhibit similar semantics to entities, which can provide additional prior knowledge in few-shot scenarios. Compared to traditional similarity-based methods (e.g., prototype networks), label semantics provide more generic similarity representations, especially in situations where the target domain has a scarcity of samples.

Ultimately, our model learns to match span representations with label representations. We employ a two-stage training strategy using source and target domains, enabling the model to transfer knowledge from the high-resource source domain to the low-resource target domain.

Furthermore, to promote research and applications in low-resource domains, we developed an NER dataset named RISK, which was specifically designed for the food safety risk domain. RISK comprises 5 coarse-grained and 20 fine-grained entity types, each labeled and organized in a hierarchical structure of coarse-grained + fine-grained. We also conducted a performance evaluation of our model on the RISK dataset, and the experimental results demonstrate the challenging nature of the RISK dataset. Constructing an NER dataset in the food safety domain can drive advancements in related research areas such as food traceability and food safety regulation. Additionally, this dataset can serve as a foundation for the development of applications in food safety, including food safety warning systems and food recall management.

We have documented the experimental results on three sampling benchmark Chinese NER datasets and a self-built food safety risk domain dataset. Our contributions can be summarized as follows:

• We propose a simple and effective model named SLNER, which leverages enhanced span representations and label semantics to address the issues of inadequate prior knowledge and limitations in feature representation in Chinese few-shot named entity recognition;
• We created a challenging food safety risk domain dataset, RISK, which is divided into 5 coarse-grained and 20 fine-grained entity categories. This dataset provides data support for the development of named entity recognition applications in the domain of food safety;
• Our proposed model achieved promising performance on the four sampling Chinese NER datasets (including our self-built dataset). Specifically, our model outperformed previous works with F1 scores ranging from 0.20% to 6.57% in different few-shot settings (following the settings of PCBERT) on the Ontonotes, MSRA and Resume datasets. It also achieved promising F1 scores on our self-built RISK dataset.

2. Related Work

2.1. Few-Shot NER

Compared to traditional standard NER [14,15], few-shot NER aims to learn how to recognize named entities with limited data. Currently, research on few-shot named entity recognition based on the traditional standard NER framework primarily focuses on two aspects: incorporating prior knowledge (external information) at the data level and enhancing model generalization at the model level.

In the first aspect, Tong et al. [16] proposed a model, Mining Undefined Classes from Other-class (MUCO), that can automatically induce different undefined classes from the other class to improve few-shot NER. On the other hand, Cui et al. [9], Lee et al. [17], and Chen et al. [18], using a prompt-based approach, treated named entity recognition as a generation task. By manually or automatically adding templates to the training data, the model is trained to predict the [MASK] positions within the templates. This method reduces the gap between pretraining and fine tuning, allowing the model to perform better, even with limited data. Based on multitask instructions, Wang et al. [19] enhanced the knowledge of the training data by adding different auxiliary tasks and corresponding instructions and options; this allowed the model to accurately identify entity information, even with few samples. Additionally, Chen et al. [20] effectively leveraged illustrative instances to precisely transfer knowledge from external resources by describing both entity types and mentions using a universal concept set.

For the second aspect, methods based on meta-learning [21] aim to enable machines to “learning to learn”. The essence of these methods is to train the network model to learn a more robust initialization, allowing it to quickly generalize to new tasks with only a few examples. Methods based on contrastive learning also focus on learning shared features among instances of the same class and distinguishing differences between instances of different classes. Das et al. [22] trained an encoder that produces similar encodings for instances of the same class while ensuring that the encodings of different classes are as dissimilar as possible. This approach reduces the limitations of generalization in the target domain, thereby improving performance in the few-shot setting [23], which augments the distribution of entity labels by assigning k-nearest neighbors retrieved from the training set. This strategy makes the model more capable of handling long-tail cases, along with better few-shot learning abilities.

2.2. Span-Based Method

Previously, the majority of research work treated NER as a sequence-labeling task, where entity classification is performed at the token level. These methods often employ the BIO tagging scheme, where each character’s category is determined to achieve entity recognition for Chinese NER tasks, as shown in Figure 2. As a representative example, Huang et al. [24] utilized BiLSTM as an encoder to learn contextual representations, then employed a conditional random field (CRF) as a decoder to label tokens. Additionally, leveraging the power of pretrained language models such as ELMo [25] and BERT [5] significantly improved the performance of NER.

While the aforementioned token-based NER methods have achieved considerable success, they suffer from two inherent limitations: token-wise label dependency and difficulty handling nested entities. As shown in Figure 2, this method may fail to correctly classify nested entities such as “Taojie greengrocer” and “Hami Melon”. Additionally, if errors occur during the labeling process, incorrect beginning tags (B) or internal tags (I) may affect subsequent labeling, resulting in the erroneous tagging of the entire entity.

Therefore, recent work has increasingly embraced span-based approaches to address NER tasks. This method involves partitioning all possible spans in a sentence into predefined types (e.g., PER or LOC) and determining whether a given text span belongs to a particular category, as shown in Figure 3. Yu et al. [26] adopted a biaffine attention model to assign scores to all potential spans and achieved state-of-the-art performance on both flat and nested English NER datasets. Shen et al. [27] also employed a span-based framework for Chinese NER datasets, effectively addressing the issue of nested entities. Yu et al. [28] and Wang et al. [29] proposed span-level metric learning to bypass the token-wise label dependency problem while still explicitly utilizing phrase representations. Wang et al. [30] introduced a span-based prototype network and a global boundary matrix to learn explicit span boundary information.

In this study, we also utilize a span-based approach for named entity recognition and extensively explore the internal information of entity word spans to obtain enhanced span feature representations.

2.3. Label Semantics

In the task of few-shot text classification, Luo et al. [31] directly appended the label name to the text input in BERT, obtaining feature vectors with more enriched semantic information, thus demonstrating the effectiveness of label semantics in few-shot scenarios. Cui et al. [9] employed a similar approach by reconstructing the input text template, reframing NER as a cloze task, and using a sequence-to-sequence model to fill the entity label names in predefined templates, achieving few-shot named entity recognition through a prompt-based method. Ma et al. [32] abandoned the template construction process while retaining the word prediction paradigm of pretrained models to predict class-related pivot words (or label words) at entity positions. Inspired by prompt-tuning methods, Zhong et al. and Ye et al. [33,34] initialized markers in the NER task not with random initialization but with meaningful words (such as label names), resulting in a certain degree of improvement in model performance. Our proposed model leverages the connection between label semantics and entity spans (Figure 4) to learn aligning span representations with label representations, exhibiting promising results.

3. Method

This chapter first formalizes the problem of few-shot named entity recognition (Section 3.1). Then, we propose our model, SLNER, for Chinese few-shot NER (Section 3.2). The model consists of two encoders: one for encoding the text and its span to obtain better feature representations (Section 3.3.1) and another for encoding the full label name to capture additional prior knowledge (Section 3.3.2). Additionally, we adopt a two-stage training strategy using source and target domains (Section 3.4). The details are outlined as follows.

3.1. Few-Shot NER Task Formalization

For the few-shot NER task, assume that we have a resource-rich source domain NER dataset, ${\mathrm{Ɗ}}^R=\left\{\left[T_1,L_1\right],\cdots ,\left[T_n,L_n\right]\right\}$, where $T_i=\left\{t_1,t_l\right\}$ represents the $i$-th text ($i\in \left[1,n\right]$), $t_i$ represents the $i$-th token ($i\in \left[1,l\right]$), and $L_i$ represents the labels corresponding to the entity spans in the $i$-th text ($i\in \left[1,n\right]$). We use ${ℂ}^R$ to denote the label set of the source domain dataset. Then, given a resource-scarce target domain dataset, ${\mathrm{Ɗ}}^T=\left\{\left[T_1,L_1\right],\cdots ,\left[T_m,L_m\right]\right\}$, the number of texts in the target domain dataset is limited (i.e., $m\ll n$), and the label types in the target domain may differ from those in the source domain (i.e., ${ℂ}^R\ne {ℂ}^T$). We aim to leverage the knowledge from the source domain dataset to improve the model’s performance on the target domain dataset.

3.2. Overall Structure

The overall architecture of the SLNER model is illustrated in Figure 5. For span representation, given a sentence ($T=\left\{t_1,\cdots ,t_l\right\}$) of length $l$, we use BERT as our encoder, which encodes the context of the $i$-th token in the sentence as follows:

$$h_i=BERT\left(t_i\right) \forall t_i,t_i\in T$$

(1)

where $d_h$ is the hidden dimension of the encoder, and the output dimension after passing the original sentence through the encoder is ${ℝ}^{l\times d_h}$.

To further enhance the modelling of the sequential order of the text, the embedding representation obtained from BERT is then passed through a bidirectional LSTM layer. The forward LSTM network captures the hidden forward states (historical features), while the backward LSTM network captures the hidden backward states (future features), resulting in a context-aware encoding representation:

$$x_i=\left[ \overrightarrow{LSTM}\left(h_i,x_{i-1}\right) ; \overleftarrow{LSTM}\left(h_i,x_{i+1}\right)\right]$$

(2)

At this stage, the output dimension of the original sentence through the bidirectional LSTM layer is ${ℝ}^{l\times d_x}$.

We predefine an n-gram value ($w$), which represents the maximum length of spans that can be formed in a text. The number of possible spans that can be formed in a sentence of length $l$ is given by:

$$N_s=\left\{ \begin{array}{c}w\left[l-\left(w-1\right)\right]+{\displaystyle {\displaystyle \sum }_{i=1}^{w-1}}i l\ge w\\ {\displaystyle {\displaystyle \sum }_{i=1}^{w-1}}i l<w\end{array}\right.$$

(3)

Next, the token-embedding sequence obtained from the LSTM layer is used to construct the span-level feature vector representation ($h_{span}$) through a span extractor (details in Section 3.3.1). The dimension of $h_{span}$ is finally expanded to ${ℝ}^{N_s\times d_h}$ through a feed-forward layer.

For label representation (details in Section 3.3.2), we manually define the appropriate full label name for each label. Similarly, we use BERT as the encoder and directly encode the label using Equation $\left(1\right)$ to obtain the global semantic feature ($h_{\left[CLS\right]}$). The difference from span encoding is that we further pass $h_{\left[CLS\right]}$ through a pooler layer to obtain the semantic feature vector representation ($h_{label}$), which serves as the final representation of the label:

$$h_{label}=Tanh\left(W_{\left(p\right)}\times h_{\left[CLS\right]}+b_{\left(p\right)}\right)$$

(4)

where $W_{\left(p\right)}$ represents the weight parameters in the pooler layer, $b_{\left(p\right)}$ represents the bias parameters, and $Tanh$ is the activation function. The dimension of $h_{label}$ needs to be expanded to ${ℝ}^{d_h\times N_c}$, where $N_c$ is the number of entity categories.

According to our approach, there is a correlation between labels and spans appearing in the text. Therefore, we capture this correlation through the dot product:

$$\mathrm{H}=h_{LST}\times h_{span}$$

(5)

where the similarity matrix ($\mathrm{H}$) has dimensions of ${ℝ}^{N_s\times N_c}$. We use a standard linear classifier with a softmax function to predict the entity type for each span, resulting in the final predicted output (${\widehat{y}}_{span}$):

$${\widehat{y}}_{span}=Softmax\left(W_{\left(c\right)}\times h_{span}+b_{\left(c\right)}\right)$$

(6)

where $W_{\left(c\right)}$ is the trainable parameter of the classifier, and $b_{\left(c\right)}$ is the bias. Finally, we use the cross-entropy loss function to compute the loss, which measures the difference between the predicted results and the ground truth labels:

$${\mathcal{L}}_{PSNNER}=-\sum y_{span}\log \left({\widehat{y}}_{span}\right)$$

(7)

3.3. Specific Structure

3.3.1. Enhanced Span Representation

In previous span-based NER models [35], it was common practice to concatenate the embedding information of the start-position token and the end-position token of the entity (referred to as the “outer span”) to represent the span of that entity, which is then used for the final classification decision:

$$h_{span}=\left[x_{start};x_{end}\right]$$

(8)

This approach lacks interaction between the start and end tokens and fails to fully utilize the informative content within the span. Moreover, this span representation is coarse-grained. To address these limitations, Yu et al. [26] proposed a biaffine decoder that utilizes two fully connected layers to enable interaction between the start and end tokens while simultaneously predicting the span type. However, in this biaffine method, the information within the span is still ignored.

To fully utilize the informative content within the span, we employ enhanced span to generate the final span representation (as shown in Figure 6). Specifically, we pass the token-embedding information through the outer and inner span modules. The outer span module, similar to the biaffine decoder method, utilizes the biaffine attention mechanism to obtain the outer span representation:

$$h_{outer span}=h_{start}^{T}U{h}_{end}+W\left(h_{start}\oplus h_{end}\right)+b$$

(9)

where $h_{start}$ and $h_{end}$ represent the start and end token embedding of spans in a text, respectively; $U$ and $W$ are learnable parameters; and $b$ is the bias. The dimension of $h_{outer span}$ is expanded to ${ℝ}^{N_s\times d_x}$.

We designed the inner span module to capture the token-level information within the span. For this purpose, we use linear attention to generate information interaction for each token. Specifically, this module uses span and the start- and end-position indices of the span as input. It starts by applying a feed-forward neural network (FFNN) to non-linearly transform the input representations, obtaining context-aware representations. Then, it computes normalized scores for each position. Finally, these representations are fed into a self-attention layer, which combines the representations of each position with those of other positions, weighted by the attention scores. This allows the model to capture potential relationships between the tokens within the span. The result is the inner span representation:

$$a_i=W_{\left(f\right)}\times x_i+b_{\left(f\right)}$$

(10)

$$s_i=\frac{\mathit{exp}\left(a_i\right)}{{{\displaystyle \sum }}_{k=n}^m\mathit{exp}\left(a_k\right)}$$

(11)

$$h_{inner span}={\displaystyle \sum }_{i=n}^m{s}_i\times x_i$$

(12)

where $x_i$ represents the hidden representation from the bidirectional LSTM, and $W_{\left(f\right)}$ and $b_{\left(f\right)}$ are the learnable weights and biases of the feed-forward neural network, respectively. The indices $i\in \left\{n,n+1,\cdots ,m\right\}$ correspond to the token indices within the span, where $n$ and $m$ represent the start and end indices of the span, respectively. When $n=m$ (indicating a span of length 1), we do not extract additional features and simply use the hidden representation ($x_i$).

To predict the entity type, we integrate the outer span representation and inner span representation in a gate network to obtain the weight coefficient ($\mathsf{\gamma }$) (as shown in Figure 7):

$$\mathsf{\gamma }=\mathsf{\sigma }\left(U_{\left(g\right)}\left[h_{outer span};h_{inner span}\right]+b_{\left(g\right)}\right)$$

(13)

where $U_{\left(g\right)}$ and $b_{\left(g\right)}$ are trainable parameters of the gate network, and σ represents the sigmoid function. The dimension of $\mathsf{\gamma }$ is ${ℝ}^{N_s\times 1}$.

The final enhanced span representation is obtained by weighting the inner span representation and outer span representation using $\mathsf{\gamma }$:

$$h_{span}=\mathsf{\gamma }\odot h_{outer span}+\left(1-\mathsf{\gamma }\right)\odot h_{inner span}$$

(14)

where $\odot$ represents element-wise multiplication, and the resulting $h_{span}$ has dimensions of ${ℝ}^{N_s\times d_x}$.

3.3.2. Label Representation

We believe that label semantics can provide additional prior knowledge. Label semantics carry the semantic information of entities in the same category, as this information is manually summarized and induced from a large amount of data. Therefore, when data are limited, especially in small-sample scenarios, we can introduce label semantics to allow our model to make generalizations from the available data. Furthermore, full label names themselves are mentions that appear in various contexts within the text. Their frequencies are synchronized to some extent with the corresponding entity words of their respective categories. Thus, there exists a semantic correlation between label names and the span tokens appearing in the text, and this correlation can be leveraged and utilized.

Considering that our label encoder is based on BERT and incorporates prior knowledge from pretraining, our label representation module allows any form of text to be used as input. This design not only enables easy and rapid expansion to unseen label sets in low-resource domains but also prevents the model from forgetting prior knowledge. We experimented with different label forms and analyzed their effects (Section 5.1).

Table 1. The full label name format of the four datasets we used.

Dataset	Entity Label	Full Label Name
Ontonotes	PER	人名 (person name)
	ORG	组织 (organization)
	LOC	位置 (location)
	GPE	地名 (geographic name)
MSRA	NS	位置 (location)
	NR	姓名、名字 (person name)
	NT	组织 (organization)
Resume	CONT	国籍 (nationality)
	EDU	教育背景、学历 (educational background)
	LOC	位置、地名 (location)
	NAME	姓名、名字 (person name)
	ORG	组织 (organization)
	PRO	专业 (profession)
	RACE	民族 (race)
	TITLE	职称、职业 (title and occupation)
RISK	LOC-PROV	省 (province)
	LOC-PREF	市、区 (city, district)
	LOC-COUNT	县 (county)
	FOOD-VEG	蔬菜 (vegetable)
	FOOD-FRUIT	水果 (fruit)
	FOOD-MEAT	肉 (meat)
	FOOD-GRAIN	粮食 (foodstuff)
	FOOD-DAIRY	奶制品、饮品 (dairy products and beverages)
	CO-PROC	加工生产方式 (processing and production methods)
	CO-PROD	公司、厂 (company, factory)
	CO-TRAD	售卖商店 (sales store)
	CO-CATE	饭店 (hotel)
	CO-MATE	超市 (supermarket)
	ORG-REGU	监督局 (supervisory authority)
	ORG-ADMI	管理部门 (management department)
	ORG-LEGA	法律机构 (legal agency)
	ORG-SOCI	社会组织 (social organization)
	RISK-HIGH	病毒 (virus)
	RISK-MID	化学物质、细菌 (chemicals, bacteria)
	RISK-LOW	添加剂 (additive)

presents the final forms of full label names used in this study (for the non-entity type in each dataset, we uniformly use “其他 (other)” as the label name).

3.4. Training Strategy

Compared to the previous work of traditional NER neural architecture, our model does not require a new randomly initialized top-layer classifier for new datasets with new unseen label names. Therefore, our model allows domain transfer for different label categories, which is very beneficial for few-shot learning. On this basis, we adopt a two-stage training program. In the first stage, we pretune our model on the source dataset to obtain a prior knowledge-rich source domain model. In the second stage, we fine tune the source model from the previous stage as the initial model on the target domain dataset. During model training, two encoders are updated at each iteration of the two stages, which helps align the span-embedding space with the label-embedding space.

4. Experiments

4.1. Datasets

• Target Domain Datasets

To validate the effectiveness of our model, we used three benchmark Chinese datasets and a self-built dataset as target domain datasets:

Ontonotes 4.0 [36]: This dataset comprises corpora from the news and broadcast domains, covering four entity types.

MSRA [37]: This dataset comprises corpora from the news domain and includes three entity types.

Resume [11]: This dataset consists of abstracts from resumes of senior managers in publicly listed companies. It encompasses eight entity types.

RISK: This dataset was created by us and focuses on food safety risk-related corpora. We hierarchically categorized the dataset into 5 coarse-grained and 20 fine-grained entity types, as shown in Figure 8. The entity count for each fine-grained category is illustrated in Figure 9. The distribution of entity categories is not uniform, making RISK challenging at the fine-grained level.

The detailed statistics of the target domain datasets are presented in

Table 2. Detailed statistical table for the target domain dataset.

Dataset	Train	Dev	Test	Entity Types
Ontonotes	15.7 k	4.3 k	4.3 k	4
MSRA	41.7 k	4.6 k	4.3 k	3
Resume	3.8 k	0.46 k	0.48 k	8
RISK	1.2 k	0.26 k	0.26 k	5 (20) 1

¹ 5 (20) representing 5 coarse-grained and 20 fine-grained entity types of the RISK dataset.

. To ensure the comparability of experimental results, we followed the sampling approach of PCBERT [10] to simulate a few-shot scenario (see Section 4.4 for details). Compared to the N-way K-shot setting, we believe that this sampling approach is more representative of realistic few-shot scenarios. We further divided the datasets into scenarios with even fewer examples and tested our model, demonstrating its effectiveness even in scenarios with fewer data.

• Source Domain Dataset

We adopted the source domain dataset used in the PCBERT to ensure experimental consistency and reliability. This high-resource dataset is a combination of multiple datasets, including CLUENE [38], CNERTA [39], RenMinRiBao [40], and datasets from unknown sources.

4.2. Implementation Details

We adopted a two-stage training approach, starting with 5 epochs of pretraining on the source domain dataset, followed by 100 epochs of fine tuning using the pretrained model as the initial model in the target domain. For the upstream (BERT) part, we used the standard BERT-wwm-ext pretrained language model with 768-dimensional hidden representations for character embeddings and the learning rate set to 1 × 10⁻⁵. For the downstream model, the learning rate was set to 1 × 10⁻⁴. The hidden state size of the BiLSTM encoder was set to 200. We used the Adam optimizer [41]. We performed three different random samplings for all datasets to reduce randomness and reported the average performance as the final experimental result. All experiments were conducted on a computer with an Intel Core i9 13900K/F CPU^®5.8 GHz and a GeForce RTX 3090 GPU with 24 GB of memory. The models were implemented using the PyTorch 1.12.0 framework.

4.3. Baseline

BERT [5] is the fundamental BERT-based NER method that adds a token classifier layer to the downstream of BERT.

BERT-LC is an effective baseline method for handling Chinese NER. It extends the regular BERT by adding a BiLSTM layer and employs a CRF layer as a decoder to predict token types.

Lattice LSTM [11] is a character-based Chinese NER method that introduces lexicon information by using a lattice-structured LSTM model.

FLAT [42] is a lattice-structured NER method based on a transformer. It constructs a flat-structured transformer to fully utilize lattice information and leverage parallel computing on GPUs.

LEBERT [43] addresses Chinese sequence-labeling tasks using lexicon-enhanced BERT. It incorporates lexical information into the encoding process of BERT’s underlying layers using lexicon adapter layers.

LEBERT-LC is an extension of LEBERT that further adds a BiLSTM layer after the BERT output layer to facilitate a comparison with our proposed SLNER model.

PCBERT [10] is a prompt-based Chinese few-shot NER model. It consists of the P-BERT component and the C-BERT component, integrating lexical features and implicit label features.

4.4. Experimental Results

Our model was thoroughly evaluated on four sampled Chinese NER datasets. Specifically, we sampled K = 250, K = 500, K = 1000, and K = 1350 examples from the Ontonotes, MSRA, and Resume datasets, respectively, as training sets to cover different levels of data scarcity and comprehensively evaluate the training effectiveness and robustness of the model. The standard F1 score is used as the evaluation metric to measure the final performance of the model.

Table 3. F1 scores for the four Chinese sampling NER datasets. The best results are highlighted in bold.

Dataset	Method	K = 250	K = 500	K = 1000	K = 1350
Ontonotes	BERT	63.85	69.50	71.33	72.42
	BERT-LC	65.69	73.54	74.97	77.19
	Lattice LSTM	39.71	45.46	54.54	57.48
	FLAT	49.01	46.35	49.34	57.44
	LEBERT	69.48	69.01	73.78	74.84
	LEBERT-LC	70.26	69.89	73.83	76.01
	PCBERT	74.42	75.62	78.33	81.52
	SLNER (ours)	77.66	80.46	81.53	82.23
MSRA	BERT	68.44	72.28	81.21	82.28
	BERT-LC	79.01	83.13	87.84	89.32
	Lattice LSTM	54.69	63.61	74.27	76.31
	FLAT	59.62	70.20	80.79	64.95
	LEBERT	79.11	85.18	87.77	89.35
	LEBERT-LC	80.92	86.09	88.11	88.70
	PCBERT	81.08	85.25	87.88	89.72
	SLNER (ours)	87.65	89.30	89.67	90.08
Resume	BERT	53.80	62.64	69.36	70.65
	BERT-LC	92.26	94.66	95.16	96.41
	Lattice LSTM	85.63	89.60	92.01	93.13
	FLAT	84.62	90.77	92.97	87.79
	LEBERT	89.15	92.56	94.02	95.19
	LEBERT-LC	91.60	93.03	95.40	95.16
	PCBERT	93.42	94.01	94.96	95.97
	SLNER (ours)	94.02	94.86	95.99	96.36

shows the experimental results of our proposed model and the baseline models on three benchmark NER datasets. The results indicate that, except for the 1350-shot setting in the resume dataset, our method outperforms the state-of-the-art models by 0.2–6.57 percent, demonstrating the excellent performance of our method in few-shot settings.

Table 4. The results on the RISK dataset, showing the F1 scores for different sampling sizes in both coarse-grained and fine-grained categories.

Dataset	K-Shot	Coarse-Grained	Fine-Grained
RISK	K = 250	69.67	63.97
	K = 500	71.73	65.62
	K = 1000	72.70	68.06
	Full 1	72.62	68.98

¹ Full represents K = 1218 (the maximum size of RISK).

presents the experimental results of our proposed model on the RISK dataset. Compared to coarse-grained classification, the model shows a maximum decrease of 3.64–6.11 percent in F1 score under fine-grained classification. This indicates that the dataset is more challenging in fine-grained classification.

In addition, to further validate the feasibility of our model in a low-resource setting, we also sampled data under K = 10, K = 20, and K = 50 settings and evaluated the effectiveness of our model with even fewer examples. The results are shown in

Table 5. Results with fewer samples.

K-Shot	Dataset
	Ontonotes	MSRA	Resume	RISK
K = 10	65.90	73.18	77.23	59.50 (44.61) 1
K = 20	70.92	81.19	81.70	67.57 (45.02)
K = 50	74.91	86.34	90.55	68.87 (57.31)

¹ Parentheses represent the fine-grained results of the RISK dataset.

. The experimental results demonstrate that our model can still achieve good results, even with as few as a few dozen samples. We analyze this phenomenon in Section 5.3.

5. Analysis

5.1. Ablation Study

To validate the impact of enhanced span representation and label representation on SLNER, we conducted extensive ablation experiments on the Ontonotes, MSRA, Resume, and RISK datasets under different sampling settings (K = 250, 500, 1000, and 1350) as shown in

Table 6. The results of the ablation experiments. ESR, enhanced span representation; LR, label representation; SD, source domain dataset.

Dataset	Method	K = 250	K = 500	K = 1000	K = 1350
Ontonotes	SLNER	77.66	80.46	81.53	82.23
	-ESR w/SD	75.63	80.02	81.04	81.91
	-LR w/o SD	72.90	75.62	77.98	78.09
MSRA	SLNER	87.65	89.30	89.67	90.08
	-ESR w/SD	87.99	89.60	89.42	89.80
	-LR w/o SD	80.72	84.06	84.87	84.54
Resume	SLNER	94.02	94.86	95.99	96.36
	-ESR w/SD	93.52	94.46	95.38	95.95
	-LR w/o SD	87.64	94.17	95.48	95.35
RISK	SLNER	63.97	65.62	68.06	N/A 1
	-ESR w/SD	63.27	65.26	67.29	N/A
	-LR w/o SD	55.67	58.94	61.23	N/A

¹ Since the maximum size of the RISK dataset is less than 1350, we did not conduct experiments with K = 1350 for this dataset.

. Specifically, we first removed the enhanced span representation module and observed a decrease in performance across all datasets with different sampling sizes. This demonstrates the effectiveness of the module, which we analyze in Section 5.2. Next, we removed the label representation module. It is worth mentioning that, due to the removal of the label representation module, we had to introduce a top-layer classifier, which resulted in the inability to use the source-domain-initialized model. As a result, the performance of our model significantly dropped, confirming our hypothesis that the label semantics carry rich prior knowledge. This also indicates the importance of prior knowledge from the pre-fine-tuning stage in low-resource environments. We provide a detailed analysis of this in Section 5.3.

To further analyze the impact of label representation, we experimented with different definitions: full label name (the one used in our model), misleading label name, and indistinguishable label name. Misleading label name refers to randomly shuffling the full label name, for example, changing the label name corresponding to “PER” from “人名 (person name)” to “组织 (organization)”. Indistinguishable label name refers to unifying all different label names into a single label name, such as using “人名 (person name)” for all labels. We conducted a comparative experiment on the Ontonotes dataset, as shown in Figure 10. The results indicate that a misleading label name slightly negatively affects the model, but an indistinguishable label name has a significant negative impact. This suggests that the selection of label names must be category-related and that a label name carries entity-specific prior knowledge.

5.2. The Impact of Enhanced Span Representation

From Table 6, it can be observed that removing the inner span module results in a decrease in F1 scores. Particularly, when there are fewer samples (K = 250), the impact of removing the inner span module is more significant. This is because in the two-stage training process, during the source domain training, the lack of accurate span representations leads to a decrease in NER accuracy. However, when transitioning to the target domain, this inaccuracy is amplified, and the effect is more pronounced with fewer samples. Furthermore, during the ablation experiments, it was found that removing the inner span module leads to slower convergence of the model, indicating that without intra-span information, entity span localization becomes more challenging.

5.3. The Impact of Label Representation

The experimental results in Table 6 indicate that the impact of label name becomes more pronounced as the number of samples decreases (compared to K = 1350, the model’s performance drops more significantly when label name is removed at K = 250). This is because as the number of training samples decreases, the number of words representing the same type of entity also decreases, resulting in less specific entity descriptions. On the other hand, label name itself is a broad concept that can represent entity categories, and its effectiveness increases when the number of samples is reduced. This leads to improved generalization ability.

6. Conclusions

In this study, we propose the SLNER model with enhanced span and label semantics to address the issues of inadequate prior knowledge and feature representation limitations in Chinese few-shot named entity recognition (NER). To tackle the feature representation limitations, we employ the biaffine attention mechanism and self-attention to obtain enhanced span representations, fully leveraging the internal composition of entity mentions. For the problem of inadequate prior knowledge, we introduce label semantics, which provide a highly abstract representation of specific entity categories with similar semantics and provide additional prior knowledge in few-shot scenarios. Our model learns to match span representations and label semantics to achieve entity recognition. Additionally, we constructed the RISK dataset in the food safety risk domain, which consists of 5 fine-grained and 20 coarse-grained entity types, providing a data foundation for NER development in low-resource domains. We extensively validated our proposed model on three benchmark datasets and our self-built dataset, and the results demonstrate the effectiveness of our model in addressing the issues of Chinese few-shot NER.