Multi-Label Classification of Complaint Texts: Civil Aviation Service Quality Case Study

Abstract

Customer complaints play an important role in the adjustment of business operations and improvement of services, particularly in the aviation industry. However, extracting adequate textual features to perform a multi-label classification of complaints remains a difficult problem. Current multi-label classification methods applied to complaint texts have not been able to fully utilize complaint information, and little research has been performed on complaint classification in the aviation industry. Therefore, to solve the problems of insufficient text feature extraction and the insufficient learning of inter-feature relationships, we constructed a multi-label classification model (MAG, or multi-feature attention gradient boosting decision tree classifier) for civil aviation service quality complaint texts. This model incorporates multiple features and attention mechanisms to improve the classification accuracy. First, the BERT (Bidirectional Encoder Representations from Transformers) model and attention mechanisms are used to represent the semantic and label features of the text. Then, the Text-CNN (a convolutional neural network) and BiLSTM (bidirectional long short-term memory) multi-channel feature extraction networks are used to extract the local and global features of the complaint text, respectively. Subsequently, a co-attention mechanism is used to learn the relationship between the local and global features. Finally, the travelers’ complaint texts are accurately classified by integrating the base classifiers. The results show that our proposed model improves the multi-label classification accuracy, outperforming other modern algorithms. We demonstrate how the label feature representation based on association rules and the multi-channel feature extraction network can enrich textual information and more fully extract features. Overall, the co-attention mechanism can effectively learn the relationships between text features, thereby improving the classification accuracy of the model and enabling better identification of travelers’ complaints. This study not only effectively extracted text features by integrating multiple features and attention mechanisms, but also constructed a targeted feature word set for complaint texts based on the domain-specific characteristics of the civil aviation industry. Furthermore, by iterating the basic classifier using a multi-label classification model, a classifier with higher accuracy was successfully obtained, providing strong technical support and new practical paths for improving the civil aviation service quality and complaint management.

1. Introduction

Complaint forms are an important feedback channel for air travel customers. Complaint information often exposes problems in the operations of enterprises and can play an important role in guiding changes in business operations and improving services based on customers’ needs [1]. Therefore, quickly and accurately categorizing customer complaints, particularly by converting unstructured data into structured forms, is crucial for the handling of complaints and the exploration of customer needs [2]. It helps the complained party to have a more intuitive understanding of the overall situation of the complaint, so as to systematically handle similar issues. A complaint text can consist of multiple complaint topics, meaning each instance corresponds to more than one kind of label, and the number of labels is uncertain. Multi-label classification has recently become a popular research topic. However, current multi-label classification methods for complaint texts are not able to fully describe travelers’ complaint information, and extracting robust textual features remains challenging.

With the development of machine learning and deep learning techniques, increasing numbers of researchers have applied such methods to complaint text classification. Traditional machine learning methods such as support vector machines (SVMs), K-nearest neighbors (KNNs), and plain Bayesian methods have been introduced to solve text classification problems. With regard to deep learning models, recurrent neural networks (RNNs), convolutional neural networks (CNNs), and their variants are widely used. Recently, pre-trained language models (PLMs) such as BERT (Bidirectional Encoder Representations from Transformers) have achieved state-of-the-art results on a number of natural language processing (NLP) tasks and have seen widespread use. However, these approaches are based on generalized data and often ignore the importance of domain-specific features in classifying complaint texts. Moreover, when applied directly to data from civil aviation, these models often achieve lower performance compared to generalized domains.

In civil aviation, travelers’ complaints are often directed at different service categories, such as ticketing, flight irregularities, baggage, and air service. Complaints falling under different categories reflect the service quality from different perspectives and to different degrees. To address gaps in the research on multi-label classification for improving the civil aviation service quality and improve upon the performance of generalized methods, we propose a textual multi-label classification model for complaints that incorporates multiple features and attention mechanisms. Specifically, the method accounts for service quality feature words, accurately extracts the textual features of the complaints, and successfully learns the relationships between features, thus improving the multi-label classification model’s performance.

The main contributions of this study can be summarized as follows: (1) The proposed model can extract textual features more fully by integrating multiple features and attention mechanisms; this remedies the common problem of insufficient textual feature extraction. (2) The domain-specific characteristics of the civil aviation industry were accounted for by constructing a feature word set of complaint text labels, improving the effectiveness of the multi-label classification model. (3) A classifier with higher accuracy was obtained by iterating the base classifier using GBDTs (gradient boosting decision trees), enabling the automatic selection of important features and further improving the performance of the multi-label classification model. Experiments showed that our constructed model performs well at its task of the multi-label classification of travelers’ complaint texts. Compared with the traditional BR-GBDT model, the F1 value increased by 3.08 percentage points, which is also better than other modern algorithms.

We begin by introducing related research and theoretical methods of text classification and multi-label classification, analyzing their respective advantages and disadvantages. Next, the multi-label classification model involving multiple features and attention mechanisms is introduced in detail. Third, we describe an experimental evaluation we conducted of the proposed model and analyze its performance. Finally, applications of the model are discussed and improvements upon previous work are summarized; moreover, avenues for future research are proposed.

2. Related Work

With the rapid development of the internet and social media, the volume of customer complaint text data has increased dramatically. Extracting useful information from these complaint texts has become an important research direction in the field of natural language processing. In particular, multi-label classification tasks are crucial in customer complaint text classification, as a single complaint may involve multiple issues or topics. This section reviews recent research progress in this field, with a particular focus on classification studies for complaint texts and the latest developments in multi-label classification methods.

2.1. Classification of Complaint Texts

Text classification refers to the automatic categorization of text information according to set labels using natural language processing techniques. Text classification methods are mainly divided into two categories: machine learning and deep learning methods. The most commonly used machine learning algorithms for customer complaint text classification are support vector machines (SVMs) [3,4,5], K-nearest neighbors (KNNs) [6,7,8], Naive Bayes [9,10], and others. However, many of these algorithms suffer from the problems of high dimensionality and data sparsity when performing text classification, and their performance is generally lacking. In recent years, with the rise of deep learning, more researchers have used neural networks for text classification. This shift towards deep learning has improved the performance of text classification models, particularly when handling complex or large datasets.

Most of these algorithms focus on a certain subtask to improve the performance of the overall model. For instance, some algorithms focus on more comprehensive representations of the text. Ref. [11] proposed the Vector Space Model (VSM), which treats texts as a collection of words and considers elements such as the TF (term frequency) or TF-IDF (term frequency–inverse document frequency); this is the most widely used type of feature representation for text classification tasks. For example, Zhu [12] used the VSM (Viable System Model) to calculate the information gain of mutual information and determine the characteristics of news text data. However, this type of model does not consider the positional relationship between words. To address the high dimensionality of text feature vectors and data sparsity, Blei et al. [13] proposed the Latent Dirichlet Allocation (LDA) topic model, which maps the high-dimensional word item space to a low-dimensional implicit semantic space, effectively reducing the dimensionality of text vectors. This method has been widely used in the field of text classification. For example, Pavlinek et al. [14] presented the ST LDA method for text classification. Although the probabilistic topic model considers the correlations between words, it cannot utilize the contextual information constituted by neighboring words. Therefore, Word2Vec, as proposed by Mikolov et al. [15] at Google, became one of the most widely used word embedding models. The Word2Vec word vector model can solve high-dimensional sparseness and can also introduce semantic features in combination with the context [16]. However, Word2Vec is a relatively shallow method because it only places pre-trained information in the first layer of the model. Compared with the above traditional text representation models, the text vector representation model BERT reached a leading level in tasks of text classification and information extraction. It can effectively alleviate the issues of the multiple meanings of words, repetition of first words, and long-term dependency.

Some studies have focused on optimizing feature extraction methods. Deep learning algorithms commonly used for text classification are recurrent neural networks (RNNs), convolutional neural networks (CNNs), and their variants. Hu et al. [17] added long short-term memory (LSTM) and an attention model to an independently recurrent neural network (IndRNN) to avoid suffering the exploding problem and gradient vanishing of training in RNNs. Chen et al. [18] used a CNN for encoding and an RNN for decoding (i.e., an encoder–decoder framework) to capture global and local text semantics and improve the performance of a multi-label classification model. Text-CNNs employ different sizes of convolutional kernels to extract local features shared between words within the range of the kernels, but cannot reveal long-distance dependencies due to the limitations of the size of the kernels [19]. RNNs and their variants, long short-term memory (LSTM) and bidirectional long short-term memory (BiLSTM) models, are a class of neural networks designed to model sequences. They are inherently suitable for modeling text and can capture its global structural information, but can also face the issues of gradient dispersion and gradient explosion [20]. Thus, combining these approaches is expected to lead to more robust and adaptable text classification systems.

An attention mechanism has been applied in many natural language processing tasks. Unlike maximum pooling, which only selects the most important information, attention mechanisms learn the proportion of the contribution of each part of the text to the overall semantic information. Important words or phrases are assigned higher weights so that the key patterns are identified; however, word order information is ignored, resulting in an inability to use the text’s global structural information.

It is clear that individual classification algorithms have their respective limitations, and combining different deep learning networks to obtain better text classification results is a promising research topic. Deep learning fusion methods can be divided into two categories according to their network structure: single-channel and multi-channel. Single-channel networks generally process text vectors sequentially, and the feature extraction process is usually expressed as a serial structure. Ma et al. [21] combined the advantages of BERT and multi-scale CNNs to construct a multi-label classification model, which first captures the local features of the text and then combines utterance features of different scales in the convolutional layers to obtain richer semantic feature information, thus improving the performance in multi-label classification. Gu et al. [22] combined the basic principles and network structure of a CNN in deep learning to construct a community security risk prediction model based on the improved single-channel CNN and decision tree, which extended the single channel to a multi-channel network, greatly expanding the feeling field of the CNN and improving the accuracy of the prediction model. Rhanoui et al. [23] proposed a CNN-BiLSTM model with an attention mechanism, which can extract both the local semantic and multi-scale contextual features of words in sentences, compensating for the shortcomings of CNNs in global feature extraction. Kavianpour et al. [24] employed a deep learning model based on convolutional neural networks (CNNs), bidirectional long short-term memory (BiLSTM), and an attention mechanism, as well as a zero-order hold (ZOH) preprocessing methodology to improve the performance of prediction results. The single-channel network structure means that the feature extraction process is not independent, and the structure of sequential processing can lead to feature extraction issues at the ends of texts.

Multi-channel feature extraction networks can use different methods to extract features independently from the input sequence, which helps alleviate the problem of short and sparse features in complaint texts. Xu et al. [25] used a Text-CNN to capture local features and improved the effect of multi-label text classification by using a dual-channel aggregation of historical text and labels through Bi-LSTM. Traditional multi-channel customer complaint text classification methods directly use concatenation or the dot product for the fusion of feature vectors output from different extraction networks; this does not account for inter-feature relationships and interactions, which cause these relationships not to be learned. As a resource allocation method, an attention mechanism can assign different weights to different parts of the encoder input, thus solving the problem of the lack of differentiation of input features [26]. Yan et al. [27] proposed R-Transformer-BiLSTM, a model that combines R-Transformer, BiLSTM+CRF, and self-attention to enhance the multi-label text classification efficiency and accuracy. Zhao et al. [28] proposed a multi-label text classification model that uses keyword-based label representation and self- and interactive attention mechanisms to extract features, resulting in improved performance. Wang et al. [29] proposed SemFA, a large-scale multi-label text classification model based on semantic features and associative attention, which establishes potential associations between labeled features and text features through an associative attention mechanism. By synthesizing global information and local semantic features at different granularities, the model outperforms most existing large-scale multi-label classification algorithms.

A co-attention mechanism involves feeding multiple feature vectors into a network at the same time and jointly learning their respective attention weights. Thus, it is often used for the mutual extraction of key information shared between image and text features in visual question and answer tasks [30]. Hu et al. [31] designed an image–text matching-aware co-attention mechanism which captures the alignment of images and text for better multimodal fusion to improve fake news detection. The multi-head attention mechanism is a combination of attention mechanisms proposed by Vaswani et al. [32], which maps the input features to different subspaces for representation. In this way, the attention part can jointly focus on the information from different representation subspaces, which improves the learning effect. Khan et al. [33] proposed a novel MSER model that uses a multi-headed cross-attention mechanism for the deep feature fusion of audio and text cues, achieving SOTA results with a 4.5% improved recognition rate on the IEMOCAP and MELD datasets. Wang et al. [2] used dual-channel word embedding to enrich text representation and inputted features extracted by a Text-CNN into a corresponding BiLSTM network to extract the inflectional order information. They then fused the extraction results of each channel by using the multi-head collaborative attention mechanism, thus improving the text classification results.

2.2. Multi-Label Text Classification

Many methods have been proposed for multi-label text classification, with traditional methods belonging to two main categories: Problem Transformation (PT) and Algorithm Adaptation (AA). The first type is the most classical method, whose strategy is to convert a multi-label classification problem into a series of single-label classification problems, so that existing single-label learning algorithms can be applied to solve the problem. Representative PT methods include binary relevance (BR) [34,35,36], label powerset (LP) decomposition [37,38], and classifier chains (CCs) [39,40,41]. The BR method decomposes the multi-label classification problem into multiple binary classification problems for processing; the LP method transforms the multi-label classification problem into a multi-classification problem by viewing label combinations as classification categories; and the CC method transforms the multi-label classification task into a chain of binary classification problems, and the subsequent chain of binary classifiers makes predictions based on the previous ones.

The strategy of the second class, algorithmic adaptation methods, is to improve and extend current single-label learning algorithms so that they can be used in multi-label classification tasks. Representative methods include the ML-DT (multi-label decision tree), Rank-SVM (ranking support vector machine), and ML-KNNs (multi-label K-nearest neighbors). The MLDT method performs classification operations by constructing a decision tree; the Rank-SVM method handles the multi-label classification problem by means of an SVM; and the ML-KNN method improves upon the KNN algorithm to handle multi-labeled data.

Compared with individual classifiers, the performance of integrated classifiers is more robust. Therefore, some researchers have focused on integration algorithms, which combine multiple weak classifiers to obtain a new classifier with a better generalization ability. Common integration methods mainly include bagging and boosting. Bagging [42] is the most famous example of a parallel integration learning method, which creates an ensemble of classifiers over bootstraps through learning and then generates diverse classifiers [43]. Boosting [44,45,46] improves the accuracy of base classifiers by iterating them to detect misclassified data more finely, thus improving the classification accuracy. Freund et al. [47] proposed the adaptive boosting (AdaBoost) integration algorithm, which can effectively avoid overfitting compared to other classifiers. In 2000, Schapire et al. [48] applied the boosting algorithm to text categorization, using experiments to demonstrate the superiority of the boosting integration algorithm, with better performance compared to that of a single classifier. Winata et al. [49] applied the boosting algorithm to text categorization and demonstrated that adaptive boosting (AdaBoost) has high potential in boosting the performance of text categorization. Subsequently, Friedman H [50] proposed the GBDT algorithm: by iteratively training weak decision trees and optimizing the loss function using gradient descent, a classifier with high accuracy was obtained. The GBDT algorithm is able to deal with high-dimensional, nonlinear, and noisy data and can automatically select important features with high accuracy. Moreover, it has strong generalization and anti-interference abilities.

Based on the current state of research, we propose a multi-channel feature extraction model that leverages semantic features and label features, specifically a BR-GBDT multi-classification integration model. First, the BERT model is used to obtain the contextual representation of the text, and the association rule is used to obtain the label features of the text. Then, local and global features are extracted by a Text-CNN and Bi-LSTM, respectively, and an attention mechanism is used to achieve synergy between the features; in this way, the features can be successfully harnessed by the classification model to improve the category prediction accuracy.

3. Study Design

3.1. Modeling Framework

The multi-label classification model of civil aviation service quality complaint texts incorporating multiple features and attention mechanisms proposed in this study mainly consists of an input layer, a text vectorization representation layer, a multi-channel feature extraction layer, a feature interaction layer, a multi-label classification layer, and an output layer. The overall structure is shown in Figure 1.

In the input layer, raw text data are received and preprocessed to ensure that they enter the model in the appropriate format and quality. This includes removing irrelevant information, performing necessary text cleaning, and formatting operations.

In the text vectorization representation layer, the BERT model is used to obtain a vectorized representation of the text at a word-vector-level granularity, which is then combined with the attention weight based on the label information. The result of the vectorized representation then fuses the word embedding and the label embedding. The label information is used to guide the learning of the text feature representation.

In the multi-channel feature representation layer, the text vectors are input into the feature extraction layer involving local and global feature extraction. The local feature extraction part is a Text-CNN consisting of three convolutional kernels of different sizes, which is used to extract local features shared between word vectors within the range of the kernels. The different layers of local features obtained from the three channels are spliced together and output as local text features. The global feature extraction part is a BiLSTM network that captures the longer range dependencies. The text sequence is modeled in both forward and backward directions using the BiLSTM network to better capture bidirectional semantic dependencies; the feature outputs from the two directions are spliced together as the global text feature output.

In the feature interaction layer, local and global features are jointly input into a multi-head co-attention mechanism to extract key information in another feature vector, revealing the interactions between features. Finally, the features are spliced with the original feature vector as inter-feature relationship information to enable feature fusion.

In the multi-label classification layer, a separate binary classifier is trained for each label using the BR algorithm, with binary classifiers being constructed using SVM, KNN, NB, and DT methods among other algorithms. Then, the most effective classifier is used to predict the samples.

In the output layer, different thresholds are set for each label as needed to control the predictions, and the final output is the category labels.

3.2. Text Vectorization Representation Layer

Text representation is the process of transforming text into a numerical matrix, which is the first step of our proposed method. Traditional text vectorization models include Word2Vec, Doc2Vec, LDA, and others, but these models cannot solve the problem of the multiple meanings of words and cannot account for labeling information. The BERT model is a pre-trained language model released by Google [51], which can solve the problem of multiple word meanings. Tagged feature words are frequent features obtained based on association rule mining, which can account for the correlations between text and tags. The model learns a text vector representation for each tag based on the tag features, which improves the classification accuracy to an extent.

3.2.1. BERT Word Embedding

Text information is intuitively understandable for human beings but cannot be directly processed by computers. Therefore, it is necessary to transform text into numerical data. Traditional text representation methods include one-hot and matrix decomposition, but they have the disadvantages of dimensional disasters and a high computational cost. With the development of neural networks, word embedding has emerged as a new method of word representation, which uses a continuous, low-dimensional, dense vector to represent words, referred to as a word vector. Word2Vec and Glove are static word vector representations, meaning that for any word, its vector is constant and does not change with its context. However, static word vectors still cannot solve the problem of multiple word meanings.

In order to better represent the word features of text, we adopted a pre-trained model, BERT, to calculate the contextual representation of each word. Based on how different contexts can have different representations of the same word, BERT consists of a multi-layer transformer, accepts the input of a sequence of 512 words, and outputs the representation of the sequence. Its process is shown in Figure 2. For a document consisting of k words as an input, $W=[w_1,w_2,\dots ,w_k]$, the corresponding word vector $E_w=[e_1,e_2,\dots ,e_k]$ is obtained through the blue transformer module in the BERT model.

3.2.2. Label Feature Word Embedding

To consider the characteristics of civil aviation service quality and to improve the performance of the classification model, we independently constructed a feature word set for each label and assigned a higher weight to the feature words corresponding to the labels in the text vector representation. In this way, we made them play a greater role in the classification of each label. The steps of this workflow are described as follows:

• Feature word set construction for civil aviation service quality complaint text labeling:
  First, the corpus corresponding to each tag was segmented and lexically labeled sentence by sentence. The noun component of the sentence was taken as the basic transaction, and single-word nouns were filtered to create the basic transaction set. Then, the frequent 1-item sets $L_1$ were obtained using the association rule Apriori algorithm, and the frequent 2-item sets $L_2$ were obtained from $L_1$.
  The Apriori algorithm for association rules is a classic algorithm widely used in the field of data mining, aimed at discovering interesting association relationships between item sets in a dataset. This algorithm adopts an iterative strategy of a layer-by-layer search, first identifying all frequent item sets, then constructing new candidate sets based on these frequent item sets and calculating their support to filter out the next order of frequent item sets. This process continues until no more frequent item sets containing more items can be found.
  Since the frequent item set mining algorithm of association rules does not consider the positional relationships of nouns in the complaint text, the two nouns of $L_2^{\prime }$ may not form a phrase. So, when filtering according to the dependency relation $L_2$ of non-phrase features, only the noun phrases whose relation was a noun phrase (NP) were considered to obtain a new set of frequent 2-item sets, $L_2^{\prime }$. Then, from the candidate feature set $L_1$, we introduced domain dictionaries such as “civil aviation” and “civil aviation thesaurus” to supplement the mining of infrequent domain features. This enabled us to collect more comprehensive and accurate domain features, which we used to obtain $L_1^{\prime }$.
  The redundant features were then filtered by the minimum independent support. The minimum independent support is the support of the frequent 1-item set containing the feature word minus the absolute support of the frequent 2-item set containing the word. If the minimum independent support of a feature word was less than a threshold value, T, the word was filtered as a redundant feature, and the final set of frequent 1-item sets satisfying the independent support constraint was obtained, referred to as $L_1^{″}$.
  Finally, the semi-automatic updating of the civil aviation domain feature dictionary was realized by manually updating the dictionary with feedback on the mined frequent features, which saves human effort. After candidate feature mining, pruning, filtering, and infrequent item set supplementation, the domain feature set was finally obtained as $F=L_2^{\prime }\bigcup L_1^{\prime }\bigcup L_1^{″}$.
• Feature word embedding:
  (1) Training the Word2Vec model:
  First, we chose the model type. Word2Vec has two main model types, Skip-gram and CBOW (continuous bag of words). Skip-gram usually performs better on small datasets, while CBOW is faster for large datasets.
  Next, we set the hyperparameters of the model, including the vector dimension, window size, number of training iterations, and others.
  Then, the model was trained. The Word2Vec model was trained using preprocessed text data. The model learns word vectors by predicting the context of a word (Skip-gram) or predicting a word using the context (CBOW).
  (2) Obtaining word vectors for the topic feature words:
  For a feature word set consisting of t words, $F=[f_1,f_2,\dots ,f_t]$, we obtained the word vectors of label feature words from the trained Word2Vec model as $E_f=[e_{f_1},e_{f_2},\dots ,e_{f_t}]$.

3.2.3. Text Vector Representation Based on Attention Weighting of Label Features

In this subsection, we propose a text vector representation based on the attentional weighting of label features. First, we calculated the IDF value for each feature word, which was used to measure the importance of that feature word in the whole corpus:

$$ID{F}_f=log{\displaystyle \frac{N}{D{F}_f}}$$

(1)

Here, N is the total number of comment documents under this tag, and $D{F}_f$ is the number of documents in which the feature word f appears.

Second, the attention weight of each feature word was computed using TF-IDF, which was represented as a weight vector of the feature word. For a feature word in a document, f, its TF-IDF weight is

$$TFID{F}_f=T{F}_f\times ID{F}_f$$

(2)

where $T{F}_f$ is the frequency of the feature word f in the current document.

Then, the computed attention weights were used as the weight coefficients of the feature word embedding matrix and multiplied with the corresponding word vectors in order to construct the label feature word embedding weight matrix $E_{f-Attention}$.

$$E_{f-Attention}=E_f\odot TFID{F}_f$$

(3)

Here, $E_f$ is the feature word vector of the label, and $TFID{F}_f$ is the attention weight of the feature word, and ⊙ denotes the dot product operation of the vectors, which multiplies the corresponding positional elements of the matrix.

Finally, the BERT word embedding matrix $E_w$ was multiplied with the label feature word embedding weight matrix $E_{f-Attention}$ to obtain the label feature attention-weighted text vector representation X:

$$X=E_w\odot E_{f-Attention}$$

(4)

In Equation (4), $E_w$ is the word vector obtained from the BERT model.

3.3. Multi-Channel Feature Extraction Layer

Although the vectors obtained from the text representation layer contain contextual relationships and labeling information, they still have the problems of noise and high dimensionality. At the same time, the features of the complaint text are sparse and need to be extracted and reduced in dimension by the feature extraction network. In this section, we describe how Text-CNN and BiLSTM multi-channel feature extraction networks with parallel structures were used to extract the local and global features of the text, respectively.

3.3.1. Text-CNN-Based Localized Feature Extraction

The Text-CNN [51,52,53] is a network with a strong text feature extraction capability. In this study, a Text-CNN with three different sizes of convolutional kernels was used to extract feature information within the range of the kernels, and maximum pooling was performed to reduce the dimensionality. The Text-CNN’s structure is shown in Figure 3.

The output, X, of the text representation layer is first fed into the Text-CNN, and the convolutional kernel performs sequential convolution operations in the text matrix to extract local features. A local feature is extracted as $c_j$, as shown in Equation (5):

$$c_j=f(w·y_{j:j+r-1}+b)$$

(5)

Here, $f(·)$ is a nonlinear activation function, ReLU, and b is a bias term, and w is a convolutional kernel parameter matrix of the dimension $r\times k$. The heights of the convolutional kernel for the three channels’ r values are set to 3, 5, and 7, respectively.

For all the samples after convolution, a collection of feature maps is obtained as follows:

$$C=\{c_1,c_2,\dots ,c_{n-h+1}\}$$

(6)

In Equation (6), C denotes the set of feature maps after convolution, $c_j$ is the convolutional kernel used to extract the first j localized features, $j=1,2,\dots ,n-h+1$, and $n-h+1$ is the number of feature maps after convolution is performed by the original feature dimension n and the height of the convolution kernel h is computed.

In this study, we used maximum pooling for the feature extraction and dimensionality reduction of the set of feature maps output from each channel. Maximum pooling means extracting the maximum value in each feature map, as shown in Equation (7).

$$C^{\prime }=max\left\{C\right\}$$

(7)

Here, $C^{\prime }$ denotes the feature that has undergone the maximum pooling output. $max$ means that the largest value in the feature region, which is the most responsive part of the feature, is selected to proceed to the next layer.

Three convolutional kernels of different sizes are used for feature extraction, and the output of each channel is pooled to obtain a feature vector of the size $1\times r$, and then the outputs of the three channels are spliced together, as defined by

$$A=C_1^{\prime }\oplus C_2^{\prime }\oplus C_3^{\prime }$$

(8)

where $C_1^{\prime }$, $C_2^{\prime }$, and $C_3^{\prime }$ are the output features after convolution and maximum pooling operations with convolutional kernels with heights of 3, 5, and 7, respectively. A is the output of the extracted localized feature vectors, and ⊕ represents the concatenation of vectors.

3.3.2. BiLSTM-Based Global Feature Extraction

BiLSTM [54,55,56] is a variant of LSTM which can model text in both forward and reverse directions, obtain the dependencies between the previous and current text, and realize global feature extraction from text sequences. The “gate” structure of BiLSTM is shown in Figure 4.

At time t, the current hidden layer state is noted as $h_t$, and the state of each gate is updated as shown:

$$f_t=\sigma (w_c·[h_{t-1},x_t+b_f])$$

(9)

$$c_t=f_t\times c_{t-1}+i_t\times tanh(w_c·[h_{t-1},x_t+b_c])$$

(10)

$$i_t=\sigma (w_i·[h_{t-1},x_t]+b_i)$$

(11)

$$o_t=\sigma (w_o·[h_{t-1},x_t]+b_o)$$

(12)

$$h_t=o_t\times tanh\left(c_t\right)$$

(13)

Here, $x_t$ is the current input cell state. $f_t$ is the forgetting gate at moment t, and $c_t$ is the memory cell. $i_t$ and $o_t$ are the input gates and output gates, respectively, at moment t. $w_c$, $w_i$, and $w_o$ are the weight matrices. $b_f$, $b_c$, $b_i$, and $b_o$ are the biases of the forgetting gate, memory cell, input gate, and output gate, respectively. $\sigma$ is the activation function, and $h_t$ is the output of the hidden layer at time t. The operator × computes the vector product of the two vectors before and after the current point. · denotes the dot product, whose result is a numerical value.

BiLSTM models the text using LSTM in both forward and reverse directions and concatenates the outputs of the last moment of the forward and reverse networks together as the final output, as defined by

$$B=h_a\oplus h_b$$

(14)

where $h_a$ and $h_b$ denote the outputs of the last hidden layer state for forward and reverse modeling, respectively. $B\in R^{d\times 2}$ is the output of the extracted global feature vector. ⊕ represents vector concatenation.

3.4. Feature Interaction Layer

The co-attention mechanism [30] reflects the relationship between feature vectors by calculating the shared similarity matrix, and then the shared similarity matrix is applied to the original feature vectors to reveal information interactions between features. On the basis of the cooperative attention mechanism and multi-head attention mechanism [34,57], we constructed a multi-head co-attention mechanism for the interactions between the local and global features of the complaint text. In the proposed model, the input of the multi-head co-attention mechanism is the local feature matrix output from the multi-channel feature extraction network, $A\in R^{d\times 3}$, and the global feature matrix $B\in R^{d\times 2}$, and h is the number of co-attention layers.

The input features are first downscaled through h individual linear layers to downscale the number of input features to obtain, thus obtaining the representation of the same feature vector in several different vector spaces as shown in Equations (15) and (16):

$$A_1,A_2,\dots ,A_h=Linea{r}_{1,2,\dots ,h}^a\left(A\right)$$

(15)

$$B_1,B_2,\dots ,B_h=Linea{r}_{1,2,\dots ,h}^b\left(B\right)$$

(16)

Here, a and b represent the linear dimensionality reductions of the A and B feature matrices, respectively. d is the vector dimension of a single channel, and h is the number of dimensionality reduction layers. The features $A_i\in R^{(d/h)\times 3}$ and $B_i\in R^{(d/h)\times 2}$, after dimensionality reduction using the function $Linea{r}_{1,2,\dots ,h}^b(·)$, are then inputted into multiple collaborative attention mechanisms to learn the relationships between the features. The structure of the ith collaborative attention mechanism is shown in Figure 5.

The shared similarity matrix $C_i\in R^{3\times 2}$ between feature vectors is first calculated within the collaborative attention mechanism, and the normalized similarity matrix $C_i$ contains the information about the similarity between local and global features. In particular, the vector inner product is performed to compute the similarity matrix, so the widths of the feature vectors must be equal. It is calculated as follows:

$$C_i=tanh(A_i^T·B_i)$$

(17)

where $A_i^T$ is the transpose of the local feature matrix $A_i$, and $B_i$ is the global feature matrix.

The similarity matrix is given by the outer product with the original feature vector matrix, which is used to extract the key information of the output feature vector of the other channel to reveal feature interactions. The extracted features $V_i^a$ and $V_i^b$ are calculated as

$$V_i^a=tanh\left(A_i{C}_i\right),V_i^b=tanh\left(B_i{C}_i^T\right)$$

(18)

Here, $A_i$ is the original local eigenmatrix. $B_i$ is the original global feature matrix. $C_i$ is the similarity matrix of the two, and $C_i^T$ is the transpose of $C_i$.

The input and output feature dimensions of the co-attention mechanism are unchanged, and the local and global features of the ith outputs of the co-attention mechanism are denoted as $V_i^a\in R^{(d/h)\times 2}$ and $V_i^b\in R^{(d/h)\times 3}$, respectively. Finally, the outputs of multiple co-attention layers are stitched together as shown in Figure 6 to obtain the feature vector interrelationship outputs $Vie{w}_{ab}\in R^{d\times 2}$ and $Vie{w}_{ba}\in R^{d\times 3}$, calculated as shown:

$$Vie{w}_{ab}=concat(V_1^a,V_2^a,\dots ,V_h^a)$$

(19)

$$Vie{w}_{ba}=concat(V_1^b,V_2^b,\dots ,V_h^b)$$

(20)

In these equations, h is the number of layers of the multi-head co-attention mechanism. $V_1^a$ is the result of each local feature interacting with a global feature, and $V_1^b$ is the result of each global feature interacting with a local feature. $concat(·)$ denotes the concatenation of the pointers.

Since the width of the output matrix after dimensionality reduction is set as $d/h$, the input features of the multi-head co-attention mechanism and the dimension of the output features after concatenation remain the same.

In this study, we drew on operations similar to residual concatenation in ResNet and the operation in transformers that involves the attention mechanism being applied before being added to the original features. Specifically, the feature vectors of the interactive $Vie{w}_{ab}$, the $Vie{w}_{ba}$, and the local feature A and global feature B of the customer complaint text before the interaction are spliced into the fully connected layer, so as to avoid vanishing gradients and network degradation caused by the large number of layers in the model. Finally, a real feature vector, $F\in Rd\times 10$, containing information about the relationship between feature vectors is obtained, defined as

$$F=concat(Vie{w}_{ab},Vie{w}_{ba},A,B)$$

(21)

where $concat(·)$ is the concatenation operation of the pointers. $Vie{w}_{ab}$ is the local–global relational feature obtained on the basis of local features. $Vie{w}_{ba}$ is the global–local relation feature obtained on the basis of global features, and A is the local feature before the interaction. B is the global feature before the interaction.

3.5. Multi-Label Classification Layer

3.5.1. Binary Relevance Model

Binary relevance [34] is the most straightforward of the multi-label classification methods. Its main idea is to decompose the multi-label classification task into q single-label supervised classification tasks. In particular, we trained individual classifiers to make predictions for individual labels and finally combined the q predictions of the individual classifiers into the final label set of the sample.

According to this basic idea, for the first label j in the label space $y_j$, by considering each training sample, $x_i$, and the label $y_j$ correlations, a corresponding binary training set is constructed:

$$\phi (Y_i,y_i)=\left\{\begin{array}{cc}+1,& if y_i\in Y_i\\ -1,& otherwise\end{array}\right.$$

(22)

$$D_j=\{(x_i,\phi (Y_i,y_i))\mid 1\le i\le m\}$$

(23)

Here, $Y_i$ is the sample $x_i$ of the set of labels of $\phi (Y_i,y_i)$, where m represents the total number of samples.

A binary learning algorithm, B, can then be used to train a classifier for the labels $y_j$, training to obtain a classifier, $g_j:X\to R$, such that $g_j:X\leftarrow B\left(D_j\right)$. Thus, any multi-label training sample, $(x_i,Y_i)$, in the text $x_i$ will be involved in the q during the training process of the individual classifiers. For relevant labels, $y_j\in Y_i$, the $x_i$ for a classifier, $g_j$, can be regarded as a positive example. On the other hand, for irrelevant samples, $y_k\in Y_i$, the $x_i$ can be regarded as a negative example.

For the unseen samples, x, the binary relevance is predicted by q individual classifiers to predict their associated set of labels, Y. Eventually, all the predicted independent binary labeling results are stitched together to obtain the final prediction:

$$Y=\{y_j\mid g_j\left(x\right)>0,1\le j\le q\}$$

(24)

Using the binary relevance method for classification, the original problem is transformed into a number of unrelated single-label classification problems. For the samples, an empty set of predicted labels will be generated if each classifier predicts it as being a negative example. To avoid this, we can use the T-criterion rule, i.e., when every classifier predicts sample x as a negative sample, the label with the highest prediction probability among all the classifiers is selected and added to the label set.

3.5.2. GBDT Integration Algorithm

The GBDT algorithm is trained in an iterative manner, where the next iteration utilizes a negative gradient to measure the performance of the previous round of the weak learner. The previously occurring errors are corrected by fitting a mean square error (MSE) loss function in the negative gradient direction. Eventually, a fitting function, $f_0\left(x\right)$, is determined that makes the result approach the true value.

The basic principle is to first initialize the inaccurate value, $f_0\left(x\right)$, of the residual $r_0\left(x\right)=y-f_0\left(x\right)$, which is calculated based on the difference between it and the true value, y, of the sample. Thus, in order to find a suitable fitting function, $h_0$, to fit $r_0\left(x\right)$, the new fitting function for the next round can be expressed as $f_{i+1}\left(x\right)=f_i\left(x\right)+h_i\left(x\right)$ to calculate the new residual value $r_{i+1}\left(x\right)=y-f_i\left(x\right)$. This is performed in an iterative manner. This iterative correction makes the sample prediction value $f\left(x\right)$ approach the true value y. The GBDT algorithm is convergent and can achieve local or global optimization.

Specifically, the weak classifier is initialized first:

$$f_0\left(x\right)=argmin\sum _{i=1}^{N}L(y_i,\rho )$$

(25)

$$L(y,f\left(x\right))={(y-f\left(x\right))}^2$$

(26)

where $y_i$ is the actual value of the sample and $\rho$ is the constant. $L(·)$ is the loss function. We determined the constant $\rho$ so that the initial prediction loss was minimized, and then the total loss for all N samples was

$$L_{all}=\sum _{i=1}^{N}L(y_i,f_m\left(x_i\right))$$

(27)

where $y_i$ and $f_m\left(x_i\right)$ are the sample, $x_i$, of the actual value and its first m prediction functions according to the first model, respectively. The purpose of the iteration is to minimize the loss value and find the direction of the steepest gradient descent, so the negative gradient function computed in each iteration is

$$-g\left(x_i\right)=-{\displaystyle \frac{\partial L(y_i,f\left(x_i\right))}{\partial f\left(x_i\right)}}$$

(28)

We constructed a fitting function, $h(x_i;\alpha )$, to fit the negative gradient $-g\left(x_i\right)$ where $\alpha$ is the residual coefficient. The residual parameters are obtained according to the following equation:

$${\alpha }_m=argmin\sum _{i=1}^N{(-g\left(x_i\right)-\beta h(x_i;\alpha ))}^2$$

(29)

in which $g\left(x_i\right)$ is the gradient and $\beta$ is the coefficient.

The next step is to calculate the optimal ${\beta }_m$ value:

$${\beta }_m=argmin\sum _{i=1}^N(y_i,f_{m-1}\left(x_i\right)-\beta h(x_i;\alpha ))$$

(30)

where ${\beta }_m$ is the weighting factor and $f_{m-1}\left(x_i\right)$ is the fitting function for the first $m-1$ iterations.

Eventually, the results of the calculations are merged into the model to update the prediction function:

$$f_m\left(x\right)=f_{m-1}\left(x\right)+{\beta }_m{h}_m(x;{\alpha }_m)$$

(31)

Here, $h_m(x;{\alpha }_m)$ is the first negative gradient fitting function after $m-1$ iterations. When the number of iterations preset by the algorithm is reached, the iteration is terminated.

3.6. Output Layer

We set different thresholds for each label to control the prediction results of the labels as needed. The final output is the category labels predicted by the model.

4. Experimental Results and Analysis

In the experimental phase, we tested whether the proposed multi-label classification model outperforms existing algorithms with regard to civil aviation complaint texts. We also verified whether the model can achieve the efficient multi-label classification of the complaint texts. The classification outcomes facilitated an in-depth analysis of the primary issues and trends in civil aviation complaints. This helped identify common complaint areas and potential areas for improvement, thereby providing valuable insights for airlines, airports, and sales platforms to enhance their services.

4.1. Experimental Data

In this study, the performance of the proposed model was tested using a complaint text dataset from civil aviation and related departments. The dataset consisted of complaint texts from between 1 and 11 January 2022, initiated by passengers and targeted to domestic airlines, domestic airports, and foreign and Hong Kong, Macao, and Taiwan airlines, as well as aviation sales network platforms. These texts were mainly in Chinese, consisting of 13 categories and totaling 5597 samples. The data were divided into training, testing, and validation sets according to the ratio of 7:2:1. During the data processing, we gave special attention to the segmentation of Chinese text and the construction of domain-specific feature word sets, aiming to ensure that the model accurately captures the unique characteristics of Chinese text. Notably, despite the test data being Chinese text, the proposed model is theoretically applicable to any text. This was achieved by simply considering the linguistic conventions of different languages during text segmentation and the construction of domain-specific feature word sets. The statistical results of the data utilized in this study are presented in

Table 1. Experimental data description.

Form	Number of Texts	Proportion of Samples
Ticketing service	2988	53.39%
Irregular flight service	1380	24.66%
Check-in and boarding	468	8.36%
Baggage services	248	4.43%
Air services	127	2.27%
Member services	108	1.93%
First aid	99	1.77%
Notice of information	75	1.34%
Special passenger services	35	0.63%
Terminal basic services	29	0.52%
Overselling	25	0.45%
Airport merchant services	10	0.18%
Ground transportation services	5	0.09%

.

4.2. Measurement Indicators

The algorithm performance was evaluated using the classical multi-label classification performance metrics: the F1 value, recall, and Hamming loss. Each performance statistic of multi-label classification prediction is defined as follows: T is the number of samples that predicted positive case samples as positive, F is the number of samples that predicted negative case samples as negative, and A is the number of samples that predicted negative case samples as positive cases or positive case samples as negative cases.

(1) Recall:

$$R={\displaystyle \frac{T}{T+F}}$$

(32)

The recall, R, was used to evaluate the average proportion of all samples that successfully predicted the category.

(2) $F_1$ value:

$$P={\displaystyle \frac{T}{T+A}}$$

(33)

$$F_1={\displaystyle \frac{2PR}{P+R}}$$

(34)

The precision rate P denotes the proportion of the sample that is correctly categorized, and R is the recall of the predicted results. The $F_1$ is the harmonic mean of the precision rate and the recall, which is a more generalizable metric than each individually.

(3) Hamming loss:

$$H={\displaystyle \frac{1}{p}}\sum _{i=1}^p{\displaystyle \frac{1}{q}}\left|h\left(x_i\right)\Delta Y_i\right|$$

(35)

The Hamming loss was used to assess the proportion of misclassified samples. Here, p denotes the number of samples, q denotes the number of labels, $h\left(x_i\right)$ is the predicted labeling of the sample $x_i$, and $Y_i$ is the true labeling of sample $x_i$. $\Delta$ denotes the dissimilarity between the two sets, and thus $\left|h\left(x_i\right)\Delta Y_i\right|$ denotes the number of samples with incorrect predictions. When each sample in the dataset is related to only a single label in the label set, the Hamming loss will degenerate to the traditional misclassification rate of ${\displaystyle \frac{2}{q}}$.

4.3. Result Analysis

4.3.1. Model Performance Comparison

In order to verify the effectiveness of combining multi-feature co-attention mechanisms and integration algorithms for classification, the following multi-label classification methods were chosen for comparative experiments:

(1) BR-GBDT. This is a multi-label classification method based on the integrated learning of the binary relevance (BR) and gradient boosting decision trees (GBDTs) which uses maximum pooling to extract features. It also combines the BR and GBDTs to achieve automated, high-accuracy multi-label classification.

(2) Multi-Head Co-attention Classifier. This is a multi-label classification model based on the multi-head co-attention mechanism. First, the BERT model is used to realize text vectorization representation, then the Text-CNN and BiLSTM multi-channel feature extraction networks are used to extract the local and global features of the complaint text, respectively. Finally, a co-attention mechanism is used to learn the relationship between the local and global features and realize the accurate classification of the customer complaint text.

(3) Multi-Feature Attention GBDT Classifier (MAG). This is a text classification algorithm that fuses multiple features and attention mechanisms and is the method proposed in this study. It considers multiple features, including label features, in the text vectorization representation stage, fuses the extracted global and local features of the complaint text using a multi-head co-attention mechanism in the feature extraction stage, and uses an integration method to obtain optimal multi-label classification. The model can fully extract text features as well as achieve efficient integration, thus enhancing the performance of the classifier.

As shown in

Table 2. Comparison of model performance results. The bold values represent the best performances.

Model	F1 Value	Recall	Hamming Loss
BR-GBDT	0.7200	0.7504	0.0856
Multi-Head Co-attention	0.6952	0.7086	0.0905
MAG (Ours)	0.7260	0.7656	0.0847

, the proposed MAG model significantly outperforms the traditional BR-GBDT and Multi-Head Co-attention Classifier models in terms of both the F1 value and recall. Compared with the other two types of models, the MAG model improved the F1 value by 3.08 and 0.60 percentage points, respectively. Also, the Hamming loss of the MAG model was significantly lower than that of the other two methods. This means that the MAG can predict labels more accurately in multi-label classification tasks, reducing the risk of misclassification.

The combined application of multi-feature fusion, the multi-head co-attention mechanism, and GBDT integration enables the MAG model to capture the relationships between features and labels more accurately. Therefore, the MAG model has great potential to be used for the multi-label classification of complaint texts, improving the classification effect and enhancing the feasibility of application. The results of this research provide support for applying multi-feature co-attention mechanisms and integration algorithms in the field of multi-label text classification.

4.3.2. Analysis of Ablation Experiments

To verify the effectiveness of the prior extraction of labeled feature words, the global feature extraction, and the local feature extraction, five sets of comparative ablation experiments were performed. They involved a model with no label information (Model 1), a model with no global features (Model 2), a model with no local features (Model 3), a model with no fusion of features using the multi-head attention mechanism (Model 4), and the MAG model proposed in this paper. The results are shown in

Table 3. Results of ablation experiments. The bold values represent the best performances.

Ablation Experiment	F1 Value	Recall	Hamming Loss
Model 1	0.7130	0.7549	0.0876
Model 2	0.7164	0.7656	0.0876
Model 3	0.7073	0.7362	0.0876
Model 4	0.7182	0.7656	0.0868
MAG (Ours)	0.7260	0.7656	0.0847

. It can be seen that the MAG model showed different degrees of improvement in each index compared to the four ablation models; this suggests the effectiveness of the model in considering the label information, global features, and local features.

As shown in Table 3, the proposed method achieved the best F1 value, recall, and Hamming loss index compared with feature extraction methods that do not use label features, global features, local features, or multi-attention fusion. Thus, the label features are able to enrich the textual information, and the fusion of the multi-channel feature extraction network improves the effect significantly compared with the single-channel feature extraction network. Moreover, the combination of the global and local features enables complementarity, which can facilitate the more comprehensive extraction of the complaint text information and alleviate the issues with short and sparse complaint texts. The results of this ablation experiment emphasize the importance of these elements in the original model, which play a key role in improving the performance for multi-label classification.

4.3.3. Label Classification Effect Analysis

Using the same dataset, the MAG model was applied for multi-label classification, and a comparison of the recognition effect of each label with the number of texts was obtained, as shown in Figure 7.

According to the sample proportion situation shown in Table 1, it is clear that ticket service and irregular flight service are the two types of problems with the highest degree of concern for civil aviation passengers: the number of texts on ticket service accounted for 53.39% of all texts, and the number of texts on irregular flight service accounted for 24.66% of all texts. An analysis of possible reasons for this is given as follows:

(1)

High-frequency incidents: Complaints in the categories of ticketing services and irregular flight services are usually closely related to passengers’ daily travel. Problems in these two categories are common and therefore have a higher chance of giving rise to complaints.

(2)

Direct impact on the passenger experience: Ticketing and flight services have a direct impact on the overall passenger experience and travel plans. Any problems related to ticketing errors, itinerary changes, flight delays, cancellations, etc., may lead to traveler complaints, as they can significantly affect passenger travel.

(3)

Measurability: Problems in the ticket service and irregular flight service categories are usually easily measurable and recordable and therefore more likely to be recorded in complaint data. In contrast, other types of problems may be less easily captured and recorded.

(4)

Cumulative effect: Problems with ticketing or flight services that occur during a single trip may result in multiple complaints, as they may affect multiple stages and aspects of the trip. This can lead to an increase in the percentage of complaints in both categories.

In summary, the larger share of complaints in the ticket service and irregular flight service categories is likely because these issues are more prevalent, directly affect the traveler’s experience, are easily documented and measured, and may generate multiple complaints during a single trip. These issues constitute important areas that airlines, airports, sales platforms, and other related institutions need to pay attention to and address.

According to the classification results, we can see that the majority class samples appeared more frequently in the training data, and the model is more likely to learn the features and patterns of the majority class. This makes the classification performance on the majority class samples tend to be more accurate. As shown in Figure 7, ticket service has the highest multi-label classification F1 value and the best classification effect; irregular flight service has the second-best classification effect. In addition, the recognition of minority sample labels is less effective due to the fact that other labels appear less in the training data, and their textual features are not learned as well.

5. Conclusions and Future Work

In order to better realize the multi-label text classification of traveler complaint data in the field of civil aviation, we proposed a novel multi-label classification model for complaint texts. This model fuses label information and attention mechanisms, combines the semantic features extracted by BERT and the label features extracted based on association rules to represent text vectors, and then extracts local and global features using a Text-CNN and Bi-LSTM, respectively. It fuses features using a multi-head co-attention mechanism, which reduces feature loss by fusing information at different granularities, and finally constructs a GBDT integrated classifier for each label. This improves the classification performance of the model. Experiments showed that compared to other models, the proposed model achieved better results on the civil aviation complaint text dataset, effectively improving the classification performance.

Through the analysis of the classification results, we can provide the following recommendations for airlines, airports, sales platforms, and other related customer service teams:

First, as the two main areas where passenger complaints are most concentrated are ticketing services and abnormal flight services, service providers should focus on improving these areas. Airlines can optimize ticketing processes to reduce errors and disputes, while airports can enhance real-time flight information updates and passenger services to minimize the inconvenience caused by flight delays or cancellations. Sales platforms can provide more accurate ticketing information and after-sales services to boost passengers’ confidence and satisfaction in their purchases.

Second, in response to the differences in the classification performance, service providers can target improvements in data collection and labeling. For categories such as ticketing services and abnormal flight services, which exhibit better classification results, valuable information from the data can be further explored to support service improvements and decision-making. For categories with poorer classification results, particularly those with few sample labels, data collection and labeling efforts should be increased to enhance the model’s generalization ability and accuracy.

Additionally, our research findings suggest that service providers should pay attention to the cumulative effect of passenger complaints. Issues related to ticketing or flight services that arise during a single trip may trigger a chain reaction, leading to multiple complaints. Therefore, service providers should establish effective complaint handling mechanisms, respond promptly to passengers’ issues, and prevent the escalation and worsening of problems.

In conclusion, our research not only reveals the primary issues and distribution patterns of civil aviation passenger complaints but also provides actionable directions and measures for airlines, airports, sales platforms, and other customer service teams. By targeting service process optimization, strengthening data collection and labeling, and establishing effective complaint handling mechanisms, service providers can further enhance service quality, increasing passenger satisfaction and loyalty.

In future work, hierarchical label features will be considered in multi-label text classification to further refine and extract the feature relationships of labels, potentially yielding more accurate category prediction. Simultaneously, we will explore innovations in natural language processing methods and tools to optimize the model performance, providing more precise and efficient solutions for the processing and analysis of passenger complaint data in the civil aviation industry.