Civil Aviation Travel Question and Answer Method Using Knowledge Graphs and Deep Learning

Abstract

In this paper, a civil aviation travel question and answer (Q&A) method based on integrating knowledge graphs and deep learning technology is proposed to establish a highly efficient travel information Q&A platform and quickly and accurately obtain question information and give corresponding answers to passengers. In the proposed method, a rule-based approach is employed to extract triads from the acquired civil aviation travel dataset to construct a civil aviation travel knowledge graph. Then, the ELECTRA-BiLSTM-CRF model is constructed to recognize the entity, and an improved ALBERT-TextCNN model is used for intent classification. Finally, Cypher query templates are transformed into Cypher query statements and retrieved in the Neo4j database, and the query returns the result, which realizes a new civil aviation travel Q&A method. A self-built civil aviation dataset is selected to prove the effectiveness of the proposed method. The experimental results show that the proposed method based on integrating knowledge graphs and deep learning technology can achieve better Q&A results, and it has better generalization and high accuracy.

1. Introduction

In recent years, with the development of the economy, the volume of air transportation has increased year by year, and the passenger flow of civil aviation has climbed year by year. At the same time, passengers’ demand for civil aviation travel information is getting higher and higher, and the total number of consumer complaints is increasing. Traditional manual customer service has problems such as slow response speed and low service quality. In order to solve this problem, we want to improve the performance of the question answering system and improve the algorithm.

1.1. Question and Answer System

As a popular research direction in the field of natural language processing, question and answer systems were initially developed for applications in the medical field [1,2,3], industrial field [4,5,6,7], agricultural field and other fields [8,9,10,11,12]. Cui et al. [13] proposed optimization for factual QA in terms of automatic template generation methods. Cocco et al. [14] proposed an object-oriented QA system based on SPARQL templates through the use of machine learning methods. Sarrouti, M et al. [15] proposed a semantic biomedical QA system named SemBioNLQA, which has the potential to deal with a large amount of question-and-answer types. Noraset et al. [16] proposed WabiQA, a new system for automatically answering Thai language questions using Thai Wikipedia articles as a source of knowledge. Li et al. [17] developed a travel QA system that combines knowledge from the fundamentals of large-scale language models, such as BERT. In contrast, the application of question-and-answer systems in civil aviation is late and still in a preliminary stage of development compared to other fields. If an advanced system for civil aviation can be developed, it will enhance the reputation of the civil aviation industry among passengers and promote the development of the civil aviation industry.

The entity recognition and intent recognition in intelligent Q&A are the technical difficulties and core aspects of this module, and different researchers have adopted different approaches toward its implementation. The task of named entity recognition in automatic knowledge graph-based Q&A systems is to accurately identify proper nouns in users’ question statements. Luo et al. [18] implemented document-level electronic medical record entity recognition based on Bi LSTM-CRF by incorporating an attention mechanism, whose role in entity recognition is to obtain global information about the document. Wang et al. [19] proposed a cross-lingual entity recognition method based on an attention mechanism and adversarial training to improve the effectiveness of question-and-answer systems to focus on intent recognition. For the intent recognition module, most of the traditional text classification methods are based on machine learning methods; for example, He et al. [20] proposed a classification framework for commodity trade declarations based on machine learning (ML) models that achieved high accuracy rates. These traditional machine-learning-based approaches require a large number of labeled training sets, are costly, have low accuracy in feature extraction and require significant training time.

In recent years, with the advent and increasing maturity of deep learning techniques, researchers have started to apply deep learning methods to text classification. Elnagar et al. [21] investigated the use of the word91vec embedding model to improve the performance of classification tasks. Al-onazi et al. [22] proposed a hyperparameter-tuned hybrid deep learning (AATC-HTHDL) model for the automatic classification of Arabic texts. Kompally et al. [23] developed a novel decentralized deep learning method called MaLang to detect abusive textual content to address the problem of using toxic or abusive content on any messaging application within a company’s network. Khattak et al. [24] proposed a poetry text sentiment classification system using a deep neural network model. Yang et al. [25] proposed a deep neural network (MTDNN)-based multi-applicable text classification method. We try to study more advanced algorithms to ensure the superior performance of the civil aviation question answering system.

1.2. Knowledge-Based Question Answering

With a development in knowledge graphs, knowledge base question answering (KBQA) has become a hot research topic. Compared with traditional search engines, it improves the convenience of users’ access to information, saves time in accessing information to a certain extent and improves the quality of information. Knowledge graph technology has strong potential in deep Q&A, and its construction method has been a hot research topic. Qu [26] synthesized the structure and construction techniques of medical knowledge graphs according to the characteristics of big data in the medical field, such as strong specialization and complex structure, and reviewed the application of knowledge graph technology in the medical field. Chen et al. [27] proposed a forest fire prediction method based on knowledge graphs and representation learning. Huang et al. [28] constructed methods and proposed applications of knowledge graphs in oil exploration and development. Gupta et al. [29] designed a knowledge graph for missiles. Chen et al. [30] constructed a knowledge graph containing information about the activities of COVID-19-infected individuals. The above studies show that knowledge graphs are widely used in education, tourism, clothing, law and agriculture, and these different fields provide good data support for various industries. However, there are many kinds of civil aviation data, which are highly secure and not easily accessible, and how to complete the construction of the knowledge graph of civil aviation travel is one of the hot spots of current research. Cheng et al. [31] used the graph database Neo4j to store knowledge in the field of civil aviation security, which was the initial construction of a knowledge graph in the field of civil aviation security. Electra [32] proposed a new framework for model pre-training, using a combination of a generator and discrimination. ALBERT is a lightweight BERT designed with parameter reduction to reduce memory consumption while speeding up training [33]. In this paper, the above two models are used for the pre-training of entity recognition and intention recognition.

1.3. The Development of Question Answering System in Civil Aviation

In the field of civil aviation Q&A, Ankush et al. [34] proposed a knowledge-graph-guided deep learning method based on a question answering system for aviation safety, constructed a knowledge graph from aircraft accident reports and contributed this resource to the community of researchers. Li et al. [35] proposed a reading comprehension model based on cooperative attention and adaptive adjustment for the problem that traditional Q&A systems do not have language comprehension, but there is a problem of slow model inference. Luo et al. [36] constructed an intelligent Q&A-QACAPS system for civil aviation passenger self-service, but there are still some unanswerable questions that need to be obtained from this text.

Section 1 is the introduction and literature review. The related theories and main basic models are described in Section 2. Section 3 gives the knowledge graph and deep-learning-based Q&A approach for civil aviation travel in detail. The experimental procedure and analysis for the entity recognition model and the intention recognition model are provided in Section 4. Conclusions and future research are given in Section 5.

2. Related Theories and Main Basic Models

2.1. Model Framework

The framework of the proposed method is shown in Figure 1, which consists of three main modular parts: (1) knowledge graph construction module, (2) question pre-processing module and (3) answer generation module.

The main modular parts are described in detail as follows:

(1) Knowledge graph construction part: Pre-process the acquired data, and then integrate and extract the entity and relationship-related attributes of the data. Build <entity-relationship-entity> triad data model. Import the obtained triples into Neo4j for storage.

(2) Question pre-processing part: the user input questions are divided into words, deactivated words, etc., by jieba; then, the entity recognition is performed using the ELECTRA-BiLSTM-CRF model, and finally the intention classification is performed using the improved ALBERT-TextCNN model to obtain the classification results of the questions.

(3) Answer generation part: different Cypher query templates are selected according to the identified entities and classification results, and then the corresponding query statements are constructed to obtain the answers required by users from the knowledge graph.

2.2. Related Main Basic Models

2.2.1. ELECTRA Model

The structure of the ELECTRA model has two main components: the generator and the discriminator. Both generator and discriminator structures use the Transformer’s coding network. The structure of the ELECTRA model is shown in Figure 2.

The model randomly selects 15% of the words to be tokenized to obtain the token sequence x, which is shown in Equation (1).

$$x=\left[x_1,\cdots \cdots ,x_n\right],$$

(1)

Next, a vector representation of the context of the token is performed to obtain the sequence $h_G\left(x\right)$, which is shown in Equation (2).

$$h_G\left(x\right)=\left[h_1^G,\cdots \cdots ,h_n^G\right],$$

(2)

where the superscript G is used to distinguish the generators.

The generator is used to firstly predict the masked words and then replace the labeled words with the predicted words, where the prediction uses softmax (normalized exponential function) to calculate the probability of token generation, which is calculated as shown in Equation (3).

$$P_G\left(x_t|x\right)=\frac{exp\left(e{{\left(x_t\right)}^{T}h}_G{\left(x\right)}_t\right)}{{\sum }_{x^{\prime }}exp\left(e{\left(x^{\prime }\right)}^T{h}_G{\left(x\right)}_t\right)},$$

(3)

where $h_G\left(x\right)$ denotes the encoded vector, $e\left(x_t\right)$ denotes the word embedding vector of $x$ and t denotes the position.

Finally, the processed sentence by the generator is inputted to the discriminator to predict each token in the sentence and determine whether the token is replaced according to the prediction result. The sigmoid function is used for binary discrimination, which is shown in Equation (4).

$$D\left(x,t\right)=sigmoid\left(w^T{h}_D{\left(x\right)}_t\right),$$

(4)

where $w^T$ is the prediction model parameter of the discriminator.

To better understand the actual effect of the generator and discriminator, the recognition efficiency of the generator and discriminator is to be analyzed using the loss function of both. Denote $x^{mask}$ according to the input sequence in the input generator. Denote the output sequence of the generator according to $x^{corrupt}$. Then, the loss function of the generator is shown in Equation (5).

$$L_{MLM}\left(x,{\theta }_G\right)=E\left({\sum }_{i\in m}-log{P}_G\left(x_i|x^{mask}\right)\right),$$

(5)

Then, the loss function of the discriminator part of the model is the cross entropy, which is shown in Equation (6).

$$L_D\left(x,{\theta }_D\right)=E\left({\sum }_{t=1}^{n}1\left(x_t^{corrupt}=x_t\right)logD\left(x_t^{corrupt},t\right)+1\left(x_t^{corrupt}\ne x_t\right)log\left(1-D\left(x_t^{corrupt},t\right)\right)\right),$$

(6)

A coefficient λ is added before the loss function of the discriminator and the calculation procedure is shown in Equation (7).

$$\underset{{\theta }_G,{\theta }_D}{\min }{\sum }_{x\in X}{L}_{MLM}\left(x,{\theta }_G\right)+\lambda L_{Disc}\left(x,{\theta }_D\right),$$

(7)

2.2.2. ALBERT Model

The ALBERT model is based on the Transformer bi-directional encoder; a lightweight network model based on the BERT pre-trained language model was developed and the model framework is shown in Figure 3. The model design was based upon the aims of being more lightweight and showing better results, as well as there being faster training. Based on BERT, ALBERT has three improvement points, as follows:

(1) Factorization of embedding parameters. The word embedding layer is mapped into the low-dimensional space for dimensionality reduction, and then mapped into the hidden layer.

(2) The strategy of sharing parameters across layers is used to avoid the increase in the number of parameters with the increase in network depth.

(3) The loss offset problem caused by the BERT next sentence prediction (NSP) task is solved, and the sentence-order prediction (SOP) task is used as the training task.

3. Knowledge Graph and Deep-Learning-Based Q&A Approach for Civil Aviation Travel

3.1. Knowledge Graph Construction of Civil Aviation

The knowledge graph is a semantic network with many nodes and relationships, which can systematically display the knowledge system related to keywords to users. For example, the constructed knowledge graph of civil aviation can make users understand the information about civil aviation more comprehensively and accurately, and also make users spend less time in having a better search experience.

(1) Data acquisition

To construct the knowledge graph of the civil aviation domain required by the Q&A system, a web crawler based on Python’s Scrapy framework was designed to obtain the required civil aviation airline ticket data information from Ctrip.com (accessed on 18 January 2023).

(2) Construction of knowledge graph

The Neo4j graph database was used for the knowledge storage and visualization display of civil aviation knowledge graph. The data were imported into Neo4j through the use of a Load CSV format file. The triads were stored into the Neo4j graph database to form the civil aviation knowledge graph, which can be viewed in graphical form through the browser provided by Neo4j.

3.2. Pre-Processing

Question pre-processing, as the core function module of the Q&A system in this paper, aims to obtain the entity information in the question after the user’s question has been parsed, and at the same time aims to identify the intention of the question to provide the basis for the subsequent query answer. Therefore, this paper divides it into two sub-tasks of entity recognition and intent recognition, i.e., identifying information words in question sentences and judging user intent.

3.2.1. The Entity Recognition Process of ELECTRA-BiLSTM-CRF Model

In this paper, entities in natural language interrogatives correspond to nodes in the knowledge graph, and the accuracy of entity recognition directly affects the process of final answer retrieval, which is one of the two core subtasks. In acquiring the entities in the interrogative sentences, the ELECTRA-BiLSTM-CRF model is constructed in this paper to identify civil aviation entities to ensure an efficient Q&A experience for users. The model consists of three parts: ELECTRA pre-training model, BiLSTM layer and CRF layer, and the model structure diagram is shown in Figure 4. The user’s interrogative sentences are firstly inputted into the model, which are sequentially encoded by the ELECTRA embedding layer for word vectors, then inputted into the BiLSTM layer to obtain the score corresponding to each word in all tags and finally inputted into the CRF layer for feature constraints among tags to obtain the optimal classification choice.

The training process of the proposed entity recognition model is mainly divided into the following steps.

Step 1. Firstly, the sentences inputted in the training set are entered into the word embedding layer for encoding vector representation; then, some of the characters are masked and replaced with special markers according to a certain proportion, identified by [MASK], and then the sentences processed by the markers are entered into the ELECTRA model generator.

Step 2. The predecessor network of the ELECTRA model generator processes the input sequence, and according to the context of the masked [MASK] identifier in the sentence, the keywords that may be composed are predicted, and the words predicted by the generator are replaced with the [MASK] identifier in the original position, that is, substitution is performed, and the sentence that has been predicted and replaced is directly outputted.

Step 3. The replaced sentences are inputted to the discriminator of the ELECTRA model, which discriminates which words in the sentences are to be replaced, and finally obtains the word vector or word vector predicted by the discriminator of the ELECTRA model.

$$h_{ELECTRA}=\left\{{h1}_{ELECTRA},{h2}_{ELECTRA},\cdots ,{hn}_{ELECTRA}\right\},$$

(8)

where $h_{ELECTRA}$ denotes the vector of the i-th word or words.

Step 4. Input the word vector or word vector generated by the ELECTRA model into the BiLSTM module for bidirectional encoding of sequence characters to obtain the predicted labels.

$$h_{LSTM}=\left\{{h1}_{LSTM},{h2}_{LSTM},\cdots ,{hn}_{LSTM}\right\},$$

(9)

where ${hi}_{LSTM}$ denotes the predicted label of the i-th sequence character.

Step 5. The added constraint modules are decoded in the CRF module to obtain the optimal label sequence corresponding to the input sequence, i.e., the final result of entity recognition.

$$P=max\left\{Y\right\}P\left(Y|X\right),$$

(10)

3.2.2. The Intention Classification Process of ALBERT-TextCNN Model

In this paper, an improved ALBERT-TextCNN model is proposed to perform the semantic classification task of interrogative sentences, and the structure is shown in Figure 5. Firstly, the text is encoded using ALBERT, and then adversarial training is added to the model training to improve the robustness and generalization ability of the model, i.e., according to the min–max formula, the basic idea of adversarial training, find the amount of perturbation that can make the model in the sample space. Then, the optimal amount of perturbation that generates the maximum loss is added to the input samples, and then the expected loss on the adversarial sample set is minimized by updating the model parameters. Additionally, the obtained adversarial samples are inputted to the TextCNN model to extract text features, and the key information is extracted through the pooling layer; in order to obtain better text features, the usual three-layer global maximum pooling is changed in the pooling layer to “global average pooling + global maximum pooling + global average pooling”, and the pooling results are spliced. Finally, the fully connected layer is mapped to the category data, the output results are found for the category with the maximum probability and the category results are outputted.

The training process of the civil aviation text classification model based on the improved ALBERT-TextCNN model is mainly divided into the following steps.

Step 1. Obtain the civil aviation text, and input the text to the ALBERT layer through the input layer for serialization operation. The serialized text vector is obtained.

$$l_{ALBERT}=\left\{{l1}_{ALBERT}+{l1}_{ct},{l2}_{ALBERT}+{l2}_{ct},\cdots ,{ln}_{ALBERT}+{ln}_{ct}\right\},$$

(11)

Among them, ${li}_{ALBERT}$ represents the text vector obtained after the i-th character is serialized by ALBERT.

Step 2. By calculating the loss function and gradient of the serialized text vector, the optimal perturbation is obtained and the adversarial samples are obtained.

$$\underset{\theta }{\min }{E}_{(i,y)}~D[\underset{r_{\mathit{adv}}\in S}{\max }L(\theta ,i+r_{adv},y)],$$

(12)

$$g={\nabla }_{{li}_{ALBERT}}L\left(\theta ,{li}_{ALBERT},y\right),$$

(13)

$$r_{adv}=\mathsf{ϵ}g/{‖g‖}_2\left(\theta ,{li}_{ALBERT},y\right),$$

(14)

$$i_{adv}=i+r_{adv},$$

(15)

where $g$ denotes the gradient of the input sample, $y$ is the label of i, $\theta$ is the parameter of the model, L() is the loss function, $r_{adv}$ denotes the amount of perturbation of the input sample, $\mathsf{ϵ}$ is the hyperparameter and $i_{adv}$ denotes the adversarial sample after adding the perturbation.

Step 3. Input the adversarial samples to the multi-layer bidirectional Transformer encoder, and output the text feature vector after training.

$$l_{Transformer}=\left\{{l1}_{Transformer},{l2}_{Transformer},\cdots ,{ln}_{Transformer}\right\},$$

(16)

where ${li}_{Transformer}$ denotes the text feature vector obtained after the i-th adversarial sample has been encoded by a multi-layer bidirectional Transformer.

Step 4. Use the feature vector extracted by the Transformer encoder as the input matrix; the text feature output from the ALBERT layer is trained in the TextCNN convolution layer and pooling layer to obtain the pooled text feature vector.

$$l_{pooling}=\{l_{GA},l_{GM},l_{GA}\},$$

(17)

where $l_{GA}$ denotes the output of global average pooling, $l_{GM}$ denotes the output of global maximum pooling and $l_{pooling}$ denotes the output of the pooling layer.

Step 5. Input the obtained text feature vector to the fully connected layer and complete dropout, keeping the vector dimension unchanged, and output the text label prediction result in the current stage using the activation function.

$$Z=\mathrm{W}\odot \mathrm{f}\mathrm{c}+l_{pooling}+b\odot fc,$$

(18)

$$P=\{p_1,p_2,\cdots ,p_k\},$$

(19)

where $\mathrm{W}\odot \mathrm{f}\mathrm{c}$ and $b\odot fc$ denote the weights and biases of the fully connected layer, $p_i$ denotes the probability of each category and k denotes the number of categories.

Step 6. End the model training process and output the text multi-label classification results.

$$P=max\{p_1,p_2,\cdots ,p_k\},$$

(20)

3.3. Answer Generation

The Cypher statement is the query language of the Neo4j graph database; it is similar to the SQL statement, rich in content and also contains many encapsulated functions. In this paper, through named entity recognition and question classification, it is possible to convert the user input question into a Cypher query statement, and then execute it in Neo4j to obtain the answer that the user needs.

In this article, different Cypher query templates were developed for different types of questions. Taking “When is the arrival time of Xiamen Airlines’ flight MF8233?” as an example, the algorithm determines that the question entity is “Xiamen Airlines’ Flight MF8233”, and then the intent recognition model finds that the intent category of the question is “arrival time”. Finally, we obtain the question triple (Xiamen Airlines’ Flight MF8233, arrival time?), where “?” is the answer entity that needs to be obtained from the knowledge graph. By querying the knowledge graph database using Cypher, the answer to “arrival time of Xiamen Airlines’ flight MF8233” is “16:45:00”, but this structured answer triad is not easy for users to read. Therefore, it needs to be extracted and spliced into the natural language to return it to the user, e.g., “The landing time of Xiamen Airlines’ flight MF8233 is 16:45:00”, so that the user can understand it.

4. Experimental Procedure and Analysis

The accuracy and speed of a Q&A system are the keys to good performance of a question and answer system, and a good question and answer system must be paired with an advanced question and answer algorithm. To evaluate the effect of the ELECTRA-BiLSTM-CRF model for entity recognition, the BERT-BiLSTM-CRF, RoBERTa-BiLSTM-CRF and MacBERT-BiLSTM-CRF models were selected as comparisons. The prediction effect in the text classification of the proposed ALBERT-TextCNN model was evaluated. In this paper, five models of BERT-TextCNN, RoBERTa-TextCNN, MacBERT-TextCNN, ALBERT and TextCNN were designed as comparisons, with the latter two models used for ablation experiments. In addition, according to the analysis of the experimental results, there is a research direction for optimizing performance in the future.

4.1. Entity Identification

4.1.1. Dataset Construction

This experiment used a self-built civil aviation dataset. The civil aviation dataset used the information related to airline tickets on Ctrip.com and the baggage and passenger parts of the document notices issued by airlines as the text corpus. The obtained airline ticket data were filled with question templates to obtain the corresponding sentences. Then, combined with the contents of the baggage and passenger parts in the document notices issued by airlines, we adopted the entity dictionary-based approach for BIO entity annotation to obtain the annotated civil aviation entity recognition data, and of which, 80% of the text corpus was used as the training set and 20% of the text corpus was used as the test set. In this paper, nine different entity labels were predefined, and the entity labels, their label meanings and corresponding examples are shown in

Table 1. Civil aviation entity label table.

Label	Entity Label	Label Meaning and Examples
0	Flight	Flight name—Xiamen Airlines MF8233
1	Modal	Flight type—Boeing 737
2	Type	Flight type—Medium passenger aircraft
3	Delay Rate	Flight delay rate—57%
4	Meal	Flight meals—Snacks
5	Airport	Flight (take-off and landing) airport—Xianyang International Airport
6	Time	Flight (take-off and landing) time—15:45:00
7	Passengers	Flight passengers—pregnant women
8	Luggage	Flight baggage—alcohol

.

4.1.2. Experimental Parameter Settings

The experimental environment was a PC with a Windows 10_x64 operating system, and the specific experimental environment and configuration are shown in

Table 2. Table of experimental environment and configuration.

Category	Configuration
Development Software	PyCharm2022.1
Development Framework	Tensorflow1.14.0
Operating System	Windows10_x64
Processor	Intel(R) Core(TM) i7-10700 CPU @ 2.90 GHz
RAM	8 GB
Development Languages	Python3.7

.

The detailed parameter settings of the model are shown in

Table 3. Parameter settings.

Parameter	Value	Parameter Name
Batch_size	16	Training batches
Lstm_units	128	Number of neural units in long short-term memory network
Epochs	15	Number of training rounds
Learning_rate	5 × 10−5	Learning rate
Num_labels	19	Types of entity label
Max_len	100	Maximum length

.

In this paper, precision, recall and F1 values were used as indicators for judging the effectiveness of the model.

4.1.3. Experimental Results and Analysis

To evaluate the effect of the ELECTRA-BiLSTM-CRF model for entity recognition, the BERT-BiLSTM-CRF, RoBERTa-BiLSTM-CRF and MacBERT-BiLSTM-CRF models were selected as comparisons to verify the entity recognition effect of each model on the civil aviation interrogative dataset, and the experimental results are shown in

Table 4. Comparative experimental results.

	Electra-BiLSTM-CRF	Bert-BiLSTM-CRF	RoBerta-BiLSTM-CRF	MacBert-BiLSTM-CRF
P	0.9901	0.9726	0.9820	0.9861
R	0.9586	0.9395	0.9499	0.9556
F1	0.9761	0.9549	0.9650	0.9698

.

The evaluation metrics and experimental results were combined, and the constructed ELECTRA-BiLSTM-CRF model achieved the best results in all of the metrics. In terms of accuracy, recall and F1 score, the ELECTRA- BiLSTM -CRF model had different degrees of improvement over the BERT-BiLSTM-CRF, RoBERTa-BiLSTM-CRF and MacBERT-BiLSTM-CRF models, which may be because the ELECTRA pre-trained model learns more language features compared with the BERT, RoBERTa and MacBERT pre-training models, and can therefore learn more linguistic features, yielding better performance.

In order to further illustrate the effect of the ELECTRA-BiLSTM-CRF model entity recognition, the recognition effect of the ELECTRA-BiLSTM-CRF model on each class of entities in the civil aviation dataset is demonstrated, and the experimental results are shown in

Table 5. Comparative experimental results for each type of entity.

	P	R	F1
Flight arrival time	0.9668	0.9701	0.9588
Flight delay rate	0.9521	0.9634	0.9602
Passenger problem	0.9193	0.9057	0.9126
Aircraft type	0.9489	0.9422	0.9397
Flight type	0.9503	0.9525	0.9517
Baggage problem	0.9052	0.9073	0.9108
Flight departure time	0.9685	0.9647	0.9652
Flight departure airport	0.9659	0.9681	0.9667
Flight landing airport	0.9589	0.9641	0.9650

.

The evaluation indexes and experimental results were combined. The ELECTRA-BiLSTM-CRF model achieves relatively good results in terms of accuracy, recall and F1 scores, with all of the indexes above 90% for all kinds of problems, and can well meet the requirements of the Q&A system in the entity recognition module. Therefore, with more sufficient data samples, the ELECTRA-BiLSTM-CRF model can complete the entity recognition task well and achieve better results.

In addition, overfitting may occur when the ELECTRA-BiLSTM-CRF model is trained. We believe that a dropout layer or other method should be added to prevent overfitting and to effectively alleviate the occurrence of overfitting.

4.2. Intention Recognition

4.2.1. Dataset Construction

In this paper, a corpus dataset of civil aviation interrogative sentences was constructed via manual collection, 20,693 interrogative sentences were constructed and the questions were classified and organized, and the categories and classification of civil aviation interrogative sentences are shown in

Table 6. Problem category.

Label	Type	Examples of Questions
0	Flight arrival time	When will China Southern Airlines’ flight CZ6711 arrive?
1	Flight delay rate	Will Southern Airlines’ flight CZ6711 arrive on time?
2	Passenger problem	Can additional passengers travel by plane alone?
3	Aircraft type	What is the aircraft used by Southern Airlines’ flight CZ6711?
4	Flight type	What is the flight type used by Southern Airlines’ flight CZ6711?
5	Baggage problem	Can I bring a small animal checked as baggage into the cabin?
6	Flight departure time	When does Southern Airlines’ flight CZ6711 take off?
7	Flight departure airport	Which airport does Southern Airlines’ flight CZ6711 take off from?
8	Flight landing airport	Which airport did China Southern Airlines’ flight CZ6711 land at?
9	Flight meals	What type of meals do Southern Airlines’ flight CZ6711 offer?

, with 10 categories of questions.

Due to the complex diversity in the travelers’ questions and the fact that the accuracy of the question and sentence classification is limited by the quality and quantity of the dataset, it is considered that the dataset needs to be expanded. The 10 types of questions were manually constructed as a small-scale dataset to be used as a seed set, and then the dataset was expanded via synonym replacement, sentence reconstruction and entity replacement. When a dataset corpus is expanded, since the seed dataset has already been labeled, the final data are all part of the corpus with category labels, which can reduce the time cost of manual labeling while ensuring data quality. After the expansion, a total of 20,693 civil aviation interrogative datasets were constructed. To ensure a balanced sample size of different question categories, the data of each question category were divided into training and test sets in a ratio of 8:2.

4.2.2. Experimental Parameter Settings

The TensorFlow framework was chosen for the experiments to build the ALBERT-TextCNN model. The specific experimental environment and configuration were consistent with the experiments in the previous section, and the detailed parameter settings of the model are shown in

Table 7. Initial parameter settings.

Parameter	Value	Parameter Name
Batch_size	16	Training batches
Filter_size	(2,3,4)	Convolution kernel size
Epochs	10	Number of training rounds
Learning_rate	5 × 10−5	Learning rate
Hidden_size	256	Hidden layer size
Dropout Max_len	0.1 20	Loss rate Maximum length

.

4.2.3. Experimental Results and Analysis

To evaluate the prediction effect of the proposed ALBERT-TextCNN model in text classification, five models, BERT-TextCNN, RoBERTa-TextCNN, MacBERT-TextCNN, ALBERT and TextCNN, were designed as comparisons in this paper to verify the classification accuracy of each model in the civil aviation interrogative dataset, and the experimental results are shown in

Table 8. Comparative experimental results of civil aviation dataset.

	P	R	F1
ALBERT-TextCNN	0.9538	0.9540	0.9581
BERT-TextCNN	0.9326	0.9295	0.9349
RoBERTa-TextCNN	0.9420	0.9399	0.9450
MacBERT-TextCNN	0.9417	0.9372	0.9430
ALBERT	0.9372	0.9304	0.9371
TextCNN	0.9109	0.9237	0.9299

.

The evaluation metrics and experimental results were considered. The ALBERT-TextCNN model achieves the best results in all of the metrics. In terms of accuracy, recall and F1 score, the ALBERT-TextCNN model has different degrees of improvement over the BERT-TextCNN, RoBERTa-TextCNN and MacBERT-TextCNN models, which may be attributed to the fact that the ALBERT pre-trained model has better performance compared to the BERT, RoBERTa and MacBERT pre-training models in learning more linguistic features and therefore yielding better performance. Additionally, with the ablation experiments—using ALBERT or TextCNN alone—it was shown that the pre-trained model learned the universal features of the language better and TextCNN could extract the text features better; thus, the combination of both can achieve better results.

To further illustrate the predictive effect of the ALBERT-TextCNN model in text classification, the following results also show the classification effect of the ALBERT-TextCNN model on each class of questions in the Civilian Interrogative dataset, and the experimental results are shown in

Table 9. Comparative experimental results of single class problems.

	P	R	F1
Flight arrival time	0.9668	0.9701	0.9588
Flight delay rate	0.9521	0.9634	0.9602
Passenger problem	0.9193	0.9057	0.9126
Aircraft type	0.9489	0.9422	0.9397
Flight type	0.9503	0.9525	0.9517
Baggage problem	0.9052	0.9073	0.9108
Flight departure time	0.9627	0.9584	0.9618
Flight departure airport	0.9685	0.9647	0.9652
Flight landing airport	0.9659	0.9681	0.9667
Flight meals	0.9589	0.9641	0.9650

.

The evaluation indexes and experimental results were combined, and from the accuracy rate, recall rate and F1 score, it can be seen that the ALBERT-TextCNN model achieves better results, with all of the indexes above 90%, on all kinds of problems, and can well meet the requirements of the Q&A system in the intention recognition module. Therefore, with more sufficient data samples, the ALBERT-TextCNN model in this paper can complete the text classification task well and achieve better results.

In deep learning training, there is a large amount of computation, storage and other consumption during training, which has large cost consumption, and in the future, research can be conducted to reduce the consumption and shorten the training time.

4.3. Q&A Matching

After identifying the entities in the query using the entity recognition algorithm, the first two constituent elements of the triad were obtained by extracting the relational attributes in the query using the intent recognition algorithm. Since the civil aviation knowledge graph is stored in the Neo4j graph database, the Cypher query language was used to run queries in the knowledge graph. Once the entities and attributes had been identified, Cypher query statements could be constructed based on them to find the corresponding attribute values and obtain the answers through the knowledge graph. An example of the answer query process for the interrogative statement is shown in

Table 10. Example of the answer query process of questions.

Question Input	Construct a Triplet	Cypher Query Statement	Answer
When is the arrival time of Xiamen Airlines’ flight MF8233?	(Xiamen Airlines’ flight MF8233, arrival time?)	MATCH (n:Entity{name: “Xiamen Airlines’ flight MF8233”})-[r:relation{name: “Xiamen Airlines’ flight MF8233”}]->(p:Node) returnp	18:30:00
Will Air China CA4335 arrive on time?	(Air China CA4335, flight delay rate?)	MATCH (n:Entity{name: “Air China CA4335”})-[r:relation{name: “Flight delay rate”}]->(p:Node) returnp	83%

.

5. Conclusions

In summary, a civil aviation travel Q&A approach was proposed by incorporating knowledge graphs and deep learning techniques in this paper. Based on building a knowledge graph for civil aviation travel, a constructed ELECTRA-BiLSTM-CRF model was used to extract question entities for the specificity of civil-aviation-related data and then an improved ALBERT-TextCNN model was used to realize the classification of question intent. Cypher query was used to retrieve answers to question triples, and the answer templates were combined into natural language statement outputs. The experimental results show that the model and method proposed in this paper can achieve good results and prove their effectiveness. The working of the proposed approach mainly includes three aspects.

(1) Crawl text data about civil aviation air tickets on Ctrip.com using crawlers, define templates, transform information into question-and-answer pairs and integrate them to obtain the air ticket interrogation dataset and the question-and-answer dataset.

(2) Construct a knowledge graph of civil aviation travel based on the dataset to provide a high-quality database for the Q&A system, and use a Neo4j graph database to store the knowledge graph.

(3) Based on the construction of the knowledge graph of civil aviation travel, design the question-and-answer system algorithms, construct the ELECTRA-BiLSTM-CRF model for the named entity recognition task and adopt the improved ALBERT-TextCNN model for the intention recognition task.

In the future, we will make efforts to solve the overfitting problem that may occur in the ELECTRA-BiLSTM-CRF model, reduce the cost of the ALBERT-TextCNN model and improve the running speed. In addition, the generalizability of the algorithm in this paper is low; its results are optimal in the civil aviation dataset, but not in other datasets, and the algorithm can be improved in the future to enhance its generalizability.