Special Issue on “Natural Language Processing: Emerging Neural Approaches and Applications”

Nowadays, systems based on artificial intelligence are being developed, leading to impressive achievements in a variety of complex cognitive tasks, matching or even beating humans [1,2,3,4]. Natural language processing (NLP) is a field where the use of deep learning (DL) models in the last five years has allowed AI to advance toward human levels in translation and reading comprehension, as well as other real-world NLP applications, such as question answering and conversational systems, information retrieval, sentiment analysis, and recommender systems.

However, due to the difficulties associated with natural language understanding and generation, which are human capabilities among the least understood by computer systems from a cognitive perspective, and despite the remarkable success of DL in different NLP tasks, this is still a field of research of increasing interest [5,6,7]. In order to improve DL methods, current models have been scaled up, but their complexity has grown toward directions assumed by empirical engineering solutions [8,9,10,11]. Moreover, they are not applicable to languages without extensive datasets [12], and the lack of explainability inhibits further improvements [13].

This Special Issue highlights the most recent research being carried out in the NLP field to discuss these open issues, with a particular focus on both emerging approaches for language learning, understanding, production, and grounding interactively or autonomously from data in cognitive and neural systems, as well as on their potential or real applications in different domains.

There are 30 contributions selected for this Special Issue representing progress and potential applications in the NLP area from original contributions of researchers with a broad expertise in various fields: NLP, cognitive science and psychology, artificial intelligence and neural networks, computational modeling and neuroscience covering the whole range of theoretical and practical aspects, technologies, and systems.

This collection includes one review paper, which focuses on text corpus-based tourism big data mining [14]. Li et al. summarized and discussed different text representation strategies, text-based NLP techniques for topic extraction, text classification, sentiment analysis, and text clustering in the context of tourism text mining, as well as their applications in tourist profiling, destination image analysis, and market demand, among others. Their work also provides guidelines for constructing new tourism big data applications and outlines promising research areas in this field for the coming years.

One letter is also included in this issue, employing evolutionary a neural architecture search for Korean grammaticality tasks [15].

Regarding the other 28 research papers, the following NLP areas are specifically addressed:

Natural language understanding, generation, and grounding: In [16], OntologyFixer is presented, a web-based tool that supports a methodology to build, assess, and improve the quality of Ontology Web Language (OWL) ontologies. Another paper [17] addresses the problem of paraphrase identification and presents an approach for leveraging contextual features with a neural-based learning model based on lexical, syntactic, and sentential encodings, incorporating relational graph convolutional networks (R-GCNs) to make use of different features from local contexts (e.g., word encoding, position encoding, and full dependency structures). In addition, in [18], the authors revisited the recurrent neural network (RNN) language model, achieving highly competitive results with the appropriate network structure and hyperparameters.

Universal language models: In [19], Javaloy and the co-author used a method recently proposed, called the causal feature extractor (CFE), for encoder-decoder models on different text processing tasks. The same authors applied this method to text normalization in [20], which is a ubiquitous problem that appears as the first step of many text-to-speech (TTS) systems.

Conversational systems or interfaces and question answering: The authors in [21] proposed the best practices for question classification in different languages using convolutional neural networks (CNNs), finding the optimal settings depending on the language and validating their transferability. The authors in [22] addressed the time-consuming development of manual user simulator policy and introduced a multi-agent dialogue model, where an end-to-end dialogue manager and a user simulator are optimized simultaneously for dialogue management by cooperative multi-agent reinforcement learning. Moreover, in [23], the authors proposed a Medical Instructed Real-time Assistant (MIRA) that listens to the user’s chief complaint and predicts a specific disease, thus referring the user to a nearby appropriate medical specialist. Furthermore, in [24], the authors presented a multi-turn chatbot model in which the preceding utterances are exploited in response generation by using different weights.

Sentiment analysis, emotion detection, and opinion mining: The study in [25] investigated a comparison of various DL models used to identify the toxic comments in Internet discussions. Moreover, in [26], the authors proposed a novel hybrid model XGA (namely an XLNet-based bidirectional gated recurrent unit (BiGRU) network with an attention mechanism) for Cantonese rumor detection on Twitter, taking advantage of both semantic and sentiment features for detection. Furthermore, the authors of [27] proposed an intensive study regarding a domain-independent classification model for sentiment analysis using neural models, showing high performance when using different evaluation metrics compared with the state-of-the-art results. Another study in [28] tested different approaches for handling long documents and proposed a novel technique for sentiment enrichment of the Bidirectional Encoder Representations from Transformers (BERT) model as an intermediate training step. In [29], Rizkallah et al. proposed an embedding approach that is designed to capture the polarity issue for sentiment analysis.

Document analysis, information extraction, and text mining: In [30], Ronran et al. evaluated the combination of different types of embedding features in a bidirectional long short-term memory (Bi-LSTM) conditional random field (CRF) model for named entity recognition (NER). The authors in [31] investigated the transferability of the features from an open information extraction (OIE) domain to another and applied the approach for relation extraction (RE). The authors in [32] proposed a rule-based approach for text document classification. The study in [33] proposed an RE model based on a dual pointer network with a multi-head attention mechanism to address the association of multiple entities in a sentence according to various relations. The work in [34] investigated an RE method to solve the possible overlapping among multiple relational triples contained in a sentence. Another topic was introduced by the authors of [35], who introduced a novel hybrid model of extractive-abstractive text summarization to combine BERT word embedding with reinforcement learning. Two contributions to this special issue are focused on medical information extraction. The authors in [36] compared different architectures of DL models, including CNNs, LSTM, and hybrid models. Furthermore, they proposed a hybrid architecture for protein–protein interaction extraction from the biomedical literature. The authors in [37] developed a multitask attention-based Bi-LSTM–CRF model with pre-trained embeddings from language models (ELMo) in order to achieve improved performance in clinical NER.

Search and information retrieval: In [38], Boban et al. adapted language modeling-based methods for sentence retrieval to test the partial matching of terms through combining sentence retrieval with sequence similarity. This method allows for matching words that are similar but not identical. The authors of [39] proposed a reliable sentence classification model based on an encoder-decoder neural network to resolve lexical disagreement problems between queries and frequently asked questions (FAQs).

Trustworthy and explainable artificial intelligence: Two contributions [40,41] considered “sememe”, the smallest semantic unit for describing real-world concepts, which improve the interpretability of NLP systems. In particular, the study in [40] proposed a novel model to improve the performance of sememe prediction by introducing synonyms. On the other hand, the work in [41] implicitly synthesized the structural features of sememes into word embedding models through an attention mechanism. The work proposes a novel double attention word-based embedding (DAWE) model that encodes the characteristics of sememes into words with a “double attention” strategy.

Applications in science, engineering, medicine, healthcare, finance, business, law, education, transportation, retailing, telecommunication, and multimedia: The authors in [42] proposed a hybrid adversarial attack method to generate examples with the aim to explore the vulnerabilities and security aspects of deep learning systems in different application scenarios. An application in programming education was considered in [43]. In this study, the source code assessment and its classification were developed by a sequential language model that used an attention mechanism through an LSTM neural network and based on the estimated error probability.

In summary, this Special Issue contains a series of excellent research works on NLP, covering a wide range of topics. The collection of 30 contributions is highly recommended, and it will benefit readers in various aspects.