Big Data Management in Drug–Drug Interaction: A Modern Deep Learning Approach for Smart Healthcare

Abstract

The detection and classification of drug–drug interactions (DDI) from existing data are of high importance because recent reports show that DDIs are among the major causes of hospital-acquired conditions and readmissions and are also necessary for smart healthcare. Therefore, to avoid adverse drug interactions, it is necessary to have an up-to-date knowledge of DDIs. This knowledge could be extracted by applying text-processing techniques to the medical literature published in the form of ‘Big Data’ because, whenever a drug interaction is investigated, it is typically reported and published in healthcare and clinical pharmacology journals. However, it is crucial to automate the extraction of the interactions taking place between drugs because the medical literature is being published in immense volumes, and it is impossible for healthcare professionals to read and collect all of the investigated DDI reports from these Big Data. To avoid this time-consuming procedure, the Information Extraction (IE) and Relationship Extraction (RE) techniques that have been studied in depth in Natural Language Processing (NLP) could be very promising. Since 2011, a lot of research has been reported in this particular area, and there are many approaches that have been implemented that can also be applied to biomedical texts to extract DDI-related information. A benchmark corpus is also publicly available for the advancement of DDI extraction tasks. The current state-of-the-art implementations for extracting DDIs from biomedical texts has employed Support Vector Machines (SVM) or other machine learning methods that work on manually defined features and that might be the cause of the low precision and recall that have been achieved in this domain so far. Modern deep learning techniques have also been applied for the automatic extraction of DDIs from the scientific literature and have proven to be very promising for the advancement of DDI extraction tasks. As such, it is pertinent to investigate deep learning techniques for the extraction and classification of DDIs in order for them to be used in the smart healthcare domain. We proposed a deep neural network-based method (SEV-DDI: Severity-Drug–Drug Interaction) with some further-integrated units/layers to achieve higher precision and accuracy. After successfully outperforming other methods in the DDI classification task, we moved a step further and utilized the methods in a sentiment analysis task to investigate the severity of an interaction. The ability to determine the severity of a DDI will be very helpful for clinical decision support systems in making more accurate and informed decisions, ensuring the safety of the patients.

1. Introduction

In the last couple of decades, biomedical and smart healthcare research has witnessed a rapid amount of growth in terms of its presence in the literature, novel discoveries, and computational approaches, due to which a huge amount of experimental data has been generated and published (i.e., Big Data) to validate and to describe these innovations [1]. The amount of data is so large that it is impossible for a human being to analyze and read the articles related to their desired field, and even the simple extraction of field-related published articles has become nearly impossible. As such, to utilize the information from scientific publications and articles, the increased use of automation due to the diversification and explosive growth in the healthcare literature as well as in the pharmaceutical industry is a clear representation of this growth. This is why attention has now been diverted and focused on the implementation and evolution of tools for knowledge-based discovery: the tremendous amount of biomedical research in the literature requires automation to extract, represent, interpret, and maintain this information in a refined manner.

Globally, there are different organizations that are working in the healthcare sector, with some examples being the World Health Organization (WHO), European Environment and Health Committee (EEHC), and Pan American Health Organization (PAHO). The motive of these organizations is to ensure patient safety, which is considered to be the of the highest priority, and, to achieve this purpose, these organizations are actively participating in taking legislative measures to control the adverse effects that can be avoided through the use of proper strategies and plans.

The Institute of Medicine (IOM) in the United States determined [2,3] that, annually, hundreds of thousands of patients die in hospitals due to quality of care issues, such as medication errors, lack of cleanliness, hospital-acquired infection, obstetrics, and DDIs that cause adverse drug reactions (ADRs) [4,5,6,7,8]. Though there are well-established systems that are in place to maintain patient safety, the field of drug interactions require further research [9] because recent reports indicate that drug reactions are one of a major cause of hospital-acquired conditions and readmissions [10,11,12,13,14]. Pharmacovigilance, which was initiated by the WHO, is gaining more attention, with the literature showing that almost one tenth of the adverse reactions seen in ICU subjects are due to DDIs causing ADRs [15,16]. Different evaluation and automation systems are being used for ADR detection in different medicines, but, in most cases, reports of ADRs are investigated and revealed by the healthcare professionals quite a bit later.

The motive of this research is to create new strategies in already-working systems for the detection of DDIs from the medical literature, considering ‘big data’. By definition, one drug may decrease or increase the effect of another because of its mutual chemical formulation, a phenomenon known as a DDI [17,18,19]. Information on DDIs is highly important and relevant for restricting the failures of therapeutic treatments and to prevent other strong ADRs. Though DDIs could be very dangerous, this is not true in all cases, and these interactions can be effective in certain cases where they provide a desirable synergistic effect [20]. Therefore, we are interested in determining the severity of a DDI by processing the same text used for the detection of DDIs. This aspect will help clinical decision support systems in making more accurate and informed decisions by ensuring the safety of the patients. This would not only save the lives of humans, but it would also result in a huge decrease in healthcare costs as well.

DDI has an immense impact on patients’ safety and therefore it needs to be tackled seriously. There has been a rapid increase in the development of novel technologies and expert systems in the domain of healthcare and medicine. With the passage of time, the need for integrated approaches and evidence-based healthcare is increasing at a very high rate. On the flip side, size as well as heterogeneity and complexity of data generated by several sources are a big challenge to the computational approaches [21,22,23,24]. Making sense of huge biomedical text is a challenge for healthcare research [25]. Unstructured text constitutes the most important form of data in the healthcare and medical domains. Though this form of data is very difficult to process, but it contains many elaborations, nuances, and details that could not be captured in the thesaurus of nomenclature, but this information could be very useful in decision-making [26]. Dealing with this growing unstructured information requires machine learning techniques and methods in order to bring structure and extract meaningful insights [27].

There are many specialized databases (i.e., Drug Bank, Micromedex, MEDLINE) which contain known DDIs. These DDI pairs are limited in coverage as new discoveries are made and published as research articles. Consequently, there is an ultimate need for extraction of newly discovered DDI in the scientific literature [28,29]. Text mining is known as one of the best available techniques that can be applied to achieve automatic relation extraction from the published literature and articles. Text mining is being used widely for relation extraction, for instance protein–protein interactions (PPIs) and gene–disease relationships, and therefore a corpus [30] is formed specifically for the DDI extraction task [31,32,33]. The DDI extraction task is very similar to relationship extraction studied in depth in the text-mining domain as shown in Figure 1. This figure presents the working of the relationship extraction task with the demonstration of practical examples. In the processed sentence, each drug is labelled initially, such as subject and object. After obtaining pairs of drugs, the relationship among the drugs is investigated and if a pair of drugs is involved in increasing or decreasing the effect of the other, it would be classified as a positive DDI pair, as shown with solid lines in the presented example. While negative DDI pairs (i.e., drugs having no effect on each other) are shown with dotted lines.

In Figure 1, the complex structure of DDIs is presented with two practical real examples including clauses of positive and negative DDIs represented with solid and dotted lines, respectively.

(a)

Examples of positive DDIs with multiple subjects (drugs) to one object. In this scenario, subjects do not have any interaction with each other but only with the object.

(b)

Represents one drug as a subject and two drugs as objects. In this case, there is surely a connection between the subject and object individually and to the cluster of objects, but no interaction between object drugs.

2. Literature Review

In this section, we will discuss the recent research work regarding different perspectives of the DDI extraction task, including the challenges, different aspects, and strategies to extracting DDIs, as explored by researchers in recent years. Secondly, we discuss deep learning in the domain of DDI extraction and classification in Section 2.3, exploring the employment of the very few works of sentiment analysis within the bio-medical domain.

2.1. Drug Interaction Task

Due to the developments in biomedicine, there are numerous new terms evolving, and the linkage between them is of high importance. As these relations are typically mentioned in the biomedical text which includes immense volumes and is growing at a very high pace [34], this area is receiving huge attraction from the scientific community. The DDI extraction task did not receive as much attention as the gene–gene interaction and protein–protein interactions received. There is much less work in the automatic extraction of DDIs, and initially this task was tackled by [35,36], but this work was not appropriately mined, meaning that it used outdated approaches in relevance to the modern approaches that exist today in the smart health care domain.

The rise in this domain is due to the work of Herrero Zazo, M. et al. [22], who not only proposed techniques in the automatic extraction of DDIs but also provided a standardized annotated corpus to motivate the researchers in this particular task. This DDI corpus was created by [37] in 2013 and includes 18,502 manually annotated pharmacological substances as well as 5028 DDI pairs. The availability of the corpus comes with challenges. Various methods have been proposed for relation extraction from the medical literature and machine learning (ML)-based techniques remain dominant in building such systems [38]. These systems are mainly classification-based and every drug–drug pair is classified into a possible case of interaction or otherwise. In doing so, language pre-processing is heavily used for transforming text data into a structured representation.

Under SemEval-2013 (Semantic Evaluation task), an open challenge was announced to extract and classify drug names. Many researchers accepted this challenge and implemented different methods and approaches, but the best approach was proved to be CRF-based learning [39], where “Chemical Entities of Biomedical Interest” known as ChEBI were used and achieved an F-measure of 0.57 for the overall dataset. It is worthwhile to mention that the challenge targeted a 0.71 score against the F-measure [40]. Similarly, another competition was held for DDI extraction in 2013 to address the DDI extraction issues. Against the DDIExtraction-2013 challenge, there are four most prominent submissions. Among these, the Support Vector Machine (SVM-)-based method outperformed other approaches [41,42,43]. These methods mainly involve manual features which work on word level and are quite difficult to handle for long sentences with complex relationship representation as the MEDLINE-2013 corpus contains.

2.2. Deep Learning

From the last few years, in contrast to SVM-based methods, by evolving deep learning [44], neural nets-based methods such as Convolutional Neural Networks (CNN) or Recurrent Neural Networks (RNN) are used for different classification tasks and are quite robust and versatile. CNNs and RNNs are not bound to manual features for extraction and therefore have higher performance in many NLP tasks such as for sentiment analysis [45], search query retrieval [46], and fast and accurate semantic parsing [47]. CNN-based methods have also been tried for the DDI extraction task [48,49,50,51]. These methods performed well relative to SVM-based models; however, the performance of the systems was not satisfactory with the ML-2013 dataset. The reason behind unsatisfactory results of CNNs on ML-2013 is that it contains long and complex structures of sentences where the position of the relationship containing words is not fixed (i.e., before, after or middle of the drugs). Therefore, some meaningful words are ignored in long sentences which eventually makes CNNs weak in this context.

To overcome this problem, RNN-based systems [52,53] are used for relationship classification. In these systems, the mechanism of the long short-term memory (LSTM) model is combined which works on memory units (MUs). In LSTM, all the contextual information of words, whether consecutive or not, can easily be emulated in the MUs regardless of the long-distance sentences. This procedure follows a sequence to process the words of a sentence in an RNN, there is a slight chance of losing information for dependency of relations in long-distance patterns. This issue of RNN also makes it a biased network because of dominating later words in the processing of long patterns such as in ML-2013. To mitigate this issue, mechanism of ‘attention’ is exploited where the weighting method is included to assign a score to the text segments. In this way, high-scoring segments are considered to be the more influential part of the sentence and receive relatively more attention from the network. An attention-based layer mechanism is applied quite effectively and achieved better results in [54,55,56] when employed in different machine translation and question answering tasks. These RNN-based methods [57,58,59] are applied on the DDI-Extraction task and the results are 2–3% better than the existing baseline methods using SVM and neural nets but still low in the recall. To overcome the biasness issue of RNNs, we propose an LSTM-based RNN model with two distinct additional layers, i.e., Bottom RNN (RNNB) and Top RNN (RNNT). The detailed working of this method (SEV_DDI) is discussed in Section 3.3.

2.3. Severity Prediction in Drug–Drug Interactions

The sentiment analysis task is an established area of research, typically applied on opinionated text collected from social media. The ‘sentiment’ can reflect the seriousness, severity or the certainty of an interaction reported in medical text. In the biomedical domain, to the best of our knowledge, sentiment analysis is applied in very few aspects. Working on the healthcare literature with respect to sentiment analysis, the authors of [60] investigated and classified the polarity at sentence level using multiple linguistic features of text processing, i.e., n-grams. In this work, Y. Niu et al. [61] exploit a Unified Medical Language System (UMLS) to extract the context and category information of medical terms. For the purpose of sentiment extraction from medical text, a multi-step mechanism was designed, where the topics or subjects were classified initially and then the polarity of each subject was investigated [62] which shows 10% better performance.

Working within the domain of potential drug–drug interactions, the authors of [63] have reviewed 92 published papers and almost a dozen interviews to extract 56 different aspects of the knowledge for the interpretation of DDIs. In this review, the author identified the advancements and strengths of expert systems along with the limitations and weak sides of these expert systems. Severity extraction of DDIs is rarely studied, so we also propose severity extraction method using the polarity of words describing the interaction. The DDI severity extraction task did not receive much attention but could be very promising for the advancements of clinical decision support systems. Instead of merely extracting the DDI event, severity or significance of the drug’s interaction is very critical. Consequently, we designed a mechanism which extracts the severity of classified DDI as well.

3. Methods and Implementation

We propose a deep neural network-based model with a hierarchal mechanism to extract DDIs and severity from text sentences which are collected from the biomedical literature (i.e., publications) and processed by applying NLP techniques on them as shown in Figure 2. The model uses RNN with an addition of hierarchal LSTM layers.

A lexicon-based method is also designed to investigate the severity of interaction between the DDIs. In this section, we illustrate the steps involved in the RNN-based model with a hierarchal mechanism.

3.1. Pre-Processing

Common pre-processing and text cleaning operations are performed on sentences including but not limited to lemmatization. Each drug mentioned in a sentence is labelled and considered to interact with other drugs. The number of drug pairs (DP) in a sentence could be estimated by Equation (1).

$$Drug Pairs \left(DP\right)=max\left(0, {\displaystyle \sum }_{i=1}^n\left(i-1\right)\right)$$

(1)

where n is number of drugs in a sentence.

Drug blinding is also applied, where each and every drug name is assigned with a label as performed in [64], e.g., for a sentence, “Aspirin may decrease the effects of probenecid”, the labelled sentence is “Drug^A may decrease the effects of Drug^B”. The drug blinding technique helps a model to recognize these labels as ‘subject’ and ‘object’ which eventually helps a model in classification. These processed sentences are then provided to the model for detection and classification of DDIs.

3.2. Recurrent Neural Network Architecture

Our proposed recurrent neural network model, with hierarchal long short-term memory (LSTM), includes five layers. These layers include ‘word embedding’, ‘attention’, ‘dropout’, ‘bottom RNN’ and ‘top RNN’ as shown in Figure 3. The working and purpose of each layer in the purposed model are discussed in the following.

Word Embedding Layer

In this layer, each word is transformed into a real valued vector. This mapping of words into the matrix is performed by using already available Word2Vec [65] embedding information [66] using the abstracts of PubMed containing the drugs [67]. Each sentence is pre-processed and constitutes ‘si’ and ‘dj’ where dj is a drug label and si is any other word in this particular sentence. Each word ‘si’ is converted into the word vector using the word embedding matrix with the following formula. WEMB is an embedding matrix and WEMB ∈ ${ℝ}^{d_s\times \left|V\right|}$, where ‘d_s’ is the number of dimensions in word embedding and ‘V’ is the vocabulary in the training data set, $v_i^s$ in Equation (2) represents the index of word embedding.

$$s_i^{\to }=W_{EMB}·v_i^s$$

(2)

Attention Layer

The words between and around the subject and object are quite important in detection of whether the interaction or effect of the subject is influencing the object. Therefore, not all but a few of these significant words (e.g., increase, decrease, effect, impact, careful monitoring, recommendation, prohibition, etc.) are of great value in the detection of interaction between the drugs. Therefore, we are using an attention layer after embedding the words into vectors because the attention layer is dominantly working on a mechanism of learning in sentence to sentence. Similarly, it has been applied to the task of relationship extraction [68]. Therefore, this mechanism is also applied in our model as applied in [69] with the purpose of providing more refined embedded vector information to the network.

Dropout Layer

Machine learning-based algorithms tend to over-fit while processing huge amounts of data with so many dimensions. The ‘Dropout’ technique was initiated by [70], which prevents a model from overfitting by dropping the input values which can cause this problem. Therefore, the inner layer processes most of the distinct inputs. To avoid this model from overfitting, we applied this strategy in Gated Recurrent Unit (GRU) dropping with probability at both embedding and neural network training by multiple test values, and the best results are reported while tuning the hyper-parameters at 0.7 and 0.5 for embedding and dropout layers, respectively.

Bottom and Top RNNs Layers

Two RNN-based layers are used to overcome the biasness issue of RNNs with LSTM. The top and bottom layers are applied separately (like the working of Bidirectional LSTM) to produce a score from both ends and then merged up to produce a final score of the words with a zero-bias problem. RNNs can process any kind of sentence pattern through their GRU where each word ‘si’ of a sentence ‘S’ is taken as input along with a previous hidden state ‘hi-1’. The LSTM network [71] was introduced to mitigate the issue of exponential growth and decay of gradient vectors in long patterns, which showed better results [72]. LSTM is also known to be a biased network when it is applied along with RNN because, in this network dominance, initial words of a sentence are ignored and later words gradually became more dominant than the initial ones [69].

To overcome this bias, we designed a dual-layer mechanism, where LSTM is initiated from both ends of a sentence to capture its contextual information along with global semantic information. While processing a sentence Si = {s1, s2, s3, s4, … sn}, we two encoded matrices as shown below in Equations (3) and (4). At the end, output vectors of both LSTMs are concatenated in Equation (5) to obtain a biasfree contextual and sematic information of a sentence.

$$H_{S^1}^{\to }=\left\{h_1^{\to },h_2^{\to },h_3^{\to },\dots ,h_{n-1}^{\to },h_n^{\to }\right\}$$

(3)

$$H_{S^1}^{\leftarrow }=\left\{h_n^{\leftarrow },h_{n-1}^{\leftarrow },h_{n-2}^{\leftarrow },\dots ,h_2^{\leftarrow },h_1^{\leftarrow }\right\}$$

(4)

thus,

$$H=\left\{H_{S^1}^{\to }\parallel H_{S^1}^{\leftarrow }\right\}\approx \left\{h_1^{\to }\parallel h_1^{\leftarrow },h_2^{\to }\parallel h_2^{\leftarrow },h_3^{\to }\parallel h_3^{\leftarrow },\dots ,h_n^{\to }\parallel h_n^{\leftarrow }\right\}$$

(5)

Each ‘true’ drug pair is categorized among one of the four types, i.e., mechanism, effect, int and advice. The description of drug type along with their correspondent examples is illustrated in

Table 1. Description of DDI types in DDIExtraction-2013.

DDI Type	Description	Sentence
Int	The interaction is reported in a sentence but detailed information about the interaction is not provided.	Concomitant use of alcohol with phentermine hydrochloride may result in an adverse drug interaction.
Advise	For a pair of drugs, there are some recommendations about its usage.	Scopolamine should be used with care in patients taking other drugs that are capable of causing CNS effects such as sedatives, tranquilizers, or alcohol.
Mechanism	Between a pair of drugs, there is a pharmacokinetic mechanism.	Penicillin blood levels may be prolonged by concurrent administration of probenecid which blocks the renal tubular secretion of penicillins.
Effect	Between a pair of drugs, an effect is reported in the sentence which could be positive or negative.	Other HDAC Inhibitors. Severe thrombocytopenia and gastrointestinal bleeding have been reported with concomitant use of ZOLINZA and other HDAC inhibitors (e.g., valproic acid).
False	There is no interaction between the mentioned two drugs.	The in vitro binding of warfarin to human plasma proteins is unaffected by tolmetin, and tolmetin does not alter the prothrombin time of normal volunteers.

. All the sentence examples are taken from the DDIExtraction-2013 dataset.

3.3. Severity Extraction Method

We are interested in only those sentences for which at least one DDI instance is ‘true’. After classifying the interactive drug pairs, DDI reporting sentences are extracted from the annotated corpus [22]. As mentioned in Figure 2, the extracted sentences are then pre-processed to calculate the sentiments, i.e., polarity of words. In pre-processing, where drugs’ names (i.e., nouns) are removed from the concerned text because drugs’ names merely contain polarity nouns and could not be effective in polarity rating. We refined the text by retaining only verbs; adverbs and adjectives and nouns are removed to prevent our system from conducting extra processing.

Lexicons such as Sent WordNet [62] and WordNet Affect [64] are general lexicons, used to extract general sentiments of text, i.e., movies and social reviews. The subjectivity lexicon [70] is used to extract the subjective expression from arguments or text statements. Many of the general and subjectivity lexicons were adapted in medical research for different healthcare tasks. An extended pharmaceutical lexicon is also evolved specifically for the healthcare and biomedical domains, which is used to extract the sentiments of pharmaceutical and clinical text.

We extracted the polarity of the sentences by applying Sent WordNet, and interaction is categorized at low, moderate, or high level for dangerous as well as beneficial DDIs based on the polarity of the candidate sentence. The analysis of the outcomes of severity is shown in Section 4.1.

4. Data, Experimental Parameters, and Results

DDIExtraction-2013 corpus [22] which is constituted with the prescriptions of Drug Bank and the abstracts of PubMed articles are stored in MEDLINE corpora. It constitutes with manually annotated 18,502 pharmacological substances along with 5028 DDI pairs; statistics of the dataset is shown in

Table 2. Statistics of DDIExtracrion-2013.

	DrugBank			MEDLINE
Contents	Train	Test	Total	Train	Test	Total
Articles	572	158	730	142	33	175
Drug Pairs	26,005	5265	31,270	1787	451	2238
Positive DDI Pairs	3789	884	4673	232	95	327
Negative DDI Pairs	22,216	4381	26,597	1555	356	1911

.

The annotated representation of this corpus with sentences and all possible instances, i.e., drug pairs, is shown in Figure 4. For a sentence, all instances may be ‘false’, as a drug pair is shown in Figure 4 with a black dotted block. On the contrary, a reported interaction between a pair of drugs is indicated with ‘true’, as presented in Figure 4 with a red rectangle block. There could be more than one ‘true’ and ‘false’ instance for a single sentence and all could be either true or false.

We used this corpus for training and testing of the model. The documents were initially pre-processed and stored in pickle files for training and testing of the model. The parameters of the DDI classification experiment are shown in

Table 3. Tuning parameters.

Parameter	Value
Word Embedding Dimensions	100
Position Embedding Dimensions	20
Mini Batch Size	60
Hidden State’s Dimensions	230
Shortest Path Length	12
Learning Rate	0.005
Embedding Dropout	0.7
Dense Dropout	0.5

. The rest of the parameters which are not shown in the table are kept as default settings as in TensorFlow [49].

Precision, recall and the f-measure are used for the DDI extraction task and are calculated by Formulas (6)–(8), respectively. The results of the DDI classification and severity identification are shown in

Table 4. Performance comparison with existing systems on DDIExtraction-2013.

Method	System/Team	Year	Precision (P)	Recall (R)	F-Score (F1)
SVM-based methods	UTurku [30]	2013	73.2	49.9	59.4
	NIL_UCM [63]	2013	55.0	53.0	54.0
	SCAI [29]	2013	55.0	39.0	46.0
	UWM-TRIADS [64]	2013	43.0	50.0	47.0
	UColorado_SOM [65]	2013	27.0	43.0	33.0
	BioSem [66]	2014	67.0	52.0	59.0
	FBLKA [28]	2015	nil	nil	67.0
	Raihani and Laachfoubi [67]	2016	73.7	68.7	71.1
	Zheng et al. [34]	2016	nil	nil	68.4
Neural Nets-based methods	MCCNN-DDI [40]	2016	nil	nil	67.8
	SCNN [41]	2016	68.5	61.0	64.5
	CNN-DDI [37]	2016	75.3	60.4	67.0
	Joint-LSTMs [39]	2017	71.3	66.9	69.3
	RHCNN [69]	2019	77.3	73.75	75.48
	SGRU-CNN [71]	2020	76.19	73.34	74.74
	AGCN [72]	2020	78.17	75.59	76.86
	Our Method: SEV-DDI	2021	83.81	81.59	82.68

, and comparison with state-of-the-art systems is also presented.

$$Precision \left(P\right)=\frac{TP}{TP+FP}$$

(6)

$$Recall \left(R\right)=\frac{TP}{TP+FN}$$

(7)

$$F1 Measure \left(F1\right)=\frac{2 \left(Precison\times Recall\right)}{Precision+Recall}$$

(8)

Consequently, the task of extracting severity from the DDI’s text is applied. The severity prediction mechanism is applied by implementing Algorithm 1 and applying it on the DDIExtraction-2013 dataset. We collected 4672 sentences from the DrugBank corpora and 327 from MEDLINE involving 9000 distinct drugs and 4999 DDIs. In the algorithm, Pol^pos and Pol^neg represent the positive and negative polarity of the words, respectively. After summing up all the polarities of the words, the final polarity of the sentence is calculated by subtracting Pol^neg from Pol^pos. Based on the final polarity of sentence, each DDI instance is assigned with a label of severity, as elaborated in Algorithm 1. The performance of the system is shown in Table 4, illustration of severity in

Table 5. Illustration of severity categorization of DDIs.

Sentence	Drug Pairs	Severity Level
The possibility of hypotensive effects can be minimized by either discontinuing the diuretic or increasing the salt intake prior to initiation of treatment with perindopril.	(Salt, perindopril)	Highly Beneficiary
AAV2-mediated retinal transduction is improved by co-injection of heparinase III or chondroitin ABC lyase.	(AAV2, chondroitin ABC lyase)	Moderate Beneficiary
Scopolamine should be used with care in patients taking other drugs that are capable of causing CNS effects such as sedatives, tranquilizers, or alcohol.	(Scopolamine, sedatives), (Scopolamine, tranquilizers), (Scopolamine, alcohol)	Low
Pantoprazole has a much weaker effect on clopidogrel’s pharmacokinetics and on platelet reactivity during concomitant use.	(Pantoprazole, clopidogrel)	Low
Concomitant use of alcohol with phentermine hydrochloride may result in an adverse drug interaction.	(alcohol, phentermine hydrochloride)	Moderate Dangerous
Other HDAC Inhibitors. Severe thrombocytopenia and gastrointestinal bleeding have been reported with concomitant use of ZOLINZA and other HDAC inhibitors (e.g., valproic acid).	(HDAC inhibitors, ZOLINZA)	Highly Dangerous

, while

Table 6. DDI severity prediction on DrugBank and MEDLINE.

		DrugBank	MEDLINE	Total
Articles		730	175	905
Candidate DDIs		31,270	2238	33,508
Positive DDI Related Sentence		4672	327	4999
Beneficial DDIs	Low	729	89	818
	Moderate	759	73	832
	High	762	39	801
Dangerous DDIs	Low	336	22	358
	Moderate	724	56	780
	High	1362	48	1410

shows the categorized statistics of the DrugBank and MEDLINE corpus with the extracted degree of severity presented in the text.

Algorithm 1: Severity Prediction in Drug–Drug Interaction.

1: The text is split for each DDI instance. Each sentence is addressed separately. 2: Each sentence is pre-processed and normalized using the NLTK tool kit. 3: Each sentence is processed using SentiWordNet and the polarity of words is extracted and summed into Pol pos and Pol neg for positive and negative polarity, respectively. 4: Final polarity is calculated as: Final-Polarity = Pol pos–Pol neg 5: The final polarity is split into two sets, i.e., Beneficial and Dangerous: 6:  if Final-Polarity ≥ 0 then: 7:   DDI instance is Beneficial. 8:  else 9:   DDI instance is Dangerous. 10:  end if 11: Categorization of Beneficial and Dangerous instances into levels of severity. 12: For each instance in Beneficial: 13:   if Final-Polarity is 0 to 0.2, then DDI is lightly beneficial. 14:   else if Final-Polarity is 0.3 to 0.6, then DDI is moderately beneficial. 15:   else DDI is highly beneficial. 16:   end if 17: end for 18: For each instance in Dangerous: 19:   if Final-Polarity is 0 to −0.2, then DDI is lightly dangerous. 20:   else if Final-Polarity −0.3 to −0.6, then DDI is moderately dangerous. 21:   else DDI is highly dangerous. 22:   end if 23: end for

4.1. Results and Discussion

The performance of the model is compared with state-of-the-art models for the DDI extraction task in Table 4. Our method outperformed in all three metrics of precision, recall and f-score. Each of these three metrics are critically important in biomedical-related tasks and particularly in DDI extraction, our model scored more than 80% in all three metrics, i.e., 83.81, 81.59, 82.68 in precision, recall and f-measure, respectively. To the best of our knowledge, none of the other systems achieved this milestone at the latest timepoint.

The improvement in the overall performance in the DDI-Extraction task, as mentioned in Table 4, is due to addition of the attention mechanism and bidirectional LSTM. The attention mechanism provided better features in the training which were involved in most of the interactions, and the dominance of these words with respect to each candidate drug is also enhanced. Consequently, employment of LSTM from both ends of the sentence resolved the issue of bias which was critical in the DDI extraction task where most significant words could be anywhere in a sentence, i.e., before, after or around the subject. Every word in a sentence achieved equal dominance regardless of its placement in the sentence which was proven to attain more accurate prediction in DDI extraction and classification, as results illustrated.

In the severity extraction task, all the DDI-related sentences are categorized in mild, moderate, or severe levels of interaction. Table 5 illustrates the categorization of DDIs at a level or degree of severity where examples of DDIExtraction-2013 are shown with their degree of severity in an interaction based on the polarity of the sentences; positive sentiments in the DDI related sentence are considered to have mild interaction in which pharmacologists or experts recommended some drugs to be taken together for safety or curing some disease, whereas a moderate level of severity is allocated to those DDIs whose polarity was neutral, i.e., where the experts have described some mechanisms or a weak impact on other drugs. A severe level of interaction is derived from the negative sentiments where clinical experts showed danger and provided advice of careful monitoring or showed some significant effect of one drug on others. The DDI categorization statistics are shown in Table 6.

We can claim this categorization is accurate by analyzing Figure 5, which presents the statistics of the performance of the DDI classification task by existing systems on Drug Bank and MEDLINE separately. It clearly shows that the performance of all the models remains low on MEDLINE, as compared to the Drug Bank, and none of the existing models achieved better performance on MEDLINE than on DrugBank.

DrugBank has articles with less ambiguity and shows clear interaction in the text; many already implemented models showed excellent performance when applied only to the DrugBank corpus as compared to when applied to MEDLINE. Co-relating this emphasis with our extracted severity, our system categorized 36% DDIs of DrugBank as being highly interactive, whereas just 27% of the MEDLINE’s DDIs were reported to have high interactivity.

On the contrary, articles of MEDLINE corpora are less expressive and have quite long and complex sentence structuring in reporting an interaction. Many of the state-of-the-art systems showed very poor performance on the MEDLINE corpus as compared to on DrugBank. Figure 6 shows statistical analysis regarding the performance of models on the corpuses of DrugBank and MEDLINE. It clearly shows that all the implemented models failed to achieve better performance on MEDLINE than on DrugBank, even some of the models showed very bad performance on MEDLINE, while showing excellent results on the corpus of DrugBank.

It proves that articles in the corpus of MEDLINE contain huge complexity in expressing the relationship of DDIs. That is why it always remained quite challenging for any model to perform better on the MEDLINE corpus for extraction and classification of DDIs.

Similarly, our severity extraction model also showed a much lower percentage of DDIs with high severity on both ends on the MEDLINE corpora than on DrugBank, i.e., 36% and 27% on DrugBank and MEDLINE, respectively. Alternatively, low and moderate levels of severity remained dominant in prediction with 34% and 39% of the MEDLINE corpus, compared with the predictions of DrugBank showing 22% and 32% of DDIs for mild and moderated levels of severity as shown in Figure 6.

Prediction of the degree of severity on the overall DDIExtraction-2013 [22] corpus is shown in

Table 7. Prediction of levels of severity on DDIExtraction-2013.

Severity Level	DrugBank	MEDLINE	DDIExtraction-2013
Low	22%	34%	24%
Moderate	32%	39%	32%
High	36%	27%	44%

. In our extracted ratings, about 44% of the DDIs in DDIExtraction-2013 are highly interactive ones, 24% of them show low interactivity and 32% of the interactions are of a moderate level. These results may vary a little in the perspective of pharmaceutical interactions, because these interactions are categorized on the basic of polarity of the sentences, which were used to report an interaction or the consequences of the interaction of the articles.

The prediction and annotation of a DDI’s severity by a pharmaceutical and healthcare expert could vary a little from our prediction. However, it is quite challenging for an expert to categorize the huge number of pairs in the levels of severity. The addition of severity concerning the DDIs in clinical healthcare and pharmaceutical expert systems could be very promising in making more precise and accurate prescriptions. It could also be very beneficial in reducing ADRs, which is one of the major causes of healthcare costs and hospital-acquired conditions and readmissions.

5. Conclusions

In this paper, we propose a hierarchal attention-based LSTM neural network model to improve the classification performance of drug–drug interaction tasks. Our model outperformed all the existing methods in overall performance as well as recall metrics. The method achieved 83.81%, 81.59% and 82.68% in the evaluation metrics of precision, recall and f1-measure, respectively. To the best of our knowledge, none of the existing models achieved more than an 80% threshold in all three metrics when applied on the DDIExtraction-2013 benchmark.

Beyond improving accuracy, we employed severity extraction mechanism for the DDIs which are reported and classified as positive DDIs (true interactions) by our model. From the reported text of DDIs in the literature, we extracted severity of each interaction employing sentiment analysis strategy. We investigated the severity of interaction by calculating the polarity of the text used to report an interaction. This mechanism helped us to formulate a new dataset regarding the DDI’s severity extraction task. Consequently, the prediction of the severity extraction mechanism is evaluated and investigated by applying different analysis based on the DDI classification task applied on DrugBank and MEDLINE separately.