A More Fine-Grained Aspect–Sentiment–Opinion Triplet Extraction Task

Abstract

Sentiment analysis aims to systematically study affective states and subjective information in digital text through computational methods. Aspect Sentiment Triplet Extraction (ASTE), a subtask of sentiment analysis, aims to extract aspect term, sentiment and opinion term triplets from sentences. However, some ASTE’s extracted triplets are not self-contained, as they reflect the sentence’s sentiment toward the aspect term, not the sentiment between the aspect and opinion terms. These triplets are not only unhelpful to people, but can also be detrimental to downstream tasks. In this paper, we introduce a more nuanced task, Aspect–Sentiment–Opinion Triplet Extraction (ASOTE), which also extracts aspect term, sentiment and opinion term triplets. However, the sentiment in a triplet extracted with ASOTE is the sentiment of the aspect term and opinion term pair. We build four datasets for ASOTE. A Position-aware BERT-based Framework (PBF) is proposed to address ASOTE. PBF first extracts aspect terms from sentences. For each extracted aspect term, PBF generates an aspect term-specific sentence representation, considering the aspect term’s position. It then extracts associated opinion terms and predicts the sentiments of the aspect–opinion term pairs based on the representation. In the experiments on the four datasets, PBF has set a benchmark performance on the novel ASOTE task.

1. Introduction

Sentiment analysis, an important task in natural language understanding is to systematically study affective states and subjective information in digital text through computational methods [1,2]. This field has received much attention from both academia and industry due to its wide range of applications, such as the voice of the customer (VOC) in marketing [3] and gaining insights from social media posts [4]. General sentiment analysis determines if the emotional tone of a text is positive, negative or neutral. Aspect-based sentiment analysis (ABSA) [5,6,7,8] is a fine-grained sentiment analysis task and can provide more detailed information than general sentiment analysis. To solve the ABSA task, many subtasks have been proposed, such as Aspect Term Extraction (ATE), Aspect Term Sentiment Analysis (ATSA) and Target-oriented Opinion Words Extraction (TOWE) [9]. An aspect term (aspect for short) is a word or phrase that refers to an entity discussed in a sentence. An opinion term (opinion for short) is a word or phrase that expresses a subjective attitude. ATE extracts aspects from sentences. Given a sentence and an aspect in the sentence, ATSA and TOWE predict the sentiment and opinions associated with the aspect. These subtasks can work together to tell a complete story, i.e., the discussed aspect, the sentiment of the aspect and the cause of the sentiment. However, no previous ABSA study has tried to provide a complete solution in one shot. Peng et al. [10] proposed the Aspect Sentiment Triplet Extraction (ASTE) task, which attempted to provide a complete solution for ABSA. A triplet extracted from a sentence via ASTE contains an aspect, the sentiment that the sentence expresses toward the aspect and one opinion associated with the aspect. The example in Figure 1 shows the inputs and outputs of the tasks mentioned above.

However, the triplets extracted from a sentence with ASTE are not self-contained when the sentence has multiple opinions about the aspect and these opinions express different sentiments toward the aspect. This is because the sentiment in a triplet extracted with ASTE is the sentiment that the sentence expresses toward the aspect rather than the sentiment of the aspect and opinion pair. The third column in Figure 2 shows the extraction results of ASTE for the corresponding sentences. The triplets whose sentiments are marked in red are not only unhelpful to people, but can also be detrimental to downstream tasks such as opinion summarization (opinion summarization is generated by aggregating the extraction triplets. When extracted triplets are erroneous, the opinion summarization built based on these triplets will not be accurate) [6].

In this paper, we introduce a more fine-grained Aspect–Sentiment–Opinion Triplet Extraction (ASOTE) task. ASOTE also extracts aspect, sentiment and opinion triplets. In the triplet extracted with ASOTE, the sentiment is the sentiment of the aspect and opinion pair. The fourth column in Figure 2 shows the extraction results of the ASOTE task for the corresponding sentences. In addition, we build four datasets for ASOTE based on several popular ABSA benchmarks.

Additionally, we propose a Position-aware BERT-based Framework (PBF) to address ASOTE. PBF first extracts aspects from sentences. For each extracted aspect, PBF then extracts associated opinions and predicts the sentiments of the aspect and opinion pairs. PBF obtains triplets by merging the results. Since a sentence may contain multiple aspects associated with different opinions, to extract the corresponding opinions of a given aspect, similar to previous models proposed for the TOWE task [9,11,12], PBF generates aspect-specific sentence representations. To accurately generate aspect-specific sentence representations, both the meaning and the position of the aspect are important. Some methods have been proposed to integrate the position information of aspects into non-BERT based models for some ABSA subtasks, such as refs. [13,14] for ATSA. However, how to integrate the position information of aspects into BERT [15]-based modes has not been studied well. PBF generates aspect-specific sentence representations considering both the meaning and the position of the aspect. We explore several methods which integrate the position information of aspects into PBF.

Our contributions are summarized as follows:

• We introduce a new aspect-based sentiment analysis subtask: Aspect–Sentiment–Opinion Triplet Extraction (ASOTE).
• We build four datasets for ASOTE and release the datasets for public use as a benchmark.
• We propose a Position-aware BERT-based Framework (PBF) to address ASOTE.
• In the experiments on the four datasets, PBF has set a benchmark performance on the novel ASOTE task.

2. Related Work

Aspect-based sentiment analysis (ABSA) [5,6,7,8] is a fine-grained sentiment analysis task. ABSA has many subtasks, such as Aspect Term Extraction (ATE), Opinion Term Extraction (OTE) (OTE extracts opinions from sentences), Aspect Term Sentiment Analysis (ATSA) and Target-oriented Opinion Words Extraction (TOWE) [9]. Many methods have been proposed for these subtasks. Most methods only solve one subtask, such as [16,17,18,19,20] for ATE, refs. [21,22,23,24,25,26,27,28,29,30] for ATSA and refs. [9,11,12,31] for TOWE. Some studies also attempted to solve two or three of these subtasks jointly. Refs. [32,33] jointly modeled ATE and ATSA, then generated aspect–sentiment pairs. Refs. [34,35,36] jointly modeled ATE and OTE, then output the aspect set and opinion set. The extracted aspects and opinions are not in pairs. Refs. [37,38] jointly modeled ATE and TOWE, then generated aspect–opinion pairs. Refs. [39,40] jointly modeled ATE, OTE and ATSA, then output the aspect–sentiment pairs and opinion set. However, the extracted aspects and opinions are also not in pairs; that is, the aspects, sentiments and opinions do not form triplets. The Aspect Sentiment Triplet Extraction (ASTE) task proposed by [10] extracts aspects, the sentiments of the aspects and opinions, which could form triplets. However, ASTE has the problem mentioned in Section 1.

Many methods have been proposed for ASTE [10,41,42,43,44,45,46,47]. Since most of these methods for ASTE do not utilize the fact that the sentiment of an ASTE triplet is the sentiment of the entire sentence toward the aspect term and may be from more than one opinion term and predict the sentiment of the aspect–opinion triplet as the triplet sentiment, these methods can be directly used for our proposed ASOTE task. For example, ref. [41] proposed an end-to-end model with a novel position-aware tagging scheme for ASTE. This model is capable of jointly extracting the triplets and can obtain better performance compared with previous pipeline approaches. However, this model has at most one triplet for an aspect term. The sentiment of a triplet is predicted only based on the extracted opinion term of the aspect term. Ref. [44] proposed a Grid Tagging Scheme (GTS) for ASTE. GTS first predicts the relationships between the words in a sentence, then decodes triplets from the relationships. The relationships includes the sentiment polarities. That is, the sentiment of an ASTE triplet is predicted based on the words in the aspect term and the opinion term and is the sentiment of the aspect–opinion pair. Different from these studies, we proposed a Position-aware BERT-based Framework (PBF) to address ASOTE.

3. Dataset Construction

Data Collection. We annotate four datasets (i.e., 14res, 14lap, 15res, 16res) for our proposed Aspect–Sentiment–Opinion Triplet Extraction (ASOTE) task. First, we construct four Aspect Sentiment Triplet Extraction (ASTE) datasets. Similar to previous studies [10,41], we obtain four ASTE datasets by aligning the four SemEval Challenge datasets [7,8] and the four Target-oriented Opinion Words Extraction (TOWE) datasets [9]. The four SemEval Challenge datasets are restaurant and laptop datasets from SemEval 2014 and restaurant datasets from SemEval 2015 and SemEval 2016. The four SemEval Challenge datasets provide the annotation of aspect terms and the corresponding sentiments and the four TOWE datasets were obtained by annotating the corresponding opinion terms for the annotated aspect terms in the four SemEval Challenge datasets. Compared with the ASTE datasets used in previous studies [10,41], the ASTE datasets we generate (1) keep the triplets with conflict sentiments and (2) keep all the sentences in the four SemEval Challenge datasets. That is, the sentences that do not contain triplets and therefore are not included in the ASTE datasets used in previous studies [10,41] are included in the ASTE datasets we generate. We believe datasets including these sentences can better evaluate the performance of ASOTE methods, since ASOTE methods can encounter this type of sentences in real-world scenarios.

Data Annotation. We invited a researcher who works in natural language processing (NLP) and an undergraduate student to annotate the sentiments of the aspect–opinion pairs in the triplets of the four ASTE datasets. The annotation tool we used is brat [48]. Each time, we only provided the annotators only with triplets of one aspect term. For each aspect term, not only the aspect term and its corresponding opinion terms but also the sentiment of the aspect term were provided to the annotators. Figure 3a shows an example of what we provided to the annotators and Figure 3b shows the results of annotation. When annotating the sentiment of an aspect–opinion pair, the annotators need to consider both the opinion itself and the context of the opinion. For example, given the sentence, “The decor is night tho…but they REALLY need to clean that vent in the ceiling…its quite un-appetizing and kills your effort to make this place look sleek and modern” (the triplets extracted with ASOTE from this sentence, i.e., (“place”, negative, “sleek”) and (“place”, negative, “modern”), are also not self-contained, since the sentiment shifter expression is complicated and therefore is not annotated as part of the opinions. One simple solution to this problem is to add a reversing word (e.g.,“not”) to this kind of opinion (e.g., “not sleek” and “not modern”) when we annotate opinions, which is left for future exploration) and one aspect–opinion pair, (“place”, “sleek”), the sentiment should be negative, even though the sentiment of “sleek” is positive. The kappa statistic [49] between the annotations of the two annotators is 0.85. The conflicts have been checked by another researcher who works in NLP.

Dataset Analysis. The statistics of the four ASOTE datasets are summarized in

Table 1. Statistics of our ASOTE datasets. #zero_t, #one_t and #m_t represent the number of aspects without triplet, with one triplet and with multiple triplets, respectively. #d_s1 represents the number of aspects that have multiple triplets with different sentiments. #d_s2 represents the number of aspects which only have one triplet and whose sentiments are not in conflict and are different from the sentiment of the corresponding triplet. #t_d represents the number of the triplets whose sentiments are different from the sentiments of the aspects in them.

Dataset		#sentence	#aspects	#triplets	#zero_t	#one_t	#m_t	#d_s1	#d_s2	#t_d
14res	train	2429	2984	2499	1662	1834	304	45	39	181
	dev	606	710	561	412	446	54	5	10	24
	test	800	1134	1030	464	720	144	14	9	42
14lap	train	2425	1927	1501	1868	1128	176	22	26	92
	dev	608	437	347	444	268	37	2	2	10
	test	800	655	563	553	411	69	9	9	40
15res	train	1050	950	1031	471	721	143	22	11	46
	dev	263	249	246	134	182	30	4	4	9
	test	684	542	493	390	385	51	13	5	26
16res	train	1595	1399	1431	793	1032	186	35	17	74
	dev	400	344	333	209	252	37	4	3	7
	test	675	612	524	412	395	61	14	6	28

. Since #diff_s2 is always greater than 0, the annotators have to annotate the sentiments of the triplets in which the aspect only has one triplet and the sentiment of the aspect is not conflicting. That is, we cannot treat the sentiment of the aspect in these triplets as the sentiment of these triplets. For example, for the third sentence in Figure 2, the aspect “Food” has a negative sentiment, while the correct sentiment of its only one triplet, (“Food”, neutral, “average”), is neutral.

4. Method

In this section, we describe our Position-aware BERT-based Framework (PBF) for Aspect–Sentiment–Opinion Triplet Extraction (ASOTE).

4.1. Task Definition

Given a sentence $S=\{w_0,\dots ,w_i,\dots ,w_{n-1}\}$ containing n words, ASOTE aims to extract a set of triplets $T={\{{(a,s,o)}_t\}}_{t=0}^{T-1}$, where a is an aspect, o is an opinion, s is the sentiment of the aspect–opinion pair $(a,o)$ and T is the number of triplets in the sentence. When a sentence does not contain triplets, $T=0$.

4.2. PBF

Figure 4 shows the overview of PBF. PBF contains three models. Given a sentence $S=\{w_0,\dots ,w_i,\dots ,w_{n-1}\}$, the Aspect Term Extraction (ATE) model first extracts a set of aspects $A=\{a_0,\dots ,a_j,\dots ,a_{m-1}\}$. For each extracted aspect, $a_j$, the Target-oriented Opinion Words Extraction (TOWE) model then extracts its opinions $O=\{o_j^0,\dots ,o_j^k,\dots ,o_j^{l_j-1}\}$, where $l_j$ is the number of opinions with respect to the j-th aspect and $l_j\ge 0$. Finally, for each extracted aspect–opinion pair $(a_j,o_j^k)$, the Aspect–Opinion Pair Sentiment Classification (AOPSC) model predicts its sentiment $s_j^k\in P=\{positive,neutral,negative\}$. PBF obtains the triplets by merging the results of the three models: $T=\{(a_0,s_0^0,o_0^0),\dots ,(a_{m-1},s_{m-1}^{l_{m-1}},o_{m-1}^{l_{m-1}})\}$. In PBF, all three models use BiLSTM [50] with BERT [15] as sentence encoder.

4.3. Input

The input of PBF is a sentence $S=\{w_0,\dots ,w_i,\dots ,w_{n-1}\}$ consisting of n words.

Given the sentence, to obtain the inputs of the ATE model (Figure 4a), we first convert the sentence S to a new sentence, $S_B=\{w_0,\dots ,w_i,\dots ,w_q\}$, where $w_0$ is “[CLS]” and $w_q$ is “[SEP]”. We then generate segment indices $I_{seg}=\{0,\dots ,0\}$ and position indices $I_{pos}=\{0,\dots ,q\}$ for the new sentence.

Since a sentence may contain multiple aspects associated with different opinions, to extract the associated opinions of a particular aspect, the TOWE model generates aspect-specific sentence representations for the aspect. It is intuitive that both the meaning and the position of the aspect are important for producing aspect-specific sentence representations. In other words, we need to tell the TOWE model what the aspect is and where the aspect is in the sentence. Given the sentence S and an aspect $a_j$ in the sentence, we first replace the words of the aspect with the word “aspect”, which tells the TOWE model where the aspect is in the sentence. We then append the words of the aspect to the end of the sentence, which tells the model what the aspect is. Finally, we obtain a new sentence $S_B^A=\{w_0,\dots ,w_i,\dots ,w_q\}$. We also generate segment indices $I_{seg}^A=\{0,\dots ,1\}$ and position indices $I_{pos}^A=\{0,\dots ,q\}$ for the new sentence. The encoder of the TOWE model (Figure 4b) takes $S_B^A$, $I_{seg}^A$ and $I_{pos}^A$ as inputs and can generate aspect-specific sentence representations.

To predict the sentiment of an aspect–opinion pair, the AOPSC model (Figure 4c) also generates aspect-specific sentence representations for the aspect. The inputs of the AOPSC model are the same as the TOWE model.

4.4. ATE

We formulate ATE as a sequence-labeling problem. The encoder takes $S_B$, $I_{seg}$ and $I_{pos}$ as inputs and outputs the corresponding sentence representation, $H^A=\{h_0^A,\dots ,h_i^A,\dots ,h_q^A\}$. The ATE model uses $h_i^A$ to predict the tag $y_i^A\in \{B,I,O\}$ (B: Begin, I: Inside, O: Outside) of the word $w_i$. It can be regarded as a three-class classification problem at each position of $S_B$. We use a linear layer and a softmax layer to compute prediction probability ${\widehat{y}}_i^A$:

$${\widehat{y}}_i^A=softmax(W_1^A{h}_i^A+b_1^A)$$

(1)

where $W_1^A$ and $b_1^A$ are learnable parameters.

The cross-entropy loss of the ATE task can be defined as follows:

$$L_{ATE}=-\sum _{i=0}^q\sum _{t\in \{B,I,O\}}\mathbb{I}(y_i^A=t)log\left({\widehat{y}}_{i_t}^A\right)$$

(2)

where $y_i^A$ denotes the ground truth label. $\mathbb{I}$ is an indicator function. If $y_i^A==t$, $\mathbb{I}$ = 1; otherwise, $\mathbb{I}$ = 0. We minimize $L_{ATE}$ to optimize the ATE model.

Finally, the ATE model decodes the tag sequence of the sentence and outputs a set of aspects $A=\{a_0,\dots ,a_j,\dots ,a_{m-1}\}$.

4.5. TOWE

We aslo formulate TOWE as a sequence-labeling problem. The TOWE model has the same architecture as the ATE model, but they do not share the parameters. The TOWE model takes $S_B^A$, $I_{seg}^A$ and $I_{pos}^A$ as inputs and outputs the opinions $O=\{o_j^0,\dots ,o_j^k,\dots ,o_j^{l_j-1}\}$ of the aspect $a_j$.

4.6. AOPSC

Given an aspect $a_j$ and its opinions $\{o_j^0,\dots ,o_j^k,\dots ,o_j^{l_j-1}\}$, the AOPSC model predicts the sentiments $\{s_j^0,\dots ,s_j^k,\dots ,s_j^{l_j-1}\}$ of all aspect–opinion pairs, $\{(a_j,o_j^0),\dots ,(a_j,o_j^k),\dots ,(a_j,o_j^{l_j-1})\}$, at once. The encoder of the AOPSC model takes the new sentence $S_B^A$, the segment indices $I_{seg}^A$ and the position indices $I_{pos}^A$ as inputs and outputs the aspect-specific sentence representation, $H^S=\{h_0^S,\dots ,h_q^S\}$. We then obtain the representation of an opinion by averaging the hidden representations of the words in the opinion. The representation $h_{j^k}^o$ of opinion $o_j^k$ is used to make sentiment prediction ${\widehat{y}}_{j^k}^o$ of opinion $o_j^k$:

$${\widehat{y}}_{j^k}^o=softmax(W_1^S{h}_{j^k}^o+b_1^S)$$

(3)

where $W_1^S$ and $b_1^S$ are learnable parameters.

The loss of the AOPSC task is the sum of all opinions’ cross entropy of the aspect:

$$L_{AOPSC}=-\sum _{k=0}^{l_j-1}\sum _{t\in P}\mathbb{I}(y_{j^k}^o=t)log{\widehat{y}}_{j_t^k}^o$$

(4)

where $y_{j^k}^o$ denotes the ground truth label. We minimize $L_{AOPSC}$ to optimize the AOPSC model.

5. Experiments

5.1. Datasets and Metrics

We evaluate our method on two types of datasets:

TOWE-data [9] are used to compare our method with previous methods proposed for the Target-oriented Opinion Words Extraction (TOWE) task on the TOWE task. TOWE-data only include the sentences that contain pairs of aspect and opinion and the aspect associated with at least one opinion. Following previous works [9,11], we randomly select 20% of the training set as a development set for tuning hyper-parameters and early stopping.

ASOTE-data are the data we built for our Aspect–Sentiment–Opinion Triplet Extraction (ASOTE) task and are used to compare the methods on the ASOTE task. ASOTE-data can also be used to evaluate the TOWE models on the TOWE task. Compared with TOWE-data, ASOTE-data additionally include the sentences that do not contain aspect–opinion pairs and include the aspects without opinions. Since methods can encounter these kinds of examples in real-world scenarios, ASOTE-data are more appropriate for evaluating methods on the TOWE task.

We use precision (P), recall (R) and F1-score (F1) as the evaluation metrics. For the ASOTE task, an extracted triplet is regarded as correct only if the predicted aspect spans, sentiment, opinion spans and ground truth aspect spans, sentiment and opinion spans exactly match.

5.2. Our Methods

We provide the comparisons of several variants of our Position-aware BERT-based Framework (PBF). The difference between these variants is the way they generate the new sentence $S_B^A$, the segment indices $I_{seg}^A$ and the position indices $I_{pos}^A$.

PBF -w/o A does not append the words of the aspect to the end of the original sentence. In other words, this variant does not know what the aspect is.

PBF -w/o P does not replace the words of the aspect with the word “aspect”; namely, this variant does not know where the aspect is. This model has been used on some aspect-based sentiment analysis subtasks to generate aspect-specific sentence representations [26,51].

PBF -w/o AP neither appends the words of the aspect to the end of the original sentence, nor replaces the words of the aspect with the word “aspect”.

PBF-M1 does not replace the words of the aspect with the word “aspect”. To inform the model about the position of the aspect, the words of the aspect in the original sentence and the words of the aspect appended to the original sentence share the same position indices. This method has been used on relation classification [52].

PBF-M2 does not replace the words of the aspect with the word “aspect”. To inform the model about the position of the aspect, the position indices of the words of the aspect in the original sentence are marked as 0 and the position indices of other words are the relative distance to the aspect. This method has been utilized in the aspect-term sentiment analysis task [13].

PBF-M3 modifies the original sentence S by inserting the special token # at the beginning of the aspect and the special token $ at the end of the aspect. Special tokens were first used by [53] to incorporate target entity information into BERT for the relation classification task.

Figure 5 displays input examples for PBF-M1, PBF-M2 and PBF-M3.

5.3. Implementation Details

We implement our models in PyTorch [54]. We use the uncased basic pre-trained BERT. The BERT is fine-tuned during training. The batch size is set to 32 for all models. All models are optimized with the Adam optimizer [55]. The learning rate is set to 0.00002. We apply a dropout of $p=0.5$ after the BERT and BiLSTM layers. We apply early stopping in training and the patience is 10. We run all models five times and report the average results on the test datasets. For the baseline models of the ASOTE task, we first convert our datasets into datasets that have the same format as the inputs of the baseline models, then run the code released by the authors on the converted datasets.

5.4. Exp-I: ASOTE

5.4.1. Comparison Methods

On the ASOTE task, we compare our methods with several methods proposed for the Aspect Sentiment Triplet Extraction (ASTE) task. These methods also extract aspect, sentiment, opinion triplets from sentences. These methods include MTL from Zhang et al. [45] (https://github.com/l294265421/OTE-MTL-ASOTE accessed on 1 January 2021), JET${}^t$, JET^o, JET${}_{+bert}^t$ and JET${}_{+bert}^o$ where $M=6$ from Xu et al. [41] (https://github.com/l294265421/Position-Aware-Tagging-for-ASOTE accessed on 1 January 2021), GTS-CNN, GTS-BiLSTM and GTS-BERT from Wu et al. [44] (https://github.com/l294265421/GTS-ASOTE accessed on 1 January 2021). All these baselines are joint models, which are jointly trained to extract the three elements of ASOTE triplets.

5.4.2. Results

The results of the ASOTE task are shown in

Table 2. Results of ASOTE task. The bold F1 scores are the best scores among PBF and the baselines. The underlined F1 scores are the best scores among PBF and its variants.

	14res			14lap			15res			16res
Method	P	R	F1	P	R	F1	P	R	F1	P	R	F1
OTE-MTL	63.8	52.1	57.3	51.3	36.8	42.7	56.3	44.0	49.3	58.3	52.4	55.0
JET${}^t$	66.0	48.4	55.8	41.0	36.5	38.6	45.3	47.8	46.5	58.1	46.9	51.9
JETo	61.5	53.9	57.5	48.8	36.7	41.9	57.5	47.2	51.8	61.0	56.8	58.8
GTS-CNN	66.4	58.5	62.2	55.3	37.4	44.6	56.3	48.1	51.8	61.4	60.0	60.5
GTS-BiLSTM	71.1	54.5	61.5	58.0	33.9	42.8	67.3	42.9	52.4	64.6	55.8	59.8
JET${}_{+bert}^t$	65.1	51.7	57.6	50.2	41.7	45.5	50.7	48.2	49.4	55.0	52.1	53.5
JET${}_{+bert}^o$	66.0	54.5	59.7	49.7	42.8	46.0	53.8	52.9	53.3	58.3	60.3	59.2
GTS-BERT	67.5	67.2	67.3	59.4	48.6	53.5	61.8	52.0	56.4	62.0	67.1	64.4
PBF	69.3	69.0	69.2	56.6	55.1	55.8	55.8	61.5	58.5	61.2	72.7	66.5
PBF -w/o A	67.3	69.3	68.3	55.9	55.7	55.8	56.4	61.6	58.8	60.7	71.3	65.5
PBF -w/o P	68.6	69.7	69.1	56.6	54.8	55.7	56.2	60.4	58.2	59.6	71.8	65.1
PBF -w/o AP	44.4	51.9	47.4	45.1	48.8	46.7	41.7	50.9	45.7	46.1	59.8	52.0
PBF-M1	66.6	69.7	68.1	58.8	54.1	56.3	57.8	59.4	58.4	59.3	72.1	65.0
PBF-M2	63.0	63.6	63.3	51.8	47.3	49.4	50.2	56.2	53.0	56.6	65.8	60.8
PBF-M3	66.8	69.2	68.0	56.8	53.3	54.9	54.2	61.7	57.7	60.4	71.1	65.2

. We have several observations from Table 2. First, MTL outperforms JET${}^t$ on all datasets, because JET${}^t$ can extract at most one triplet for an aspect. Although JET^o can extract at most one triplet for an opinion, JET^o outperforms JET${}^t$ on all datasets and surpasses MTL on 3 of 4 datasets, because there are fewer opinions belonging to multiple triplets than aspects belonging to multiple triplets. Second, GTS-CNN and GTS-BiLSTM outperform both JET${}^t$ and JET^o on all datasets and GTS-BERT also achieves better performance than JET${}_{+bert}^t$ and JET${}_{+bert}^o$. GTS-BERT is the best baseline model. Third, our proposed PBF surpasses GTS-BERT on all datasets. Since the Aspect Term Extraction (ATE) model and the Aspect–Opinion Pair Sentiment Classification (AOPSC) model in PBF are vanilla, compared with previous models, the advantages of PBF are from the TOWE model. However, GTS-BERT cannot be applied to the TOWE task directly, so we compare PBF with GTS-BERT on the aspect–Opinion Pair Extraction (OPE) [44] task. The results of OPE are shown in

Table 3. Results of the OPE task in terms of F1. The bold F1 scores are the best scores among PBF and the baseline.

Method	14res	14lap	15res	16res
GTS-BERT	71.7	60.2	61.5	68.1
PBF	74.0	63.8	63.9	70.8

, which shows that PBF also outperforms GTS-BERT on all datasets. Fourth, PBF outperforms PBF -w/o P on all datasets, indicating that integrating position information of aspects can boost the model performance. Fifth, compared with PBF -w/o A, PBF obtains better performance on 14res and 16res, similar performance on 14lap and worse performance on 15res. Similar phenomenon can also be observed from the TOWE results in

Table 4. Results of the TOWE task in terms of F1 on the ASOTE-data. The bold F1 scores are the best scores among PBF and the variants.

Method	14res	14lap	15res	16res
PBF	81.5	74.0	77.9	82.1
PBF -w/o A	80.7	74.1	78.6	81.6
PBF -w/o P	80.9	74.0	77.3	81.0
PBF -w/o AP	56.1	61.9	60.5	64.8
PBF-M1	80.1	73.0	77.4	80.5
PBF-M2	75.1	66.1	72.9	76.5
PBF-M3	80.3	73.6	77.5	80.3

. This indicates that the meaning of the aspect is useful but the method used to combine the position information with the aspect meaning in PBF is not perfect. We leave the exploration of more effective combination methods for future work. Sixth, PBF outperforms PBF-M1 in 3 of 4 datasets (14res, 15res and 16res) and surpasses PBF-M2 on all datasets, which shows that our method of incorporating aspect position information is more effective. Although the method used by PBF-M2 to integrate the position information of aspects into it has been successfully applied to non-BERT based models, it is not effective enough for BERT-based models. Moreover, PBF-M1 is a little better than PBF on the 14lap dataset. This indicates it is necessary for PBF to explore more effective methods of incorporating aspect position information. Seventh, PBF outperforms PBF-M3, indicating our method is more effective than the method of integrating the position information and meaning of an aspect into a model by inserting special aspect markers for the aspect. The possible reason is that the additional special tokens may destroy the syntax knowledge learned by BERT. Last but not least, PBF -w/o AP obtains the worst performance among all variants, which further demonstrates that both the position and the meaning of an aspect are important.

5.4.3. Case Study

To further understand the effect of the position and the meaning of an aspect, we perform a case study on two sentences, as displayed in Figure 6. In the first sentence, the bold “food” and underlined “food” are different aspects. The positions of the aspects help PBF and PBF -w/o A to extract different opinions for aspects with the same meaning. In the second sentence, with the help of the meaning of the aspect “crust”, PBF and PBF -w/o P do not extract “raw” and “cold” as the opinions of “crust”.

5.5. Exp-II: TOWE

5.5.1. Comparison Methods

On the TOWE task, we compare our methods with (1) three non-BERT models: IOG [9], LOTN [11], ARGCN [31]; (2) two BERT-based models: ARGCN${}_{+bert}$ [31] and ONG [12].

5.5.2. Results

The results for ASOTE-data are shown in Table 4 and the results for TOWE-data are shown in

Table 5. Results of the TOWE task in terms of F1 on TOWE-data. The results of the baselines are from the original papers. The bold F1 scores are the best.

Method	14res	14lap	15res	16res
IOG	80.0	71.3	73.2	81.6
LOTN	82.2	72.0	73.2	83.6
ARGCN	84.6	75.3	76.7	85.1
ARGCN${}_{+bert}$	85.4	76.3	78.2	86.6
ONG	82.3	75.7	78.8	86.0
PBF	85.9	81.5	80.8	89.2
PBF -w/o A	86.1	81.2	80.4	87.9
PBF -w/o P	86.3	80.3	79.8	88.8
PBF -w/o AP	61.6	67.9	59.0	69.3

. We draw the following conclusions from the results. First, PBF outperforms all baselines proposed for TOWE on the TOWE-data, indicating the effectiveness of our method. Second, PBF -w/o P also surpasses all baselines on the TOWE-data. To the best of our knowledge, no previous study evaluates the performance of this method on TOWE. Third, regarding PBF and its variants, we can obtain conclusions from Table 4 similar to the conclusions obtained from Table 2, because the differences in these models’ performance on ASOTE are mainly due to the differences in their performances on TOWE. Fourth, since the methods (i.e., PBF, PBF -w/o A, PBF -w/o P and PBF -w/o AP) obtain better performance on TOWE-data than on ASOTE-data, the ASOTE-data dataset is a more challenging dataset for TOWE. Fifth, on the 14res dataset, PBF does not surpasses its variants PBF -w/o A and PBF -w/o P, further indicating that it is necessary for PBF to explore more effective methods of combining the position information with the aspect meaning in the future.

6. Conclusions

In this paper, we introduce the Aspect–Sentiment–Opinion Triplet Extraction (ASOTE) task. ASOTE is more fine-grained than Aspect Sentiment Triplet Extraction (ASTE). The sentiment of a triplet extracted with ASOTE is the sentiment of the aspect–opinion pair in the triplet. We manually annotate four datasets for ASOTE. Moreover, we propose a Position-aware BERT-based Framework (PBF) to address ASOTE. Although PBF is a pipeline method, it obtains better performance than several joint models, which demonstrates the effectiveness of our method.

Since Aspect Term Extraction (ATE) and Target-oriented Opinion Words Extraction (TOWE) are highly correlated with each other and TOWE and Aspect–Opinion Pair Sentiment Classification (AOPSC) are also highly correlated with each other, we can improve PBF by turning it into a joint model which jointly trains the ATE model, the TOWE model and the AOPSC model. However, it is not easy to jointly train the ATE model and the TOWE model, since we need to use the aspects that the ATE model extracts to modify the sentences that the TOWE model takes as input. In the future, we will explore how to jointly train the ATE model and the TOWE model.