Correlations and Fractality in Sentence-Level Sentiment Analysis Based on VADER for Literary Texts

Abstract

We perform a sentence-level sentiment analysis study of different literary texts in English language. Each text is converted into a series in which the data points are the sentiment value of each sentence obtained using the sentiment analysis tool (VADER). By applying the Detrended Fluctuation Analysis (DFA) and the Higuchi Fractal Dimension (HFD) methods to these sentiment series, we find that they are monofractal with long-term correlations, which can be explained by the fact that the writing process has memory by construction, with a sentiment evolution that is self-similar. Furthermore, we discretize these series by applying a classification approach which transforms the series into a one on which each data point has only three possible values, corresponding to positive, neutral or negative sentiments. We map these three-states series to a Markov chain and investigate the transitions of sentiment from one sentence to the next, obtaining a state transition matrix for each book that provides information on the probability of transitioning between sentiments from one sentence to the next. This approach shows that there are biases towards increasing the probability of switching to neutral or positive sentences. The two approaches supplement each other, since the long-term correlation approach allows a global assessment of the sentiment of the book, while the state transition matrix approach provides local information about the sentiment evolution along the text.

1. Introduction

Sentiment analysis, or opinion mining, is an active research area in the field of natural language processing that analyzes people’s opinions that are loaded with sentiments and emotions via the computational treatment of subjectivity in text that is produced in different platforms [1,2,3,4,5]. Sentiment analysis mainly considers two aspects: categorical sentiment analysis and dimensional sentiment analysis. Categorical sentiment analysis basically classifies emotions under categories or labels based on the fact that these emotions can be classified using a low number of basic emotions. In contrast, in dimensional analysis, sentiment can be represented by emotions along continuous dimensions rather than discrete categories [6,7]. In general, two basic dimensions are included: valence and intensity. Sentiment analysis makes use of various lexicons to quantify emotional intensity [8,9,10,11,12,13]. There are several lexicons built from very diverse sources, such as the Social Contextualized Affect Lexicon (SO-CAL), SentiWordNet, AFINN, and ANEW, to name a few [14,15,16,17]. SO-CAL aims to analyze sentiment in social network contexts, captures how emotions are expressed in informal language, and evaluates emotions in terms of valence, arousal, and dominance [18,19,20]. AFINN focuses on evaluating conversations in social network posts and assigns words a score ranging from negative to positive, providing a direct numerical sentiment score [21]. SentiWordNet is an extension of WordNet that provides three scores for each word: positive, negative, and objective, allowing for a nuanced understanding of word sentiment [22,23]. ANEW provides emotional ratings of words, including valence, arousal, and dominance values, and is used in the context of emotional profiling analysis [24,25,26,27].

A recent study conducted an extensive literature survey on sentiment analysis to identify existing sentence-level methods covering several different techniques in order to perform a comparison with each other to identify their advantages, disadvantages, and limitations [28]. The authors of that study suggested that there is no single method that always achieves the best prediction performance for all different datasets. However, they found that the method called VADER (for Valence Aware Dictionary for sEntiment Reasoning) was the most consistent method in different experiments that were performed. VADER uses a combination of qualitative and quantitative methods to produce and then empirically validate a sentiment lexicon that is especially attuned to microblog-like contexts. Moreover, after incorporating some lexical rules that embody grammatical and syntactical conventions that humans use when expressing or emphasizing sentiment intensity, the accuracy of the sentiment analysis engine improves across several domain contexts (social media text, NY Times editorials, movie reviews, and product reviews) [29]. Therefore, we select VADER to perform the sentence-level sentiment analysis in the present work.

On the other hand, previous studies performed on different corpora in several languages taken mainly from Web-based content (online forums, blogs, Twitter, etc.) have found that positive words carry less information [30] and that words with a positive emotional content are more frequently used [30,31]. However, these previous studies have been centered in a word-by-word sentiment analysis. Are these conclusions still valid if the analysis is performed at a higher level, i.e., at sentence level?

Our goal is to analyze the evolution of the sentiment along the text by examining the statistics associated to the sentiment value of each sentence and then assess the presence of correlations. We find that, for all the texts analyzed, the correlations follow power laws with exponent values that differ from the randomized case, confirming the existence of long-term memory. We also quantify the dependence of sentiment values on transitions to positive, negative, or neutral, and find that either neutral or positive sentences are more likely to be followed by neutral or positive ones as well. We focus on single-author works for which the writing dynamics differ from those of Web-based content (online forums, blogs, Twitter, etc.) since the authors take more time to conclude the text after reviewing it as part of the writing process and, oftentimes, the texts are also subject to the editor’s review process. Therefore, these literary works are subject to a longer production process, implying that the selection of words in the sentences is more thoroughly performed. Thus, our approach supplements the previous studies based on a high number of authors in online opinion forums or Twitter, and then this allows to compare the behavior of an ensemble of authors on shorter texts versus the longer texts produced by single authors under more scrutiny.

This paper is organized as follows: in Section 2, we present the related work. In Section 3, we describe briefly the sample of books that were considered. In Section 4, we describe the methods to obtain series of sentiment values from the texts which are processed to investigate correlations. In Section 5, we discuss the results. Finally, the concluding remarks are given in Section 6.

2. Related Work

Sentiment analysis in literary texts is a developing field that leverages computational techniques to understand and interpret the emotional dimensions that appear in literary texts. As is widely recognized, emotions play a transcendental role in narratives or stories developed by authors, shaping characters’ motivations and influencing readers. The study of emotions in literature has been modified to include more quantitative analyses through computational methods, which has allowed for more extensive and systematic investigations of how emotions are displayed through a literary text. VADER offers an important tool for sentiment analysis within literary studies, as it allows the emotional dimensions of texts to be quantified. It has been noted that it also has certain limitations—especially in relation to sarcasm and domain specificity—but its ease of use and effectiveness make it an attractive option for both qualitative and quantitative literary analysis. As the field has progressed, integrating tools such as VADER into literary research can enhance our understanding of emotional dynamics in literature. For example, Bizzoni et al. [32] explored the complexities of analyzing sentiment in narrative texts such as those of E. Hemingway. Yeruva et al. [33] analyzed data from literary texts related to human feeling in classical Greek tragedy. More recently, S. Vinodini [34] conducted a sentiment analysis of selected novels by the celebrated Brazilian author Paulo Coelho.

On the other hand, long-range correlations are characterized by a relationship or memory between subsequences separated by significant distances. In the context of literature, this can be understood as recurrence of ideas or emotional motifs that recur throughout a text, regardless of their physical distance in the narrative. The presence of correlations in literary texts has been reported in various contexts such as word lengths [35,36,37], sentences [38], and sentiment trajectories [39]. Also, it has been reported that the temporal organization of written texts can be analyzed from a multifractal perspective [40,41].

3. Input Data

The input data are comprised of 50 ebooks in English language downloaded from the websites of the Gutenberg Project (http://www.gutenberg.org/, last accessed date: 27 May 2024) and the Project Gutenberg Australia (http://gutenberg.net.au/, last accessed date: 27 May 2024). The titles of the written texts are described online at https://figshare.com/articles/dataset/A_sentence-level_sentiment_analysis_of_some_literary_texts_inEnglish_language/27092008, last accessed date: 30 September 2024. There was no particular strategy to select the titles. The selected books were first published in different epochs and countries, therefore offering diversity in time and cultures of origin. We remove the preface, table of contents, etc., of the books and only retain the text.

4. Methods

4.1. Vader Sentiment Values

VADER provides four scores for the sentiment and intensity of a sentence: positive (pos), neutral (neu), negative (neg), and compound (com). The pos, neu, and neg scores are ratios for proportions of text that fall in each category; therefore, these should all add up to be 1. These are the most useful metrics for multidimensional measures of sentiment for a given sentence. Moreover, the compound score is computed by summing the valence scores of each word in the lexicon, adjusted according to certain rules, and then normalized to be a floating number between $-1$ (most extreme negative) and $+1$ (most extreme positive) [29]. This is the most useful metric for a single unidimensional measure of sentiment for a given sentence.

Using the compound score, standardized thresholds can be set for classifying sentences as either positive, neutral, or negative. Typical threshold values are [29]:

• positive sentiment: compound $\ge 0.05$,
• neutral sentiment: $-0.05<$ compound $<0.05$,
• negative sentiment: compound $\le -0.05$.

In the present work, we choose the compound score since we are interested in a single unidimensional measure of sentiment for each sentence in order to study correlations of sentiment along the text as well as the evolution of sentiment by studying the changes in sentiment from one sentence to the next.

In order to estimate the sentiment of the sentences in a text, we start by parsing the text to split it into sentences. Each sentence is then evaluated with VADER to obtain its compound sentiment score, and therefore, the text is mapped to a data series of sentiment values. Figure 1 shows an example of these sentiment series for an specific book. The complete resulting corpus containing the sentiment scores for each book can be found online at https://figshare.com/articles/dataset/A_sentence-level_sentiment_analysis_of_some_literary_texts_inEnglish_language/27092008, last accessed date: 30 September 2024. These series are then processed to investigate the kind of correlations that they contain, for which we apply the well-known Higuchi Fractal Dimension (HFD) [42] and the Detrended Fluctuation Analysis (DFA) [43] methods.

Moreover, we use the classification threshold mentioned previously to transform each series of compound sentiment into a series of three possible discrete values (states) of sentiment: positive, neutral, or negative (see Figure 2). In order to study the sentiment transitions along the text, we count the number of transitions of each type. For instance, let us suppose that the kth sentence is classified as positive and the $k+1$th is classified as negative, then the number of transitions of the type pos $\to$ neg is increased by one. Once the counting of transitions is finalized, we normalize the matrix to obtain a state transition matrix $\mathbf{p}$ for each one of the books in our sample:

$$\mathbf{p}=\left(\begin{array}{ccc}{p}_{\mathtt{pos}\to \mathtt{pos}}& p_{\mathtt{pos}\to \mathtt{neu}}& p_{\mathtt{pos}\to \mathtt{neg}}\\ p_{\mathtt{neu}\to \mathtt{pos}}& p_{\mathtt{neu}\to \mathtt{neu}}& p_{\mathtt{neu}\to \mathtt{neg}}\\ p_{\mathtt{neg}\to \mathtt{pos}}& p_{\mathtt{neg}\to \mathtt{neu}}& p_{\mathtt{neg}\to \mathtt{neg}}\end{array}\right)$$

(1)

Therefore, we map the evolution of the classified sentiment to a Markov chain for each book, with the state transition coefficients given by Equation (1). Shannon used Markov chains to predict sequences of words, observing that a reasonable approximation to English could be produced if each word was chosen based not just on the previous word but on the last few words, and introduced a mathematical framework for analyzing the information provided by the word [44]. Also, Markov chains have been used to compute how probable each instance of a word is based on the last few words, providing a way to measure the predictability of a word in its context [45]. Interestingly, Markov himself published a report in which he computes the probabilities that a vowel is followed by a vowel or a consonant in a text excerpt containing 20,000 Russian letters of the Russian alphabet from Pushkin’s novel [46].

In the present work, we are interested not in the word length, but in the sentiment of the sentence. Given that we wrote a sentence with certain sentiment, what is the probability that the next sentence either switches to another sentiment state or stays on the same one? How does the sentiment of a certain sentence relate to the sentiment of the previous few sentences?

4.2. Fractal Dimension Method and Detrended Fluctuation Analysis

Fractals are complex patterns that show self-similarity at different scales and are characterized by a dimension that is not necessarily an integer value, which quantifies the complexity of the fractal pattern. In the case of time series, which are considered non-isotropic fractals because they extend over time while the values of the variable remain in a bounded range (auto-affinity), the fractal characterization can be performed using several methodologies. To capture this statistical self-affinity property, it is usually resorted to quantify a measure of the object at different scales. This leads to a power law ($G\left(x\right)\sim x^a$), where the scaling exponent a represents the level of spatio-temporal organization displayed by the time series. The power spectrum is the method traditionally used to characterize autocorrelations in time series. For example, we consider a stationary stochastic process with an autocorrelation function following a power law $C\left(s\right)\sim s^{\gamma }$, where s is the lag and $\gamma$ is the correlation exponent, $0<\gamma <1$. The presence of long-range correlations is related to the fact that the mean correlation time diverges for infinite time series. According to the Wiener–Khintchin theorem, the power spectrum is the Fourier transform of the autocorrelation function $C\left(s\right)$, and for the case described above, we have the scaling relation $S\left(f\right)\sim f^{-\beta }$, where f is the frequency and $\beta$ is called the spectral exponent and is related to the correlation exponent by $\gamma =1-\beta$. However, the estimation of $\beta$ may not be as accurate when we have a nonstationary series. Alternative methods have been proposed for evaluating the correlations and fractal properties of stationary and nonstationary time series [42,47,48,49]. Here, we briefly describe two robust methods for the estimation of the fractal scaling exponents:

(i) Fractal Dimension Method. We evaluate the time organization in sequences of sentiment values by means of the Fractal Dimension Method (FDM), also known as Higuchi Fractal Dimension (HFD) method [50]. Briefly, we explain the main steps of the FDM [50,51]. Given the time series $x_1,x_2,\dots ,x_N$, we construct new time series $x_m^k$ defined as $x_m,x_{m+k},x_{m+2k},\dots ,x_{(m+[\frac{N-k}{k}]·k)}$, with $m=1,2,3,\dots ,k$, [ ] denoting Gauss’ notation, that is, the bigger integer and m and k are integers that indicate the initial time and the interval time, respectively. The length of the curve $x_m^k$, is defined as

$$\begin{array}{c}{L}_m\left(k\right)={\displaystyle \frac{1}{k}}\left[\left({\sum }_{i=1}^{\left[\frac{N-m}{k}\right]}\left|x(m+ik)-x(m+(i-1\left)k\right)\right|\right){\displaystyle \frac{N-1}{\left[\frac{N-m}{k}\right]k}}\right]\hfill \end{array}$$

(2)

and the term $(N-1)/\left[\right(N-m)/k]k$ represents a normalization factor. The length of each sequence $x_m^k$ is considered to construct the length of the original curve for the time interval k, $\langle L\left(k\right)\rangle$. Finally, if a scaling behavior of the form $\langle L\left(k\right)\rangle \propto k^{-D}$ is observed, then the curve is fractal with dimension D [50]. It is known that the fractal dimension is related to the spectral exponent $\beta$ by means of $\beta =5-2D$ [50]. We notice that this relationship is valid for $1<D<2$ and $1<\beta <3,$ and thus FDM is not a reliable method for signals with strong anticorrelated behavior, that is, for $-1<\beta <0$. To overcome this problem, we first integrate the original time series prior to applying the standard FDM. In this way, for processes which can be described as the first derivative of fluctuations with spectral exponent within the interval $1<\beta <3$, the relationship between $\beta$ and D changes to $\beta =3-2D$. A process with positive long-range correlations leads to $D<1.5$, whereas for anticorrelated processes, $D>1.5$. The irregular fluctuations with no memory are represented by $D=1.5$.

(ii) Detrended Fluctuation Analysis. This method was introduced to quantify long-range correlations in the heartbeat interval time series and DNA sequences [43,48,52]. The DFA method has been used to explore the presence of correlations in different areas of science, from heartbeat [52] and stock markets to earthquakes [53]. Briefly, the DFA is described as follows: First, we integrate the original time series to obtain $y\left(k\right)={\sum }_{i=1}^k\left[x\left(i\right)-x_{ave}\right]$. The resulting series is divided into boxes of size n. For each box, a straight line is fitted to the points, $y_n\left(k\right)$. Next, the line points are subtracted from the integrated series, $y\left(k\right)$, in each box. The root mean square fluctuation of the integrated and detrended series is calculated by means of

$$F\left(n\right)=\sqrt{{\displaystyle \frac{1}{N}}\sum _{k=1}^N{\left[y\left(k\right)-y_n\left(k\right)\right]}^2}.$$

(3)

This process is taken over several scales (box sizes) to obtain a power law behavior $F\left(n\right)\sim n^{\alpha }$, with $\alpha$ being an exponent which reflects self-similar and correlation properties of the signal. An uncorrelated signal leads to $\alpha =0.5$, $\alpha =1$ represents a long-range correlated process ($1/f$ noise), and $\alpha =1.5$ corresponds to a Brownian motion. For $0.5<\alpha <1$, the scaling exponent $\alpha$ and the Hurst exponent provide the same information, and therefore both can be used interchangeably [54]. The exponent $\alpha$ is also related to the spectral exponent $\beta$ (within the interval $0<\beta <1$) through relation $\beta =2\alpha -1$. Thus, fractal dimension D is related to $\alpha$ by means of $\alpha =2-D$, as expected for auto-affine signals [47,55].

5. Results and Discussion

5.1. Correlations Analysis

Figure 3 shows the log–log plots for the HFD and DFA analyses performed on the data series of compound sentiment scores for all the books in our sample. As can be seen, these data series exhibit fractal behavior with different scaling exponents for both analyses. In order to further study these correlations, we shuffle the sentences in each book prior to estimating the VADER sentiment and the resulting data series are analyzed with the HFD and DFA methods.

Figure 4 shows a scatter plot of $\alpha$ vs. D for the original texts and the shuffled ones. The mean values for the scaling exponents for the original texts are $D=1.38\pm 0.04$ and $\alpha =0.62\pm 0.04$. Even though there is scattering in the values for the scaling exponents, we can see from the values of $\alpha$ that the data series of sentiment are slightly long-term correlated. On the other hand, the mean values of the scaling exponents for the shuffled texts are $D=1.49\pm 0.03$ and $\alpha =0.50\pm 0.02$. We perform a two-sample t-test for each scaling exponent between the original sequences and the shuffled ones, with the null hypothesis that the two populations have no statistically significant difference. In other words, we take the population of one scaling exponent computed for the original sequence (for instance, $\alpha$) and perform the t-test to assess statistical difference to the population of the values of $\alpha$ obtained for the shuffled sequences, and similar approach is followed for the fractal dimension D. The p-values obtained for the scaling exponents are: $p_D=7.4\times {10}^{-29}$ and $p_{\alpha }=4.5\times {10}^{-22}$. They allow us to reject the null hypothesis, i.e., we find statistically significant differences in $\alpha$ and D between the original sequences and the randomized ones.

We also observe that there is a good concordance between the median values obtained from both methods. By using the relationship $\alpha =2-D$, it is easy to show that $\overline{D}=1.38$ yields to $\overline{\alpha }=0.62$, which matches the value obtained by means of the DFA. This also applies for the case of the shuffled sequences.

We compute the fraction of the number of sentences on each sentiment state to the total number of sentences for each book. Figure 5 shows box plots for each fraction for all the books considered in our study, with the median values of each population shown within the box. As can be seen, the majority of sentences are valued as neutral, followed by the positive and then the negative ones. The observation that there are more positive than negative sentences in the books under study is in agreement with recent studies reporting a positive bias in human expression [30,31], although in those works, the positivity in the language is based on a word-by-word analysis, and in our case, the analysis is made on a sentence-by-sentence basis, which supplements and extends those previously reported results.

In order to investigate further the bias towards positive sentiment in language, for each book, we obtain the normalized histograms for the compound sentiment series. Figure 6 shows the overlay of the histograms. As can be seen, there is a peak for the sentiments around the neutral values of sentiment. Moreover, we compute the area under the positive (denoted by $A_{+}$) and the negative ($A_{-}$) sides, which offer the probability of finding positive or negative sentences in the text, respectively. The scatter plot of $A_{+}$ vs $A_{-}$ in the inset of Figure 6 shows that for most of the cases, the probability of finding positive sentences is larger than the one for the negative ones. Moreover, we compute the empirical cumulative distribution function for the series of compound series and compare the positive and negative tails (see Figure 7). As can be seen, the probability of finding sentences with a compound value lower than a certain value $c_0$ decreases faster for the negative tail.

5.2. Observations from the Transition Coefficients

As mentioned in the Methods section, we discretize the compound sentiment series by applying a classification approach. After counting the number of sentiment states transition, we come up with a state transition matrix for each book. Figure 8 shows box plots for the coefficients of each possible transition, for both populations: original text and shuffled text. As can be seen, the most likely transition is from a neutral sentence to another neutral one, followed by the transitions from a positive (negative) sentences to a positive (negative) one.

We see that for the three states, the transition to a neutral sentence is the one with higher probability, which suggests that the neutral sentence is an inflection point for the sentiment when changing emotion (positive to negative or vice versa) or that these sentences are providing general and contextual information in the narrative. Starting from a positive sentence, it is more likely to move to a positive sentence, and from a negative sentence to a negative one, except that the latter transition is less differentiated with respect to the transitions to neutral or positive sentences, while moving from a neutral to a neutral sentence is markedly more likely compared to the other two transitions. Moreover, the medians of the transition coefficients show that the transition matrices are not symmetric, since there is a certain bias for transitions to positive and to neutral sentences. In summary,

Table 1. List of state transitions in descending order of transition probabilities.

Initial State	First Most Likely Transition	Second	Third
positive	positive	neutral	negative
neutral	neutral	positive	negative
negative	negative	positive	neutral

shows the list of state transitions in descending order on the probabilities of transitioning to a certain state given a start on a particular one.

On the other hand, Figure 8 also shows the state transition coefficients for the shuffled sentences. As can be seen, the transition matrix between states of the same type ($pos$–$pos$, $neg$–$neg$ and $neu$–$neu$) exhibit mean values that are smaller than the values corresponding to the original data, while for the other transitions, the differences between the mean values is less noticeable. We see that for the shuffled sentences, the transition coefficient to a certain state is the same independently of on which state we started with, while there is a bias towards neutral sentences followed by positive ones. Again, the transition to negative sentences has the lower probability. Therefore, even in this case of shuffled sentences, there is a bias towards positive sentiment in human language, as previously reported in other studies [56]. One interesting thing to note here is that the median values of the transition coefficients for the shuffled texts follow the same proportion that the fraction of text that it is in a certain sentiment state (see Figure 5).

The long-term correlations in the sentiment data series that are revealed by the DFA analysis are somehow expected since, by construction, the writing process has memory given that the narrative is about concatenating sentences that are related. Our results on the presence of long-term correlations with values in fractal exponents, which reveal slight persistence, are in agreement with previous results reported for sentiment value sequences and those based on word-length [35,37,38], sentence-length [57], and sentiment arcs [58]. In addition, this approach could eventually be used to evaluate certain individual characteristics of the texts, which would provide an alternative way of identifying them through their organization across different scales. On the other hand, by introducing a Markov chain approach, we find that the sentiment state of a sentence depends on the immediately preceding one. This approach allows us to reveal the biases on the transitions to neutral and positive sentences in the evolution of the sentiment along the text. The narrative has sentiment scores that are correlated, but the story being told can vary in emotion and intensity from one sentence to the next; therefore, the sentiment evolution along the text could be mapped to a Markov chain. In other words, the long-term correlation approach allows to make a global assessment of the book sentiment, while the state transition matrix approach provides local information on the evolution of sentiment along the text. These local and global properties have been recognized as features present in systems that exhibit, at the same time, the presence of event “clustering” and long-term correlations [59,60,61]. The main limitation of this study lies in the sentiment values returned by VADER, where the compound sentiment construction may exhibit biases when analyzing content with sarcasm or other representations where the tool is not robust. Although these evaluations can be made more robust by considering other sentiment analysis tools, it remains a complex task. Further studies are needed to explore the feasibility of obtaining specific features of sentiment evolution in literary texts, and whether the levels of correlations are representative of the type of text being analyzed to determine properties such as literary genres, authors, etc.

6. Conclusions

We perform a sentence-level sentiment analysis of several literary works in English. This study has two approaches: (i) a global one through the assessment of the correlations in the sentiment along the text and (ii) a local one through the analysis of the transitions of sentiment from a sentence to the next. The global assessment, performed by applying the Higuchi Fractal Dimension and the Detrended Fluctuation Analysis methods, shows that the data series of sentiment scores is monofractal with long-term correlations, which can be explained by the fact that the writing process has memory by construction with a sentiment evolution that is self-similar. The local assessment, performed by counting the sentiment state transitions, shows that there are biases that make more likely the transitions to neutral and positive sentences, implying a bias towards positive sentiment in the texts. Previous studies based on word-level sentiment analysis have found a similar bias to positive sentiment [56]. These two approaches supplement each other because the long-term correlation approach allows a global assessment of the sentiment of the book, while the state transition matrix approach provides local information about the sentiment evolution along the text.