Implicit-Causality-Exploration-Enabled Graph Neural Network for Stock Prediction

Abstract

Accurate stock prediction plays an important role in financial markets and can aid investors in making well-informed decisions and optimizing their investment strategies. Relationships exist among stocks in the market, leading to high correlation in their prices. Recently, several methods have been proposed to mine such relationships in order to enhance forecasting results. However, previous works have focused on exploring the correlations among stocks while neglecting the causal characteristics, thereby restricting the predictive performance. Furthermore, due to the diversity of relationships, existing methods are unable to handle both dynamic and static relationships simultaneously. To address the limitations of prior research, we introduce a novel stock trend forecasting framework capable of mining the causal relationships that affect changes in companies’ stock prices and simultaneously extracts both dynamic and static features to enhance the forecasting performance. Extensive experimental results in the Chinese stock market demonstrate that the proposed framework achieves obvious improvement against multiple state-of-the-art approaches.

1. Introduction

The uncertainty of the stock market increases the difficulties in stock prediction, while the stock price is a critical indicator for finance practitioners to reasonably allocate their assets to gain profit [1,2]. In recent decades, forecasting stocks using deep learning methods has been of interest to researchers. In particular, the performance of graph neural network (GNN)-based methods surpasses that of most competitors due to their superior ability to capture a wider range of relational features leveraging the associations inherent in stock market data [3,4].

The tremendous success of GNN-based methods is mainly attributed to the research efforts dominated by the development of advanced GNN models. For instance, refs. [5,6,7] directly construct company relations, e.g., industry relation and shareholding relation, based on explicit knowledge in the market, and [8,9,10,11] explore the stock correlations, e.g., Pearson coefficient and cosine similarity, among stocks based on historical price information in the market. Subsequently, GNN-based models have been employed to aggregate the relational features associated with target stocks for prediction.

However, the performance of a GNN-based stock predictor is also closely related to the input graph structure, which is interpreted as the result of the relations between stocks [7]. In practice, the relation between stocks can be either explicit or implicit (more details can be referred to Section 4). Compared with explicit relations which have been widely employed in most existing GNN-based methods, most recently, growing interest in discovering the intrinsic causality of stock variation is motivating a shift toward implicit relation exploration across a number of GNN-based methods for stock prediction.

Powered by the pioneering techniques of GNN, the implicit relation exploration is usually manipulated by deep learning modules embedded in GNN models [8,10]. On the one hand, it suffers flaws in the absence of interpretability. On the other hand, as per the aforementioned analysis, the most pressing question is compounded by the fact that an implicit relation is still explored based on the input graph structure without the ability to discover the potential relations between non-connected nodes.

Although there exist studies conducting correlation-based measurements (e.g., Pearson correlation) in graph building for the GNN model [9,11], the correlation, by nature, indicates the presence of an association between stocks without establishing a deep causation between them. Worse still, little a priori work has accounted for the dynamic characteristics of implicit relations that can potentially affect the prediction performance. It is worth noting that Granger causality has been proven to be an effective mechanism for modeling stock market graphs [12,13]. Nevertheless, the directional attribute of a Granger causal graph, together with its dynamic characteristic, makes it difficult to incorporate it into the GNN model for stock prediction.

In light of the aforementioned challenges, we start with a discussion about the necessity of the Granger causal graph and use this insight in our proposed GNN model with the aim of capturing the causation rather than statistical association. Furthermore, we design a unified GNN-based framework, which is not limited to capturing explicit relations, but also implicit relations, no matter whether they are static or vary with time. The contributions of this article are drawn as follows:

• We propose a unified GNN-based stock predictor with the incorporation of explicit and implicit relations with the aim of making a tradeoff between the interpretability of structural relations and the power of black-box deep learning models.
• We design an equivalent undirected graph of Granger causality as a feed of the proposed dynamic and static fusion GNN model, named DSF-GNN, which captures both static and dynamic characteristics of implicit and explicit relations by leveraging two proposed modules, namely the static-relation-based feature extractor and dynamic-relation-based feature extractor.
• Through experiments on the Chinese stock market spanning over more than three years of data, our proposed model outperforms other counterparts, demonstrating a performance improvement of 2.63% to 6.76% in terms of accuracy.

2. Related Work

2.1. Euclidean-Input Based Methods

Tasks related to stock forecasting involve predicting future price trends or prices of stocks. Accurate predictions can significantly enhance investment decisions and ultimately lead to better returns. Due to the temporal nature of stock prices, traditional RNN networks and their variants, such as LSTM and GRU, can extract the timing feature of stocks, and they are widely used by researchers. Some studies have taken into account the long-term time dependence in the stock price sequence [14,15]. Subsequently, some methods further improved upon this approach by considering the varying contribution of features at different moments for stock prediction. Attention-based RNN methods are employed in stock forecasting tasks. For example, Qin et al. [16] adopted an attention-based LSTM to extract driving input signals and hidden states. Zhang, Aggarwal, and Qi [17] introduced a deep learning model utilizing LSTM to address the dynamic market factors in stock prediction. Feng et al. [18] utilized adversarial training to simulate stochasticity during model training. Md et al. [19] introduced a multi-layer sequential long short-term memory (MLS-LSTM) model designed for predicting stock index prices. The proposed approach involves normalizing time series data and segmenting it into time steps to assess the correlation between past and future values. Smith et al. [20] proposed an optimized LSTM-RNN model that enhanced investors’ ability to understand and operate in dynamic stock market environments, enabling better prediction of stock trends.

The stock prices are also influenced by other factors, such as macro factors, investor sentiment, company news, etc. Natural language processing methods and text mining techniques are the commonly employed approaches to process such data. Wang et al. [21] considered both stock sectors and relevant macroeconomic variables to facilitate predictions. Mittal and Goel and Kraus and Feuerriegel [22,23] utilized twitter data to analyze public investor sentiment. Li et al. [24] incorporated both quantitative indicators and sentiment extracted from sentiment dictionaries into their LSTM-based model for predicting stock prices. Liapis et al. [25] proposed a multivariate prediction method that combines time convolutional networks and BERT-based multi-label sentiment classification, using sentiment-related features to predict financial time series. Yang, Zhen, et al. [26] utilized multiple investor sentiment features to investigate the effectiveness of stock index prediction, and combined the boosting method to forecast stock trends. Ding et al. [27] employed the open information extraction method to extract company events from news texts and represent them as dense vectors for prediction. Furthermore, various studies have utilized sentiment lexicons to extract features from financial news for stock price prediction. Deng et al. [28] utilized a sentiment lexicon, specifically SentiWordNet 3.0, to assess the sentiment of news articles, which employed a multiple kernel learning (MKL) regression model, integrating sentiments and stock prices, to predict stock price movements. Chen et al. [29] integrated quantitative indicators with news features extracted using a sentiment dictionary into their RNN-boost model, resulting in improved stock prediction performance. However, while the Euclidean-input based methods exhibit impressive accuracy in forecasting, they have their limitations. The majority of the aforementioned studies primarily concentrated on integrating an individual stock’s historical data with additional textual information, while neglecting the interrelations among different stocks, particularly in capturing the relationship attributes between different stocks.

2.2. Graph-Based Methods

In recent years, a novel research direction has investigated graph-structured data to capture the interconnections between stocks, which can be divided into two types, explicit-based and implicit-based. Chen, Wei, and Huang [7] constructed a corporate graph using the shareholding relationships, transforming stock prediction into a node classification problem using GCNs. Sawhney et al. [30] utilized attentional graph neural networks to analyze the connections formed by social media texts and correlations among companies, facilitating a comprehensive understanding of market dynamics. Feng et al. [5] utilized two types of explicit relationships, namely industry or Wikipedia relationship, to construct a graph and enhance the GCNs with LSTM cells to model stock interconnections for prediction. Inspired by the remarkable success of attention mechanisms, refs. [31,32] employed two layers of GATs [31] to extract meaningful relational features and filter out irrelevant ones for movement trend prediction. Ye et al. [6] built multiple graphs to find the representations of cross effects. Zhang et al. [33] proposed a novel graph attention network (GAN) driven by dynamic attributes that combined sentiment (DGATS) information, trading data, and textual data. They captured real-time market dependencies and key attribute information through graph networks, dynamically updating relationships and relationship strengths within predefined graphs for stock prediction. Shi et al. [34] introduced prior knowledge on relationships among four types of stocks, constructed a graph based on internal information of trading features, and utilized an integrated GCN-LSTM approach for predicting stock price movements. Qian et al. [35] proposed a hierarchical multi-relational dynamic graph framework that utilized discrete dynamic graphs to comprehensively capture the multifaceted relationships between stocks and their temporal changes, aiming to simulate stock investment prediction.

Although superficial connections can be observed in explicit relationships, the graphs constructed by these methods were constrained by fixed, predefined corporate relationships, which heavily relied on pre-established external knowledge. As a result, captu deeper interconnections remained elusive. Therefore, various methods were employed to uncover implicit relationships among stocks. Hamilton, Ying, and Leskovec [36] proposed the GAT model in GraphSAGE, which was good at implicitly assigning different weights to different nodes in a neighborhood, avoiding the need for labor-intensive matrix operations or relying on prior knowledge of graph structure. Li et al. [11] constructed matrices of positive and negative correlations based on historical market prices to predict overnight stock movements. Cheng and Li [10] introduced an attribute-mattered aggregator to model the momentum spillovers among firms for movement prediction. Xu et al. [8] utilized cosine similarity to construct hidden concepts and extract concept-oriented shared information from both predefined and hidden concepts. Tian et al. [9] constructed the time-varying Pearson coefficient and Manhattan matrix to predict stocks’ return rate and movement trends, respectively. Xing et al. [37] gained understanding of implicit connections through attribute-guided vague graph comprehension, followed by the derivation of relational characteristics using GNN. Despite the progress achieved in implicit-relation-based graph neural network stock prediction models, their focus was mainly on identifying stock relations grounded in correlation rather than causality. To provide a detailed overview of the related methodologies discussed above,

Table 1. The comparison between related works.

	Reference	Relation Type	Pros	Cons
Euclidean-input-based	LSTM [14,15]	Dynamic	The time-based patterns in historical price indicators are effectively utilized.	Failing to recognize that the impacts of various historical trading days differ.
	MLS-LSTM [19]	Dynamic, implicit	Taking into account the varying impacts of different historical trading days.	Overlooking the aggregate effect of associated stocks.
	ALSTM [16]	Dynamic, implicit
	LSTM-RNN [20]	Dynamic, implicit
Graph-based	GCN [38]	Static, implicit	A stock graph efficiently reveals impacts from linked stocks.	A static graph cannot model the complex, dynamic relationships between stocks.
	TGC [5]	Static, explicit	Dynamically learn and capture interactions between stocks.	Using one graph for prediction limits integrating multiple stock graphs.
	ADGAT [10]	Dynamic, implicit
Graph-based	DGATS [33]	Static, explicit	Explores multifaceted stock relationships, capturing interactions single graphs may miss.	The main focus is on the correlation between related stocks, without fully exploring causal relationships.
	MGAR [39]	Dynamic, explicit
	MDGNN [35]	Dynamic, implicit

presents a comparison between these works, shows their respective pros and cons.

3. Preliminaries

3.1. Granger Causality

With the intention of determining whether a time series variable is advantageous to predict another, Granger causality is developed on the basis of the statistical hypothesis and has been widely adopted as an operational measure of causality in the field of econometrics. Mathematically, given a sequence of d-dimensional time series $X=\{X_1,X_2,\dots ,X_T\}$, $X_{t\in T}\in {\mathbb{R}}^d$. $X_t$ can be expressed as a linear combination of $\mathcal{T}$ previous lags of the whole time series:

$$X_t=\sum _{n=1}^{\mathcal{T}}{A}_n{X}_{t-n}+{\epsilon }_t$$

(1)

where $A_n\in {\mathbb{R}}^{N\times N}$ is a coefficient matrix with each element $A_n(i,j)$ denoting the degree of influence of time series j on i, while ${″}_t$ is Gaussian noise. The null hypothesis, that a time series is said to be not a Granger cause for another, is not rejected if the following condition is satisfied:

$$H_0:A_1(i,j)=A_2(i,j)=\cdots =A_{\mathcal{T}}(i,j)=0$$

(2)

3.2. Message Passing Based GNN

Message passing is a main technique used in most existing GNN models [5,40]. We use $G=(X,A)$ to represent a stock market graph, where $X\in {\mathbb{R}}^{N\times D}$ is an attribute matrix of N nodes, assuming each node has D attributes, while $A\in {\mathbb{R}}^{N\times N}$ denotes the adjacency matrix. The message passing is facilitated between nodes as follows:

$$H^{(l+1)}=f(A,H^{\left(l\right)};{\theta }^{\left(l\right)})$$

(3)

where $H^{(l+1)}\in {\mathbb{R}}^{N\times F}$ are the features of the node (for example, “messages”) computed after $l+1$ steps, $f(·)$ is the message propagation function, and ${\theta }^{\left(l\right)}$ denotes training parameters. There are multiple variants of the function $f(·)$. We follow the work of [38]:

$$f(A,H^{\left(l\right)};{\theta }^{\left(l\right)})=\mathrm{ReLU}({\tilde{D}}^{-{\displaystyle \frac{1}{2}}}\tilde{A}{\tilde{D}}^{-{\displaystyle \frac{1}{2}}}{H}^{\left(l\right)}{W}^{\left(l\right)})$$

(4)

where $\tilde{A}=A+I$, $\tilde{D}={\sum }_j{\tilde{A}}_{ij}$, and $W^{\left(l\right)}$ is a weight matrix. GNN enables a graph node (a node usually denotes a stock) to receive features sent from its $l_{th}$-order ($l\ge 1$) neighbors. Thus, the correlation between two distinct nodes could be explored and captured.

4. Methodology

We model the Granger causal graph of stocks using a multi-layer feed-forward neural network, called neural Granger causality (NGC) [41], for capturing Granger causality, followed by the proposed GNN-based predictor on the basis of moralization. Before illustrating more details, we first define the explicit and implicit relations as follows:

Definition 1.

Explicit relation: an explicit relation is directly constructed through observable financial domain knowledge without analysis, such as industrial relation and shareholding relation [5,40].

Definition 2.

Implicit relation: an implicit relation is extracted and analyzed from observable financial domain knowledge using techniques that can be further classified into linear/nonlinear measurement techniques, e.g., Pearson’s correlation [9].

In this paper, we consider industry, shareholding, and media information as explicit relations. Correspondingly, we take Granger causality as the implicit relation. In the following, we present the details about the Granger causal graph modeling, which is then fed to the proposed GNN model.

4.1. Granger Causal Graph Modeling

The power of Granger causality in time series analysis has been discussed in extensive studies [42,43]. In this paper, we implement Granger causality detection as a pre-task of stock prediction motivated by the fact that the Granger causality verifies whether one stock is useful to forecast another. Compared to correlation-based measures, existing investigation indicates that the Granger causality measure provides a much more stringent criterion for causation than simply observing a high correlation with some lag-lead relationship [13,44]. In addition, further research shows that standard correlation can mislead taxon interactions since correlation is a sign of causality, but it is not sufficient to infer causality [45].

For clarity of exposition, we implement Pearson correlation analysis and the Granger causality detection between two stocks. As shown in Figure 1, on the one hand, there is a weak Pearson correlation between the two stocks. On the other hand, no explicit relation is found between them, e.g., shareholding relations. In contrast, Figure 1 reveals that there exist contemporaneous and lagged relations between the two stocks at different time periods, allowing us to identify the underlying causality between these two stocks.

Unfortunately, Granger causality is unable to provide insight into the true causal relationship between two variables. The follow-on task of discovering true causal relations is assigned to our proposed GNN-based model. Unlike the directed graph generated based on the Pearson correlation, the difficulties of discovering Granger causality and deploying the causal graph as the input of a GNN model are as follows:

• Most classical approaches to Granger causality detection have the limitation of being implemented on a time series with nonlinear features, resulting in an inconsistent estimation of Granger causal interactions.
• Granger causality leads to the relation between two variables being directed rather than undirected, whereas the work so far on GNN models has focused primarily on undirected graphs [46]. Moreover, this is followed by another challenge, namely that a hybrid graph structure (directed and undirected graph structure)-enabled GNN model is necessary for our stock predictor.

To overcome the first challenge, we employ a nonlinear approach to investigate Granger causality among stocks and establish a dynamic Granger causality graph, $GC$, accordingly. Taking stock i as an example, our objective is to identify stocks that contribute to its modeling, thereby revealing those that Granger-cause stock i, which consists of two phases. First, we design an L-layer feed-forward neural network to model the closing price of stock i using the historical closing prices of N stocks from the stock pool, which is formulated as:

$$c_i^t={\mathcal{N}}_i\left({\mathbf{c}}_1^{t-T:t-1},{\mathbf{c}}_2^{t-T:t-1},\dots ,{\mathbf{c}}_N^{t-T:t-1}\right)+{\mathbf{e}}_t$$

(5)

where $c_i^t$ indicates the closing price of stock i on day t. The reason for choosing the closing price is that, compared to other price indicators, it intuitively reflects the trend of stocks. T is the time lag and ${\mathbf{e}}_t$ is the error term. Specifically, the parameters of ${\mathcal{N}}_i$ consist of the weights, $\mathcal{W}=({\mathcal{W}}^1,W^2,\dots ,W^L)$, and biases, $\mathbf{b}=(b^1,b^2,\dots ,b^L)$, of L layers. To enable interpretability, we partition the network ${\mathcal{N}}_i$ into two parts: the first layer and the remaining $L-1$ layers. At the first layer, the weight ${\mathcal{W}}^1=(W_1^{1,1:T},W_2^{1,1:T},\dots ,W_N^{1,1:T})\in {\mathbb{R}}^{N\times T\times H}$ of ${\mathcal{N}}_i$ represents the influence of each stock on stock i over historical T days. The embedding of the first layer is calculated as:

$${\mathbf{h}}_t^1=\mathrm{ReLU}\left(\sum _{k=1}^T\left(W^{1,k}{\mathbf{c}}_{1:N}^{t-k}+{\mathbf{b}}^1\right)\right)$$

(6)

It is noted that for $\forall j\in N$, if $W_j^{1,1:T}$ is a zero matrix, it indicates that stock j does not Granger-cause stock i, i.e., $G{C}_{j,i}^t=0$; otherwise, it indicates stock j Granger-causes stock i, i.e., $G{C}_{j,i}^t=1$. Subsequently, the closing price of stock i at time t is simulated:

$$c_i^t=\mathrm{FC}({\mathbf{h}}_t^1;\mathrm{\Theta })+e_i^t$$

(7)

where FC$(·)$ is the fully connected network with $L-1$ layers, $\mathrm{\Theta }$ is the training parameters, and $e_i^t$ is the error term.

We utilize the weights ${\mathcal{W}}^1$ of ${\mathcal{N}}_i$ in the first layer to simulate $A^{\left(k\right)}$ in Equation (1) of VAR. Thus, if the j-th matrix $W_j^1$ contains zeros for all time steps, that means stock j does not Granger-cause i. The group lasso penalty is applied to divide the weights of the first layer into N groups, penalizing the weights that are ineffective to zeros for each network. To identify the stocks that influence the ith stock, the loss function is as follows:

$$\mathcal{L}=\underset{W}{\min }{(c_i^t-{\mathcal{N}}_i\left(c_{1:N}^{t-T:t-1}\right))}^2+\lambda \sum _{j=1}^N{\Vert W_j^1\Vert }_F$$

(8)

where the hyperparameter $\lambda$ controls the degree of group sparsity, and ${\Vert ·\Vert }_F$ is the Froebenius matrix norm. The group lasso penalty uniformly penalizes the weights associated with all lags for stock j. Since Equation (8) is not differentiable, the conventional gradient descent method is unable to update the network parameters. Thus, proximal gradient descent is employed to address the optimization problems of objective functions that involve non-differentiable convex function terms. At each timestamp, a Granger causality graph, $G{C}^t$, is constructed to model Granger causality.

$$G{C}_{j,i}^t=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{stock}\mathrm{j}\mathrm{Granger}\mathrm{causes}\mathrm{stock}\mathrm{i}\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(9)

4.2. Moralization

To tackle the challenge posed by the limited ability of GNNs to directly process directed graphs, we have adopted a method that converts directed causal graphs into equivalent undirected moral graphs. This transformation ensures the compatibility of directed graphs with the utilization of GNNs. The transformation process consists of two steps: First, $\forall x_i\in GC$, connect every pair of $Parent\left(x_i\right)$, where $Parent\left(x_i\right)$ is the set of parent nodes of $x_i$. After that, replace the directed edges in $GC$ with undirected edges.

4.3. Model Architecture

The overall architecture is illustrated in Figure 2 and consists of the following components: time feature representation module (TFR), relationship feature encoding module (RFE), relationship feature fusion module (RFF), and the final prediction layer. The input stock time series data are processed through these modules to generate predicted movement trends for each stock. The primary functions of each component are summarized as follows:

• Time feature representation module (TFR): This module is devised to capture the temporal dependencies of historical stock prices on future fluctuations. We employ a classic time series analysis model, i.e., GRU, to extract the historical dynamic features of stocks, serving as input for subsequent modules.
• Relationship feature encoding module (RFE): This module is designed to capture relational dependencies among related stocks and contains two sub-modules: a dynamic-relation-based feature extractor (DRE) and a static-relation-based feature extractor (SRE). DRE aims to capture dynamic relational dependencies, whereas SRE focuses on capturing static relational dependencies among stocks.
• Relationship feature fusion module (RFF): To capture additional potential relational dependencies, RFF is designed to integrate both dynamic and static embeddings, generating a novel high-level stock representation.
• Prediction layer: Finally, after the feature fusion process, the fused embeddings are input into the prediction layer, a fully connected network that generates predicted movement trends for each stock.

4.3.1. Temporal Feature Representation

Stock prices are highly influenced by both their own historical indicators and those of other stocks. The TFR module aims to capture the influence of the stocks’ historical information on their own stock prices. For a given stock i, the historical price attributes, $X_i=({\mathbf{x}}_i^t,{\mathbf{x}}_i^{t-1},\dots ,{\mathbf{x}}_i^{t-T+1})\in {\mathbb{R}}^{T\times D}$, are utilized as the input data for the TFR module, where ${\mathbf{x}}_i^t\in {\mathbb{R}}^D$ denotes the attributes of stock i at time t, including opening price, closing price, high price, low price, and trading volume. The TFR module maps $X_i$ to higher-dimensional features, which consist of abundant historical information,

$$H_i,{\mathbf{o}}_i=\mathrm{RNN}\left(X_i\right)$$

(10)

where $H_i=({\mathbf{h}}_i^t,{\mathbf{h}}_i^{t-1},\dots ,{\mathbf{h}}_i^{t-T+1})\in {\mathbb{R}}^{T\times F_1}$ is the output sequence, ${\mathbf{o}}_i={\mathbf{h}}_i^t\in {\mathbb{R}}^{F_1}$ is the final state, and $F_1$ is the number of hidden units of RNN. In this work, we use a GRU network as the RNN, which has the remarkable ability to retain information in long-term sequences, in contrast to vanilla RNN.

4.3.2. Relational Feature Encoding

In this paper, we consider four types of stock relationships: industry relationship, shareholding relationship, media relationship, and Granger causality relationship. Among these, the industry relationship is static, while the other three relationships are dynamic. For static relationships, we can directly capture dependencies of stocks by aggregating features according to static stock graphs. However, in contrast to static relationships, dynamic relationships change over time. Thus, it is necessary to capture the historical daily relational dependencies and consider how the accumulation of these dependencies impacts future stock fluctuations. Thus, we devise DRE and SRE for extracting dynamic and static relational features, respectively.

DRE aims to capture the influence of features from various related stocks at different time steps and incorporate them to generate embedding with dynamic relational feature, according to various dynamic relations. We achieve a comprehensive representation of stock i by taking a weighted sum of the output sequence generated by TFR for all neighboring stocks at different times, considering the various dynamic relations.

$${\tilde{\mathbf{h}}}_i^{k,n}=\sigma \left(\sum _{j\in {\mathcal{G}}_k\left(i\right)}{\displaystyle \frac{1}{d_{ij}}}{W}_a^k{\mathbf{h}}_j^n+{\mathbf{b}}_a^k\right)$$

(11)

where $n\in \{t,t-1,\dots ,t-T+1\}$, $k\in \{1,2,..,P\}$, and P indicates the number of dynamic relations. ${\tilde{\mathbf{h}}}_i^{k,n}\in {\mathbb{R}}^{D_1}$, ${\mathcal{G}}_k\left(i\right)$ represents the set of stocks related to stock i for the k-th dynamic relation. $d_{ij}$ balances the influence of different neighbor stocks on stock i. $W_a^k\in {\mathbb{R}}^{F_1\times D_1}$ is the weight matrix, and ${\mathbf{b}}_a^k\in {\mathbb{R}}^{D_1}$ is the bias vector.

Subsequently, considering the temporal characteristics of dynamic features, we construct a temporal-attention-based GRU to integrate the relational embeddings of stocks at various time steps. First of all, the GRU generates a latent vector representation of historical features,

$${\tilde{\mathbf{D}}}_i^k=\mathrm{GRU}\left({\tilde{\mathbf{h}}}_i^{k,t},{\tilde{\mathbf{h}}}_i^{k,t-1},\dots ,{\tilde{\mathbf{h}}}_i^{k,t-T+1}\right)$$

(12)

where ${\tilde{\mathbf{D}}}_i^k=({\tilde{\mathbf{d}}}_i^{k,t},{\tilde{\mathbf{d}}}_i^{k,t-1},\dots ,{\tilde{\mathbf{d}}}_i^{k,t-T+1})\in {\mathbb{R}}^{T\times D_2}$. In order to capture the impact of historical features across different time steps, we incorporate a temporal attention layer. The output, $a_i^{k,t}$, of this layer dynamically assigns weights to the vector ${\tilde{\mathbf{d}}}_i^{k,t}$.

$$a_i^{k,t}={\displaystyle \frac{exp(\tanh ({\tilde{\mathbf{d}}}_{\mathrm{i}}^{\mathrm{k},\mathrm{t}}{\mathbf{w}}_{{\mathrm{a}}^{\prime }}+{\mathrm{b}}_{{\mathrm{a}}^{\prime }}))}{{\sum }_{n=t-T+1}^{t}exp(tanh({\tilde{\mathbf{d}}}_{\mathrm{i}}^{\mathrm{k},\mathrm{n}}{\mathbf{w}}_{{\mathrm{a}}^{\prime }}+{\mathrm{b}}_{{\mathrm{a}}^{\prime }}))}}$$

(13)

where ${\mathbf{w}}_{a^{\prime }}\in {\mathbb{R}}^{D_2}$ and $b_{a^{\prime }}\in \mathbb{R}$ are learnable parameters. After that, ${\tilde{\mathbf{d}}}_i^{k,t}$ is aggregated into a high-level vector representation. $a_i^{k,t}$ assigns weights to ${\tilde{\mathbf{d}}}_i^{k,t}$ at each time step from $t-T+1$ to t. As a result, we obtain the dynamic embedding ${\mathbf{d}}_i^t\in {\mathbb{R}}_d^D$ for each stock based on temporal attention:

$${\mathbf{d}}_i^t=\sum _{k=1}^P{\alpha }_k\left(\mathrm{ReLU}\left(\sum _{\mathrm{n}=\mathrm{t}-\mathrm{T}+1}^{\mathrm{t}}{\mathrm{a}}_{\mathrm{i}}^{\mathrm{k},\mathrm{n}}{\tilde{\mathbf{d}}}_{\mathrm{i}}^{\mathrm{k},\mathrm{n}}\right)\right)$$

(14)

where P is the number of dynamic relations, and ${\alpha }_k$ controls the weights of each different dynamic feature.

Based on the characteristics of multiple static relations, SRF aims to capture the static relational embedding. We employ an attention-based GCN network to capture the association strength between stocks according to the predefined static relations. Subsequently, we aggregate the features of the associated stocks, according to the correlation strength between stocks.

For $\forall i,j\in N$, to mine the association strength between a stock pair, we first employ a linear transformation method to extract the latent representations, respectively,

$${\mathbf{h}}_i^{\prime k}=W_b^k{\mathbf{o}}_i+{\mathbf{b}}_b^k,{\mathbf{h}}_j^{\prime k}=W_b^k{\mathbf{o}}_j+{\mathbf{b}}_b^k$$

(15)

where $k\in \{1,2,\dots ,Q\}$, and Q indicates the number of static relations. ${\mathbf{o}}_i$ and ${\mathbf{o}}_j$ are the the final state generated by TFR, and $W_b^k\in {\mathbb{R}}^{F_1\times S_1}$, ${\mathbf{b}}_b^k\in {\mathbb{R}}^{S_1}$ are learnable parameters. Subsequently, the vectors ${\mathbf{h}}_i^{\prime k}$ and ${\mathbf{h}}_j^{\prime k}$ are concatenated to form a new vector. The concatenated vector is linearly transformed, parameterized by a learnable weight vector ${\mathbf{a}}_s^k$. After the nonlinear transformation, the softmax function is employed to transform the association strength values to the range [0, 1].

$$e_{ij}^k={\displaystyle \frac{exp(\mathrm{LeakyReLU}\left({\mathbf{a}}_s^k[{\mathbf{h}}_i^{\prime k}\Vert {\mathbf{h}}_j^{\prime k}]\right)}{{\sum }_{l\in {\mathcal{G}}_k\left(i\right)}exp(\mathrm{LeakyReLU}\left({\mathbf{a}}_s^k[{\mathbf{h}}_i^{\prime k}\Vert {\mathbf{h}}_l^{\prime k}]\right)}}$$

(16)

According to correlation strength scores for each neighbor of stock i, we can obtain a more comprehensive representation by performing a weighted sum of the latent representations of all neighboring stocks.

$${\mathbf{s}}_i^t=\sum _k^Q\left({\beta }_k\mathrm{ReLU}\left(\sum _{\mathrm{j}\in {\mathcal{G}}_{\mathrm{k}}\left(\mathrm{i}\right)}{\mathrm{e}}_{\mathrm{ij}}^{\mathrm{k}}{\mathbf{h}}_{\mathrm{j}}^{\prime \mathrm{k}}\right)\right)$$

(17)

where ${\mathbf{s}}_i^t\in {\mathbb{R}}_s^D$ indicates static embedding, and ${\beta }_k$ controls the weights of each static dynamic feature.

4.4. Relational Feature Fusion

Subsequently, we aim to integrate the dynamic and static embeddings to form a new embedding, which preserves both the dynamic and static characteristics. The challenge is to effectively capture their interactions while preserving the independent features. To solve the above problems, we adopt a feature fusion method to capture potential interactions between ${\mathbf{d}}_i^t$ and ${\mathbf{s}}_i^t$, while extracting high-level features.

To analyze the inherent associations between ${\mathbf{d}}_i^t$ and ${\mathbf{s}}_i^t$, we perform multiple vector-matrix-vector (VMV) product, ${\mathbf{d}}_i^t{\mathcal{F}}_i^{[1:F^{\prime }]}{\mathbf{s}}_i^t$, where ${\mathcal{F}}_i^{[1:F^{\prime }]}\in {\mathbb{R}}^{F^{\prime }\times D_d\times D_s}$ indicates the $F^{\prime }$ learnable weight matrix. More specifically, for the kth matrix, we multiply it with ${\mathbf{d}}_i^t$ and ${\mathbf{s}}_i^t$ by VMV, resulting in an temporary result. By repeating this process, we obtain $F^{\prime }$ results, where each result represents a feature of the fused embedding.

To maintain their individual features, ${\mathbf{d}}_i^t$ and ${\mathbf{s}}_i^t$ are combined through concatenation, and linear transformation is performed using a weight matrix. Thus, the formula of RFF is as follows:

$${\mathbf{f}}_i^t=\tanh ({\mathbf{d}}_i^t{\mathcal{F}}_i^{[1:F^{\prime }]}{\mathbf{s}}_i^t+W_f[{\mathbf{d}}_i^t\Vert {\mathbf{s}}_i^t]+{\mathbf{b}}_f)$$

(18)

where ‖ indicates the concatenation, and $W_f\in {\mathbb{R}}^{(D_d+D_s)\times F^{\prime }}$ and ${\mathbf{b}}_f\in {\mathbb{R}}^{F^{\prime }}$ are learnable parameters.

4.5. Prediction Layer

The relational embedding of stock i, generated by the RFF module, serves as input for the prediction layer. The prediction layer is constructed with a fully connected (FC) neural network. For stock movement prediction, an FC layer with softmax function is more suitable for classification,

$${\widehat{\mathbf{y}}}_i^{t+1}=\mathrm{Softmax}(W_p{\mathbf{f}}_i^t+{\mathbf{b}}_p)$$

(19)

where ${\widehat{\mathbf{y}}}_i^{t+1}$ is the output for stock i from the prediction layer parameterized by $W\in {\mathbb{R}}^{F^{\prime }\times C}$, $b\in {\mathbb{R}}^C$, and C represents the number of classes. The standard cross-entropy loss function can be employed for backpropagation to train the proposed model.

5. Experiments

5.1. Data

We validate the effectiveness of our proposed model using a dataset obtained from Tushare, which provides extensive information about the Chinese stock market (https://www.tushare.pro/ accessed on 21 October 2024).

More than 4000 stocks are filtered. The screening criteria are as follows: Firstly, to ensure data completeness, stocks that were not suspended from 1 January 2019 to 4 January 2022 were selected. After that, we ensured that during this three-year period, the stock prices were all above 10 yuan, as stocks below 10 yuan generally entail higher investment risk and are considered as microcap stocks with limited investment value. Among those identified, precisely 600 companies met both of the above conditions. We aim to predict the stock movement based on historical indicators, including opening price, closing price, high price, low price, volume, and other useful information in the market. Thus, stock movement prediction can be considered as a binary classification problem:

$$y=\left\{\begin{array}{cc}1,\hfill & clos{e}_t^c\ge clos{e}_{t-1}^c\hfill \\ 0,\hfill & clos{e}_t^c<clos{e}_{t-1}^c\hfill \end{array}\right.$$

(20)

5.2. Baselines

To assess the performance of the proposed model, we compare it against two types of baselines: Euclidean-based and non-Euclidean-based methods.

5.2.1. Euclidean-Input-Based Methods

• Long short-term memory (LSTM) [47] is one of the most widely recognized recurrent neural network (RNN) models used for processing time series data.
• Gated recurrent unit (GRU) [48] is another model commonly used for processing time series data. In comparison to LSTM, GRU has a simpler structure, with only two gated units.
• The dual-stage attention-based recurrent neural network (ALSTM) [16] employs a recurrent neural network model with a two-stage attention mechanism, consisting of an input attention layer and a temporal attention layer.

5.2.2. Non-Euclidean-Input Based Methods

• The graph convolutional network (GCN) [38] updates features by aggregating the information of neighboring nodes.
• Temporal graph convolution (TGC) [5] makes the prediction by considering the sequential embedding and relational embedding.
• The multi-graph convolutional gated recurrent unit (Multi-GCGRU) [6] emphasizes the diversity of relations by constructing three types of graphs to enhance the representations of cross effects.
• Shared information for stock trend forecasting (HIST) [8] makes the prediction by combining predefined concepts, hidden concepts, and individual information.
• Multi-relational graph attention ranking (MGAR) [39] employs a graph aggregation network that concurrently incorporates multiple stock relation graphs as input to examine the interactions between stocks, particularly focusing on similarity relationships.

5.3. Metrics

Following previous stock movement prediction works [21,49], we take two metrics as reference: accuracy (ACC) and Matthews correlation coefficient (MCC). The detailed definitions of the metrics are as follows.

ACC measures the proportion of correct predictions among the total number of predictions made. In the context of stock prediction, it is commonly used to evaluate the model’s accuracy in predicting the direction of stock price movements. It is calculated as:

$$ACC={\displaystyle \frac{TP+TN}{TP+FP+FN+TN}}$$

(21)

where TP is the number of true positives, TN is the number of true negatives, FP is the number of false positives, and FN is the number of false negatives.

MCC is a measure of the quality of binary classifications, considering both true and false positives and negatives. In the realm of stock prediction, it serves as a valuable metric for assessing the overall performance of a model, taking into account its ability to correctly predict positive and negative movements in stock prices. It is calculated as:

$$MCC={\displaystyle \frac{TP\times TN-FP\times FN}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}}$$

(22)

Here, MCC ranges between −1 and +1, where +1 indicates perfect prediction, 0 indicates random prediction, and −1 indicates perfect inverse prediction.

5.4. Experimental Results

To demonstrate the effectiveness of the proposed model, we compare it with the above baselines in

Table 2. Performance comparison with baseline models.

	Methods	ACC	MCC
Euclidean-input-based methods	ALSTM	52.55	0.0613
	LSTM	52.56	0.0596
	GRU	52.72	0.0620
Non-Euclidean-input-based methods	GCN	51.49	0.0295
	Multi-GCGRU	52.95	0.0582
	TGC	53.15	0.0596
	HITS	53.56	0.0618
	MGAR	53.34	0.0602
	Proposed Model	54.97	0.0821

, and the following observations are made from the table: Our method achieves the highest ACC and MCC, outperforming some of the latest stock prediction methods, such as Multi-GCGRU, TGC, MGAR, and HIST. ALSTM, LSTM, and GRU, which solely consider the temporal dependencies among historical price information, achieve similar performance. The standalone use of GCN yields the lowest accuracy among non-Euclidean-input-based methods due to its inability to handle sequential data. Multi-GCGRU, TGC, and MGAR, which take into account both temporal dependency and stock relations, achieve superior performance compared to ALTSM, LSTM, GRU, and GCN. HIST achieve the second best performance, considering implicit relation by establishing a relation by aggregating stocks with similar hidden concept characteristics. However, this approach essentially discovers connections between stocks, overlooking the causal influence of other companies on the target company in prediction. However, our model captures not only the causal effects but also the dynamic and static characteristics of other predefined relations, while considering temporal dependency. Therefore, DSF-GNN outperforms existing stock trend forecasting methods. It is important to note that all models fail to perform well in terms of MCC, which can be attributed to the fact that Chinese stock data are characterized by significant volatility and high randomness. This high randomness is due to the influence of various factors such as macroeconomic factors, policy changes, and market sentiment, and existing research also supports this view [50].

5.5. Ablation Experiments

We conduct ablation experiments to verify the effectiveness of different modules in the DSF-GNN, including the TFR, DRF, SRF, and RFF modules. To study the effects of these modules, we conduct experiments where specific components are removed, and subsequently observe the results.

Table 3. Ablation experiments for verifying the effectiveness of different modules. TFR indicates temporal feature representation module; DRF and SRF represents dynamic relational feature extractor, respectively; and RFF is the relational feature fusion module.

TFR	DRF	SRF	RFF	ACC	MCC
✓	✓	✓	×	53.86	0.0659
✓	✓	×	×	53.58	0.0632
✓	×	✓	×	53.21	0.0564
×	✓	✓	✓	54.23	0.0712
✓	✓	✓	✓	54.97	0.0821

shows the comparitive models and the results. Some conclusions could be drawn, as follows:

• To assess the effectiveness of RFF, we remove it from DSF-GNN. Instead, we concatenate the outputs of DRF and SRF for prediction. The results demonstrate that incorporating the interaction between dynamic and static relational features, while preserving their independent characteristics, can enhance the prediction outcomes.
• To assess the effectiveness of DRF and SRF, we conduct experiments by removing the DRF and SRF modules, respectively. As one of the modules is removed, the RFF module is not utilized. The results demonstrate that DRF and SRF contribute to the improvement in prediction performance.
• Removing the TFR module would decrease the predictive performance. Therefore, the utilization of historical price indicators is crucial for stock trend forecasting.

5.6. Relations Comparison

In this part, we further investigate the effectiveness of different relations in stock prediction. We evaluate the performance of a relation by removing it from DSF-GNN. As shown in

Table 4. Relational comparison. Removing the shareholding, causal, media, and industry relation are indicated by w/o -S, w/o -C, w/o -M, and w/o -I, respectively.

	ACC	MCC
w/o -S	53.41	0.0592
w/o -C	53.87	0.0656
w/o -M	54.85	0.0801
w/o -I	53.58	0.0632
DSF-GNN	54.97	0.0821

, the performances are similar when removing shareholding relation, causal relation, and industry relation, respectively. The accuracy is highest when media relations are removed. This indicates media relations have limited usefulness in stock prediction. This can be explained by the lag in news dissemination and the authenticity of news content. The major shareholders, who hold a significant portion of the stocks, have access to firsthand news and can influence public opinion through the Internet. On the other hand, minority shareholders commonly lack the resources to obtain timely news, resulting in a lag in information acquisition. In some cases, when major shareholders intend to short-sell their own stocks, they can manipulate the market by generating positive news to attract small shareholders’ attention, selling their stocks to other shareholders, resulting in a subsequent decrease in stock price.

However, for shareholding relation, the information released by the company every quarter is genuine, as other companies can only become shareholders after purchasing the stocks of the target company. Due to the sector rotation effect in the stock market, investors often select stocks based on industries. As a result, stocks within the same industry typically exhibit similar trends over a certain period of time. Therefore, industry relations are also of great importance.

5.7. Conclusions

In our research, we introduced a framework aimed at uncovering the causal relationships influencing changes in a company’s stock price, demonstrating its potential advantages in stock forecasting. First, we proposed the neural Granger causality model to identify Granger causal relationships between stocks, followed by a GNN-based model. We differentiated between explicit relations, including industry, shareholding, and media information, and implicit relations, such as Granger causality. Second, we introduced moralization to convert directed causal graphs into equivalent undirected moral graphs, making them compatible with GNNs. This model includes a time feature representation module and a relationship feature model, used respectively to capture the influence of historical information on a stock’s own price and to explore dynamic and static influences influenced by other stocks. Third, we presented a relationship feature fusion module that integrates dynamic embeddings and static embeddings to form a new feature that retains both dynamic and static characteristics. This new feature is used to effectively capture interactions between them while preserving independent features. Overall, our proposed approach addresses the challenges in discovering true causal relations between stocks, leveraging Granger causality modeling, and enhancing the predictive power of GNNs in stock movement prediction. Through extensive experimentation and analysis, our model demonstrates promising results and opens avenues for further research in this domain.