Self-Similar Growth and Synergistic Link Prediction in Technology-Convergence Networks: The Case of Intelligent Transportation Systems

Abstract

Self-similar growth and fractality are important properties found in many real-world networks, which could guide the modeling of network evolution and the anticipation of new links. However, in technology-convergence networks, such characteristics have not yet received much attention. This study provides empirical evidence for self-similar growth and fractality of the technology-convergence network in the field of intelligent transportation systems. This study further investigates the implications of such fractal properties for link prediction via partial information decomposition. It is discovered that two different scales of the network (i.e., the micro-scale structure measured by local similarity indices and the scaled-down structure measured by community-based indices) have significant synergistic effects on link prediction. Finally, we design a synergistic link prediction (SLP) approach which enhances local similarity indices by considering the probability of link existence conditional on the joint distribution of two scales. Experimental results show that SLP outperforms the benchmark local similarity indices in most cases, which could further validate the existence and usefulness of the synergistic effect between two scales on link prediction.

1. Introduction

Technology innovation is the major competitiveness of enterprises and an essential driving force for the high-quality development of the national economy. The innovation of technology could be broadly classified into two categories: (1) technology substitution, which refers to the innovative breakthrough of new technologies that replace the existing ones, and (2) technology convergence, which refers to the transfer and combination of existing knowledge among multiple areas of technology [1,2]. The former is a linear, step-by-step approach along with challenging difficulties and lengthy transition periods, whereas the latter is a nonlinear and complementary method of technology fusion that brings with it additional cooperation and inspiration. Technology convergence is regarded as an increasingly significant characteristic of current trends of technology innovations due to more realization paths and faster outcome emergence [3,4,5].

A prominent research strand of technology convergence is its anticipation, which aims to predict potential technology convergence or identify convergence movements at an early stage [6,7,8]. In many studies, the emergence of new technology convergence is defined as the first co-occurrence of two International Patent Classification (IPC) subclasses in the patent data [9,10], and the anticipation is formulated as a link-prediction problem on an IPC co-occurrence network [11,12,13]. Two main objectives of the literature for anticipation of technology convergence are developing new methods and their practical applications [8]. From the perspective of network science, these anticipation approaches could be categorized into three categories: (1) topology-based [9,11,12,13,14], (2) ensemble-learning-based [7,15,16], and (3) multi-modal-based [17,18,19]. These methods are successfully applied in a wide range of industries, such as power systems [15], smartphones [11], electric vehicles [13], manufacturing [14], and telecommunications [17].

Despite the success of applying link-prediction algorithms to the anticipation of technology convergence, as noted in a recent milestone review by Sick et al. [8], there is relatively little research on theoretical underpinnings for such applications, especially from the perspective of network science. A major research topic that has not yet been adequately explained is particularly addressed in this study: As a time-evolving network, what evolutionary process may the technology-convergence network undergo? This question is extremely essential for the anticipation of technology convergence, since studying the evolution process of a network could help us understand the growth mechanism of its links and hence develop better link-prediction algorithms [20,21,22,23,24,25].

To answer the above question, we employed the theory of fractal networks as a tool for analysis. According to the dynamic evolutionary process, fractal networks could be generated by self-similar growth [26,27,28]. From the viewpoint of static topological structure, typical properties of fractal networks include self-similarity and scale invariance [29,30,31,32]. Using historical patent data in the ITS field, this study first provides empirical evidence for the self-similar growth of the time-evolving technology-convergence network based on the commonly adopted rescaled network statistics [28,33,34,35,36,37] and then verifies the fractality of each snapshot using a community-structure-based approximation of the box-cover algorithm [38].

After discovering the fractal property of the technology-convergence network, another intriguing research question emerges: How does the microscopic structure of the original technology-convergence network and its scaled-down replica provide information for the prediction of future technology convergence? This research question has some significant theoretical implications, since it could give insights into the multi-scale modeling and prediction of technology convergence, which has not yet been thoroughly studied.

As an attempt to answer the issue raised, this study adopts the partial information decomposition (PID) framework for the link-prediction problem. In particular, we discovered that the IPC hierarchy naturally provides community partitions for the original network, based on which the scaled-down network could be obtained. We further define a microscopic variable and a macroscopic variable based on two-hop link predictors and IPC hierarchy, respectively, to represent the two scales of the network. With PID, the joint contribution of the original and the corresponding scaled-down networks is decomposed into four components: the synergistic and redundant information between two scales, and the unique information provided by each scale. It is found that the joint contribution is mainly in the form of synergistic information and much redundant information.

The synergistic effect of two different scales on link prediction implies that the joint distribution of the two scales could be more informative than the marginal distributions of each scale. To further validate the existence and usefulness of the synergistic effect, we designed a synergistic link prediction (SLP) approach, which is essentially the probability of link existence on the condition of both the microscopic and the macroscopic variables. We adopted a cumulative distribution function (CDF)-like formula to score the likelihood of link existence between two nodes to address the sparsity issue of the network, while ensuring that the link-prediction score is monotonically increasing with the likelihood of link existence. SLP could be categorized as the link/community-based strategy [39,40,41,42,43,44,45,46,47], which enhances local similarity measures based on community information (see Section 2.4). Compared with existing link/community strategies, SLP could be used more easily in a plug-and-play manner, since it does not require manual adjustment of the formulae for local similarity measures.

Experiments were conducted by using the ITS-related patent data from November 1995 to July 2022. A time-evolving technology-convergence network was constructed on the basis of the IPC co-occurrence relationships obtained from the historical patent applications. Further, the new technology convergence of the next year is anticipated based on annual snapshots. Experimental results show that, in most cases, our designed SLP algorithm outperforms the benchmark algorithms that only consider one of the two scales. As a very interesting phenomenon, the link-prediction performance is highly consistent with PID results. The PID framework provides an appropriate theoretical underpinning for deeply interpreting the link-prediction performance. In most cases where SLP outperforms the benchmark algorithms, synergistic information also dominates the interaction between two scales of the network in the PID results. In the few cases where SLP has not managed to outperform the benchmark algorithms, significant redundant or unique information can be observed in the PID results. These results imply that the performance improvement provided by multi-scale fusion for link prediction is primarily due to synergy between different scales, and the presence of significant redundant or independent information may weaken the performance and necessitates special treatment. Our findings shed light on the development of multi-scale link-prediction algorithms for the anticipation of technology convergence and demonstrate that PID could be a powerful tool for exploring the evolutionary mechanisms of networks.

In summary, the main contributions of this study could be summarized as follows:

• We provide empirical evidence for the fractal characteristic of the technology- convergence network in the ITS field. In terms of the time-evolving mechanism, the technology-convergence networks grow in a self-similar paradigm, in which the rescaled topological properties remain stable. In terms of spatial properties, the annual snapshots of the technology-convergence networks are identified as fractal networks.
• We discovered that the structural information at two different scales has a synergistic effect on link prediction. The structural information at two different scales is measured by local similarity measures and community-based indices, respectively. Therefore, this discovery implies that the joint distribution of the two could be more informative than the marginal distributions of either the local or the community-based indices.
• We designed a link-prediction approach, namely, the SLP approach, based on the joint conditional probability of link existence given both the local and the community-based indices. Experimental results show that the SLP approach could enhance the corresponding local similarity measures by incorporating community structures, which further validates the existence and usefulness of the synergistic effect on link prediction between two scales.

The rest of this article is organized as follows. Theoretical background and related work for the research in this paper are provided in Section 2. Section 3 details the methodology of this study, including the empirical criteria for self-similar growth and fractality, the use of PID, and our developed SLP algorithm. Section 4 presents experimental results obtained from applying the new methodology and the benchmark algorithms. Finally, Section 5 presents the conclusions drawn from this research and directions for future research.

2. Related Work

The aims of this study were to analyze properties of the technology-convergence network based on fractal network theory and PID, and to propose a link-prediction algorithm for the anticipation of future technology convergence. In this section, related work in the following four research areas is discussed: technology convergence and its anticipation, fractal analysis of complex networks, PID, and link prediction.

2.1. Technology Convergence and Its Anticipation

The concept of technology convergence could be traced back to Rosenberg’s study on the technology changes in the machine tool industry in the 1960s [48]. However, little attention was paid to technology convergence until the 1980s, when information and communication technology (ICT) was extended to and integrated with various other industrial sectors [49,50,51]. In 1992, Kodama’s milestone of an article [49] symbolized that technology convergence began to attract attention. Some prior studies on convergence emerged in the 2000s, including theoretical definitions of convergence [52], the distinction between demand-side and supply-side convergence [53], and the partitioning of the four stages (i.e., science, technology, market, and industry) of the convergence process [52]. This study follows the theoretical definition of convergence as, “The blurring of boundaries between at least two hitherto disjoint areas” [54], and addresses the stages of technology convergence from a supply-side perspective.

Data-driven research on technology convergence has gained appeal since the 2010s, as publicly available data have drastically multiplied. An important data source for technology convergence studies is patent data, which are considered an up-to-date and reliable indicator of creative activities and knowledge accumulation within technology fields [55,56,57,58]. In many early studies of technology convergence, citation relationships among patents were used to measure technology convergence [55,59,60]. However, a recent argument is that such citation-based measurements focus on modeling the stretching process of the knowledge from one technology field to another and may not clearly characterize the events that represent the occurrence of technology convergence [10]. The emergence of new technology convergence could be clearly defined as the first co-occurrence of two IPC subclasses in the patent data based on the International Patent Classification (IPC) of patents [9,10]. This definition of technology convergence was widely adopted in subsequent studies [5,13,17,61,62,63].

Benefiting from the IPC-based definition of the technology convergence, several studies anticipating prospective technology convergence based on IPC co-occurrence networks have emerged in recent years [9,13,17]. This branch of research constructs networks based on IPC co-occurrence relationships and forecasts potential technology convergence as missing links in the networks. A wide range of studies are based on topological similarity indices, such as the weighted resource allocation (RA) index [13], the weighted common neighbor (WCN) index [11], the Adamic-Adar index [9], and modified versions of these indices. Another stream of research applies ensemble approaches to link prediction, which employs supervised classification models to learn combinations of several individual link predictors [7,15,16]. A novel research paradigm is to develop machine learning algorithms that predict missing links based on a combination of multi-modal information, such as topology, bibliometric, and semantic information [17,18,19].

Despite the successful implementations of the proposed methods in a variety of industries, there is a paucity of research on the theoretical underpinnings of anticipating technology convergence [8]. In particular, studies that analyze the properties and evolutionary processes of technology-convergence networks from a network science perspective are very scarce. Hence, the key contributions of this paper are considering the technology-convergence network as a time-evolving network and providing its three properties: (1) the self-similar growth of the evolutionary process, (2) the fractality of the snapshots, and (3) the synergy between different scales in link prediction.

2.2. Fractal Analysis of Complex Networks

The complex network provides a powerful theoretical tool for the abstract characterization of many real-world systems composed of various objects and mutual relationships, such as technological [13,64,65], biological [66,67,68,69], and social systems [70,71,72,73,74]. A wide range of real-world networks have the fractal property, which could be roughly described as, “The network looks similar under different magnification levels” [75].

The rigorous definition of fractality in a complex network is based on the box-covering approach. Given a box of size $l_{\mathrm{B}}$, the box-covering approach groups a set of nodes into a box $b_r$ such that the shortest distance between each pair of nodes in $b_r$ is less than $l_{\mathrm{B}}$. By partitioning all the nodes into boxes, a network is covered, and the minimum number of boxes required is denoted as $N_{\mathrm{B}}\left(l_{\mathrm{B}}\right)$. The circumstance where a network is covered by $N_{\mathrm{B}}\left(l_{\mathrm{B}}\right)$ boxes is referred to as the optimal covering. If the scaling of $N_{\mathrm{B}}\left(l_{\mathrm{B}}\right)$ of the optimal covering follows a power law as

$$N_{\mathrm{B}}\sim l_{\mathrm{B}}^{-d_{\mathrm{B}}},$$

(1)

then the network is identified as a fractal network and $d_{\mathrm{B}}$ is denoted as its fractal dimension.

The optimal covering problem is known to be NP-hard [38,76]. Many heuristic algorithms have been proposed to solve this problem approximately, such as the greedy coloring algorithm [76], the compact-box-burning algorithm [76], the max-excluded mass-burning algorithm [76], the overlapping-box-covering algorithm [77], and the MCWR algorithm [78]. For a comparative analysis of the aforementioned box-covering algorithms, see the study in [79].

Most existing box-covering algorithms are designed for unweighted networks, whereas the technology-convergence network constructed in this study is a weighted network. For the box-covering problem of weighted networks, Wei et al. [80] proposed a method that transforms edge weights using a power function. Nevertheless, the partitioning of boxes in this method is affected not only by $l_{\mathrm{B}}$, but also by the exponent of the power function, which increases the difficulty of modifying parameters in actual use. Hence, selecting a proper exponent in the practical application remains a problem. Another recent step was the community-structure-based method proposed by Giudicianni et al. [38], which approximates the box-covering problem with community detection algorithms. Since there are many off-the-shelf algorithms for community detection on weighted networks, this method provides many reliable options for the coarse-graining of weighted networks. Therefore, we chose to apply this community-structure-based approach to analyze the fractal property of the technology-convergence network.

The inverse process of the box-covering algorithm mirrors the evolution of fractal networks to some extent [26,75,81]. For a time-evolving network, if its growth mechanism approximates the inverse process of the box-covering algorithm, the snapshot at time $t-1$ could be considered as a scaled-down replica of the snapshot at time t. As a result, we could observe self-similar behaviors of the network statistics on the rescaled snapshots. Commonly adopted empirical evidence for the self-similar growth is the curve overlapping of rescaled network statistics, including rescaled degree distribution, rescaled clustering coefficient, rescaled degree–degree correlation, and the community structure [28,33,34,35,36,37]. Following the convention in the literature, this study provides empirical evidence for the self-similar growth of the time-evolving technology-convergence network based on the aforementioned rescaled network statistics.

2.3. Partial Information Decomposition

Information theory, pioneered by Claude Shannon [82], is widely used to analyze the interactions between components in complex systems. However, the commonly applied mutual information only concerns the interaction between two variables, as it measures the information provided by a single variable about another. Instead, this study considers a three-way system involving the original technology-convergence network and its scaled-down replica as two inputs and the future technology convergence as an output.

A recent extension of information theory, the partial information decomposition (PID), provides a powerful tool for analyzing the three-way system with two inputs, $X_1$ and $X_2$, and one output, Y [83]. Specifically, the information that $X_1$ and $X_2$ jointly provide for Y is defined as

$$I\left(Y;X_1,X_2\right)=\sum _{x_1}{p}_{{}_{Y,X_1,X_2}}(y,x_1,x_2)\left[log\frac{1}{p_{{}_Y}\left(y\right)}-log\frac{1}{p_{{}_{Y\mid X_1,X_2}}(y\mid x_1,x_2)}\right].$$

(2)

The joint mutual information $I\left(Y;X_1,X_2\right)$ is further decomposed into four terms:

$$\begin{array}{cc}\hfill I\left(Y;X_1,X_2\right)& =I_{\mathrm{red}}\left(Y;X_1,X_2\right)+I_{\mathrm{unq}}\left(Y;X_1\mid X_2\right)\hfill \\ \hfill & +I_{\mathrm{unq}}\left(Y;X_2\mid X_1\right)+I_{\mathrm{syn}}\left(Y;X_1,X_2\right),\hfill \end{array}$$

(3)

where $I_{\mathrm{red}}$, $I_{\mathrm{unq}}$, and $I_{\mathrm{syn}}$ are named redundant information, unique information and synergistic information, respectively.

$I_{\mathrm{red}}$ is defined as “the expected value of the minimum information provided by either $X_1$ or $X_2$ about each possible outcome of Y” [83], which is formally written as

$$I_{\mathrm{red}}\left(Y;X_1,X_2\right)=\sum _y{p}_{{}_Y}\left(y\right)min\left\{I\left(Y=y;X_1\right),I\left(Y=y;X_2\right)\right\}.$$

(4)

Given the definition of $I_{\mathrm{red}}$, the other three terms could be induced as

$$\begin{array}{cc}\hfill I_{\mathrm{unq}}\left(Y;X_1\mid X_2\right)& =I\left(Y;X_1\right)-I_{\mathrm{red}}\left(Y;X_1,X_2\right),\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill I_{\mathrm{unq}}\left(Y;X_2\mid X_1\right)& =I\left(Y;X_2\right)-I_{\mathrm{red}}\left(Y;X_1,X_2\right),\hfill \\ \hfill I_{\mathrm{syn}}\left(Y;X_1,X_2\right)& =I\left(Y;X_1,X_2\right)-I_{\mathrm{unq}}\left(Y;X_1\mid X_2\right)\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill & -I_{\mathrm{unq}}\left(Y;X_2\mid X_1\right)-I_{\mathrm{red}}\left(Y;X_1,X_2\right).\hfill \end{array}$$

(7)

The practical meanings of these four terms are as follows:

• $I_{\mathrm{red}}\left(Y;X_1,X_2\right)$ represents the information that both $X_1$ and $X_2$ could provide for Y.
• $I_{\mathrm{unq}}\left(Y;X_1\mid X_2\right)$ represents the information about Y that could be provided by $X_1$ but not by $X_2$.
• $I_{\mathrm{unq}}\left(Y;X_2\mid X_1\right)$ represents the information about Y that could be provided by $X_2$ but not by $X_1$.
• $I_{\mathrm{syn}}\left(Y;X_1,X_2\right)$ represents the information about Y that could only be obtained by jointly considering both $X_1$ and $X_2$.

PID has been applied in various research fields, such as neuroscience [84,85,86], artificial intelligence [87,88,89,90], and emergent behavior recognition and analysis [91,92,93]. Among all these, the most relevant is the information-theoretic framework for causal emergence proposed by Rosas et al. [91].

In the proposed information-theoretic framework, a complex system composed of n microscopic states ${\mathbf{X}}_t=\left[X_t^{\left(1\right)},X_t^{\left(2\right)},\right.$$\left.\cdots ,X_t^{\left(n\right)}\right]$ is taken into account. It is assumed that the system conforms to Markovian stochastic dynamics with the transition probability $p_{{}_{{\mathbf{X}}_{t^{\prime }}\mid {\mathbf{X}}_t}}$, where t and $t^{\prime }$ are two time points satisfying the condition $t<t^{\prime }$. An emergent behavior is deemed to have occurred if some macroscopic states $V_t$ of the system could provide unique information beyond that provided by microscopic states ${\mathbf{X}}_t$ separately. The unique information provided by $V_t$ is denoted as $I_{\mathrm{unq}}\left(V_t;{\mathbf{X}}_{t^{\prime }}\mid {\mathbf{X}}_t\right)$, and $V_t$ demonstrates causal emergence if

$$I_{\mathrm{unq}}\left(V_t;{\mathbf{X}}_{t^{\prime }}\mid {\mathbf{X}}_t\right)>0,$$

(8)

(see Definition 2 in [91]).

By analogy with the above information-theoretic framework, the original technology-convergence network corresponds to the microscopic state, and its scaled-down replica corresponds to the macroscopic state. However, defining microscopic and macroscopic variables is not a trivial or intuitive task. The microscopic variables of a system should not contain information about the high-order interactions among its components, and the definition of macroscopic variables relies on manually designed coarse-graining functions. Since there is no universal methodology for defining microscopic variables and the coarse-graining functions for deriving macroscopic variables, this study contributes to the PID literature by proposing definitions of microscopic variables and coarse-graining functions in complex networks from the perspective of link prediction. This study could be considered as an attempt to apply PID to the field of link prediction in complex networks. In addition, from a machine learning perspective, the link-prediction method proposed in this study could be viewed as a PID-inspired feature fusion algorithm. Compared with previous studies on PID-inspired feature engineering [88,89], this study proposes a novel feature fusion method based on the cumulative distribution function (CDF).

2.4. Link Prediction

Link prediction, a basic issue in the study of complex networks, aims to predict unobserved missing links based on the the observed current network structure. The formal settings of link prediction are given as follows.

An observed network $G\left(\mathbf{N},\mathbf{E}\right)$ is considered, where $\mathbf{N}$ and $\mathbf{E}$ represent the node set and the edge set, respectively. It is assumed that $G\left(\mathbf{N},\mathbf{E}\right)$ is an undirected network without multi-edges and self-loops. The number of nodes is denoted as $N=\left|\mathbf{N}\right|$, and the number of the observed edges is denoted as $E=\left|\mathbf{E}\right|$. The adjacency matrix of $G\left(\mathbf{N},\mathbf{E}\right)$ is denoted as $\mathbf{A}=\left[a_{ij}\right]$, which is an $N\times N$ real symmetric matrix with $a_{i,i}=0$. The entry $a_{ij}=0$ indicates that no link is observed between node i and node j—i.e., $(i,j)\notin \mathbf{E}$, whereas $a_{ij}>0$ indicates that a link is observed between node i and node j with a weight of $a_{ij}$. Note that only the undirected weighted network with positive edge weights is considered in this study.

In such a network, the total number of node pairs is $\frac{N(N-1)}{2}$, and the number of unconnected node pairs is $\frac{N(N-1)}{2}-E$. We use $\mathbf{U}$ to denote the set of unconnected node pairs. The link-prediction problem assumes that there are unobserved or future links between some node pairs $(i,j)\in \mathbf{U}$, the set of which is denoted as ${\mathbf{E}}^{\prime }$. The task of link prediction is to find the missing links in ${\mathbf{E}}^{\prime }$ based on the observed network $G\left(\mathbf{N},\mathbf{E}\right)$. Generally, a link-prediction algorithm f, also known as a link predictor, computes a score $s_{ij}^f$ for each node pair $(i,j)\in \mathbf{U}$ based on the adjacency matrix $\mathbf{A}$, which could also be written in a matrix form as

$${\mathbf{S}}^f={\left[s_{ij}^f\right]}_{N\times N}=f\left(\mathbf{A}\right).$$

(9)

The node pairs with higher scores are the most likely to be the missing links.

The existing link predictors could be broadly classified into three categories: (1) similarity-based indices, which score node pairs according to their topological similarities [94,95,96,97]; (2) maximum likelihood methods, where some parametric network models are presumed and the parameters of the models are estimated by maximizing the likelihood of the observed network structure [98,99,100]; (3) machine-learning-based methods, in which the link-prediction problem is regarded as a binary classification problem and is solved through machine learning algorithms (e.g., embedding learning [101,102,103], ensemble learning [104,105,106], and graph neural networks [107,108,109]). For the latest advances in the field of link prediction, two seminal reviews were published in 2020 [110] and 2021 [111].

The similarity-based indices are the most widely used approaches in the anticipation of technology convergence [9,11,12,13,14], mainly due to their low computational complexity and high interpretability. Widely applied similarity-based indices include the common neighbor (CN) index [94], the preferential attachment (PA) index [112], the Jaccard (JC) index [95], the resource allocation (RA) index [96], and the Adamic-Adar (AA) index [97]. Since the technology-convergence networks are essentially weighted networks, the weighted similarity-based indices have attracted much attention, such as the weighted resource allocation (WRA) index [13] and the weighted common neighbor (WCN) index [11]. Other weighted similarity-based indices that could be applied or extended for technology-convergence networks include the weighted Jaccard coefficient (WJC) [113], the weighted preferential attachment (WPA) index [113], the selectivity index [114], and the inverse selectivity index [114]. In particular, Martinčić-Ipšić et al. [114] provided a clear and comprehensive example of how to design and evaluate similarity-based indices for real-world weighted networks.

The above-mentioned mainstream methods could be summarized as local similarity measures, since they mainly exploit the information of 2-hop paths in complex networks. However, due to the hierarchical structure of the IPC system, technology-convergence networks naturally have a priori community structures (i.e., IPC subclasses belonging to the same IPC class can be considered as nodes from the same community), which could provide information beyond local similarity measures. Therefore, link prediction indices that jointly consider the local similarity and the community structure could be particularly suited for the anticipation of technology convergence.

In the link prediction literature, the methods that enhance local similarity measures based on community information are known as the link/community-based strategies [39,40,41,42,44,45,47]. A pioneering work in this research was [39], which was based on the assumption that two nodes are more likely to connect if their common-first-neighbors form a strongly inner-linked cohort (i.e., a local-community). Based on this assumption, the CN, JC, AA, and RA indices were extended to the CAR, CJC, CAA, and CRA indices. Subsequently, Ding et al. [41] proposed a link/community-based method that is suitable for sparse networks. The method first performs community detection based on local information only, and then scores each node pair based on the relevance between the communities the two nodes belong to. The community relevance indices, namely, CRCN, CRJC, CRAA, and CRRA, could be regarded as community-based extensions of the CN, JC, AA, and RA indices. Other methods that enhance local similarity measures based on community structures include [40,42,44,45,47]. Other seminal studies that exploit community structures to address the sparsity issue are [43,46], which propose cluster-based meta-paths for the link-prediction problem in multiplex networks.

From the perspective of link prediction, the SLP method proposed in this study could be regarded as a link/community-based strategy that enhances local similarity measures by incorporating community structures. The motivation and novelty of the SLP approach originate from our discovery of the synergistic information between local similarity measures and community structures in the technology-convergence network of ITS. It is the existence of the synergistic information that makes the joint distribution of the two more informative than either of the marginal distributions. Most of the existing link/community-based strategies require manual adjustment of the formulae for local similarity measures. In comparison, one advantage of the SLP method is that it could be used as a plug-and-play method without the need to manually design new computational formulas. In addition, the community information that could be integrated by the SLP method is not limited to link density. Other community-based indices (e.g., CRCN, CRJC, CRAA, and CRRA) can be directly applied to the SLP method to enhance the local similarity measures.

Moreover, the SLP approach proposed in this study is quite intuitive, lacking a careful design for the formula of the link prediction index. In particular, the sparsity of the links in real-world networks could cause serious issues for the estimation of probabilities, although this study tries to tackle this issue, at least partially, via a CDF-like formula. In the future, we would like to extend this study from the perspective of link prediction. First, we would like to examine whether the synergy between community structures and local similarity measures is widespread in real-world networks. Second, a better-designed link/community-based strategy could be proposed to make greater use of the synergistic effect. Third, extensive comparisons with different link/community-based strategies could be conducted in a broad diversity of real-world networks.

3. Methodology

This section outlines the methodology of this study, which includes four steps: (1) constructing the technology-convergence network based on IPC co-occurrence relationships obtained from patent data; (2) empirically analyzing the self-similar growth and fractality of the technology-convergence network; (3) applying the PID framework to the link-prediction problem to analyze the contributions of the original and the scaled-down networks to the anticipation of technology convergence; (4) proposing a synergistic link-prediction algorithm to predict the future technology convergence. Each of these steps is described in detail in the following four subsections, and the research framework and main findings are shown in Figure 1.

3.1. Constructing Technology-Convergence Networks

The first step in constructing a technology-convergence network for a particular industry is obtaining the relevant patents. Patent data were collected from the Derwent Innovation database, a widely adopted tool for patent analysis [13,115,116], in this study. Then, a search string was created based on the keywords of the industry, and all the query results were exported. The retrieved patent data included the title, application number, application date, IPC code, etc.

Subsequently, the annual technology-convergence network was constructed based on the collected patent data. All the IPC subclasses appearing in the patent data were first extracted as nodes in the node set in the technology-convergence network. Each IPC subclass is represented by a 4-digit IPC code (IPC4) in the patent data. An annual IPC co-occurrence network was then derived based on the patent applications filed each year. For a patent application assigned to two IPC subclasses, i and j, in year t, an IPC co-occurrence $(i,j)$ will be recorded. For a patent with three IPC subclasses i, j, and k, three additional IPC co-occurrences are recorded as $(i,j)$, $(j,k)$, and $(i,k)$. By scanning all the patent applications in year t, the annual IPC-occurrence network $G_t^{\mathrm{annl}}$ is constructed. The adjacency matrix of $G_t^{\mathrm{annl}}$ is denoted as ${\mathbf{A}}_t^{\mathrm{annl}}=\left[a_{ij}^{\mathrm{annl}}\left(t\right)\right]$, where $a_{ij}^{\mathrm{annl}}\left(t\right)>0$ indicates the times of co-occurrences between i and j in year t, and $a_{ij}^{\mathrm{annl}}\left(t\right)=0$ indicates no co-occurrence in year t.

Finally, the historically cumulative IPC co-occurrence network $G_t^{\mathrm{hist}}$ is constructed with adjacency matrix defined as ${\mathbf{A}}_t^{\mathrm{hist}}={\sum }_{\tau =1}^t{\mathbf{A}}_{\tau }^{\mathrm{annl}}$. From the perspective of the time-evolving network, $G_t^{\mathrm{hist}}$ is the snapshot of the technology-convergence network taken at the end of year t. With all the snapshots stacked in a chronological order, the time-evolving technology-convergence network could be represented by a sequence of snapshots as ${\mathcal{G}}^{\mathrm{hist}}=\left\{G_1^{\mathrm{hist}},G_2^{\mathrm{hist}},\cdots ,G_T^{\mathrm{hist}}\right\}$. The following analyses in this study were conducted based on ${\mathcal{G}}^{\mathrm{hist}}$. The complete procedure for the technology-convergence network construction is depicted in Figure 2.

3.2. Empirical Analysis of Self-Similar Growth and Fractality

In this subsection, the growth mechanism of the IPC co-occurrence network based on fractal analysis is examined. This study first identifies a self-similar growth process in network evolution, and then analyzes the fractality of the historical data. The self-similar growth provides the inspiration to predict the future evolution of the network based on the historical data, and the fractality of the historical data offers a motivation to explore the interaction between the original network and its scaled-down replica. The above findings inspire the PID analysis for link prediction in the next subsection.

The self-similar growth of a time-evolving network is characterized by the fact that its topological property remains in a steady state while its average degree increases moderately over time [28]. Based on the analysis framework widely adopted in the previous literature [28,33,34,35,36,37], this study first illustrates the evolution of the average degree in ${\mathcal{G}}^{\mathrm{hist}}$, and then demonstrates the stability of five topological properties: (1) complementary cumulative distribution of the rescaled degree ${\mathbb{P}}_{\mathrm{c}}\left(d^{\mathrm{res}}\right)$; (2) clustering coefficient over the rescaled degree $\overline{C}\left(d^{\mathrm{res}}\right)$; (3) normalized average degree of neighbors over the rescaled degree ${\overline{d}}_{\mathrm{nn}}^{\mathrm{res}}\left(d^{\mathrm{res}}\right)$; (4) modularity Q of the community partitions by the Louvain method; (5) adjusted mutual information (AMI) between the community partitions of two consecutive snapshots. The detailed definitions of the average degree and the five rescaled network statistics are presented in Appendix A.

The concept of self-similarity is closely related to fractality [29,117]. In particular, Figure 6a shows that the rescaled degree of the snapshots in ${\mathcal{G}}^{\mathrm{hist}}$ approximately follows a power-law distribution, which could be regarded as an evidence of fractal scaling. Therefore, this study further explores the fractality of each snapshot in ${\mathcal{G}}^{\mathrm{hist}}$.

To analyze the fractality of the technology-convergence network, we employed a recently proposed community-structure-based approximation of the box-covering algorithm [38], whose detailed procedures are presented in Appendix B. The basic idea of the community-structure-based box-covering algorithm is to perform community detection $\eta$ times by setting $\eta$ different resolutions for the Louvian algorithm. The average community size obtained from each time of community detection is denoted as $l_{\mathrm{B}}$, and the corresponding number of communities is denoted as $N_{\mathrm{B}}$. Based on the sequences ${\mathbf{l}}_{\mathrm{B}}=\left[l_{\mathrm{B}}\right]$ and ${\mathbf{N}}_{\mathrm{B}}=\left[N_{\mathrm{B}}\right]$, the relationship between $l_{\mathrm{B}}$ and $N_{\mathrm{B}}$ could be investigated. If the scaling of $N_{\mathrm{B}}\left(l_{\mathrm{B}}\right)$ over $l_{\mathrm{B}}$ approximately follows a power-law distribution as Equation (1), the snapshot $G_t^{\mathrm{hist}}$ could be identified as a fractal network, and $d_{\mathrm{B}}$ is referred to as its fractal dimension.

3.3. Link-Prediction-Motivated Partial Information Decomposition for Technology-Convergence Networks

The fractality of complex networks reveals the structural similarity between the original network and its scaled-down replica. This study uses PID to investigate the implication of such self-similarity, in particular, to answer the following research question: How does the microscopic structure of the original network and its scaled-down replica provide information for link prediction? Intuitively, the structural self-similarity implies the existence of redundant information between the original network and its scaled-down replica. In addition, the scaling-down process may result in information loss, allowing the original network to provide unique information for link prediction. By adopting the PID framework to the link-prediction problem, this paper quantifies such redundant, unique, and synergistic information, and analyzes the role that the scaled-down replica plays in link prediction. Figure 3 provides a overview of the link-prediction-motivated PID framework for the technology-convergence network.

In this subsection, microscopic variables represent the microscopic structure of the original network, and macroscopic variables represent the scaled-down replica. Subsequently, the link-prediction formulation is presented for the anticipation of future technology convergence and is incorporated into the PID framework. The terminology used in this subsection (e.g., microscopic variables and macroscopic variables) follows the convention in the PID-related literature [91].

3.3.1. Definitions of Microscopic Variables Based on Two-Hop Link Predictors

A key issue for the definitions of microscopic variables is to ensure that they do not contain information beyond microscopic structures in the network. This study considers the most fundamental microscopic structure, i.e., the two-hop paths in the network. From the perspective of link prediction, two-hop paths could be quantified by the scores of two-hop link predictors such as CN, AA, and RA. Therefore, microscopic variables in the network are defined based on the scores of two-hop link predictors. The detailed procedure is given as follows.

For a snapshot whose adjacency matrix is denoted as $\mathbf{A}$, first, a score matrix $\mathbf{S}=\left[s_{ij}\right]$ is calculated based on $\mathbf{A}$ using a two-hop-based link predictor $f(·)$, which is formally written as

$$\mathbf{S}=\left[s_{ij}\right]=f\left(\mathbf{A}\right).$$

(10)

Then, the score $s_{ij}$ for a randomly chosen node pair $(i,j)$ is considered as a random variable, and each element in the matrix ${\mathbf{S}}^f$ is a sample of $s_{ij}$. The random variable $s_{ij}$ is defined as the microscopic variable in the technology-convergence network.

It is worth noting that the link predictor f is restricted to be based only on two-hop paths to filter out the information provided by the paths with three hops or more. This study considers six commonly used two-hop link predictors for weighted networks, which are listed in

Table 1. Six local similarity measures based on two-hop paths.

Names	Definitions
Weighted common neighbor (WCN) index [118]	$s_{ij}^{\mathrm{WCN}}={\sum }_{k\in |\Gamma (i)\cap \Gamma (j\left)\right|}\left(a_{i,k}+a_{j,k}\right)$
Weighted Adamic–Adar (WAA) index [118]	$s_{ij}^{\mathrm{WAA}}={\sum }_{k\in |\Gamma (i)\cap \Gamma (j\left)\right|}\frac{a_{i,k}+a_{j,k}}{ln\left(1+d_k\right)}$
Weighted resource allocation (WRA) index [118]	$s_{ij}^{\mathrm{WRA}}={\sum }_{k\in |\Gamma (i)\cap \Gamma (j\left)\right|}\frac{a_{i,k}+a_{j,k}}{d_k}$
Reliable-route weighted common neighbor (rWCN) index [119]	$s_{ij}^{\mathrm{rWCN}}={\sum }_{k\in |\Gamma (i)\cap \Gamma (j\left)\right|}\left(a_{i,k}·a_{j,k}\right)$
Reliable-route weighted Adamic–Adar (rWAA) index [119]	$s_{ij}^{\mathrm{rWAA}}={\sum }_{k\in |\Gamma (i)\cap \Gamma (j\left)\right|}\frac{a_{i,k}·a_{j,k}}{ln\left(1+d_k\right)}$
Reliable-route weighted resource allocation (rWRA) index [119]	$s_{ij}^{\mathrm{rWRA}}={\sum }_{k\in |\Gamma (i)\cap \Gamma (j\left)\right|}\frac{a_{i,k}·a_{j,k}}{d_k}$

. To simplify the expression, the time t is omitted, and the adjacency matrix is denoted as $\mathbf{A}$. In Table 1, $\Gamma \left(i\right)$ and $\Gamma \left(j\right)$ denote the neighbor sets of nodes i and j. The element in the i-th row and k-th column of $\mathbf{A}$ is denoted as $a_{i,k}$. Note that $a_{i,k}$ is a positive integer rather than a binary variable, since the networks are weighted. The definition of $d_k$ is given in Equation (A2).

3.3.2. Definitions of Macroscopic Variables Based on Coarse-Graining

Macroscopic variables are obtained by a scaling-down (i.e., coarse-graining) procedure of the network, which consists of two parts: (1) a many-to-one mapping from the nodes of the original network to those of the scaled-down network; (2) a function that computes the properties of the scaled-down network based on the original network. The following part first discusses the reasons why existing coarse-graining methods in the fractal analysis may not be fully applicable to the technology-convergence network, then identifies the hierarchical structure of IPC as a natural way for coarse-graining the network. Finally, a coarse-graining function is defined based on the edge density of the original network.

A critical challenge for the coarse-graining procedure is the possible incomplete connection of the technology-convergence network. In fact, the evolution of the technology-convergence network is accompanied by the addition of new nodes (i.e., technology areas). If these new nodes are ignored and only the maximum connected component of a snapshot $G^{\mathrm{hist}}$ is considered, some important potential technology convergence may be omitted. Therefore, all the nodes for each year need to be considered to avoid such omissions. However, this may result in an incomplete connection of $G^{\mathrm{hist}}$ and many unconnected nodes.

Most existing box-covering algorithms have not elaborately dealt with the unconnected nodes. A conventional approach is to assign each unconnected node to a random box (e.g., Algorithm 2 in [30]). However, due to large amounts of unconnected nodes existing in the technology-convergence network (i.e., the IPC co-occurrence network), especially in the early stage of ITS development, completely random coarse-graining of unconnected nodes, which may entail much original network information loss, would lack some feasibility in this study.

The hierarchical structure of the IPC system naturally provides a criterion for coarse-graining the IPC co-occurrence network, which consists of four levels: section, class, subclass, and group. For an IPC subclass represented by a 4-digit IPC code (IPC4), the first digit and the first three digits in IPC4 represent its section and class, respectively. For example, for the IPC subclass “G06F”, “G” stands for its section “PHYSICAL”, and “G06” represents its class “COMPUTING; CALCULATING OR COUNTING”. As mentioned previously in Section 3.1, IPC subclasses are selected as nodes in the IPC co-occurrence network in this study. By grouping all the nodes (i.e., IPC subclasses) belonging to the same IPC class into a box $b_r$, the original network is naturally coarse-grained. The number of boxes $N_{\mathrm{B}}$ equals the number of IPC classes that appear in ITS-related patents.

From another perspective, the coarse-grained (i.e., scaled-down) network $G^{\mathrm{CG}}$ could be viewed as a network constructed with IPC classes as nodes. However, unlike the network constructed directly based on IPC classes, each node and edge in $G^{\mathrm{CG}}$ contains a property calculated by a coarse-grained function $F_{\mathrm{CG}}(·)$ based on the original network $G^{\mathrm{hist}}$. To simplify the expression of macroscopic variables, a scalar variable $B_{rs}$ is used to represent the properties of each node ($r=s$) or edge ($r\ne s$) in the coarse-grained network $G^{\mathrm{CG}}$. The definition of $F_{\mathrm{CG}}(·)$ is given in the following paragraph.

The coarse-graining function $F_{\mathrm{CG}}(·)$ is a deterministic function that maps the $N\times N$ matrix $\mathbf{A}$ to a $N_{\mathrm{B}}\times N_{\mathrm{B}}$ matrix. The vector $\mathbf{g}=\left[g_1,g_2,\cdots ,g_N\right]$ denotes the mapping relationship from nodes of the original network to nodes of the coarse-grained network, where $g_i=r$ indicates that node i in the original network is covered by box $b_r$, and thereby mapped to node r in the coarse-grained network. Then, $F_{\mathrm{CG}}(·)$ is defined as

$$B_{rs}=F_{\mathrm{CG}}(\mathrm{A},r,s)=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^N𝟙\left(a_{ij}>0,g_i=r,g_j=s\right)}{{\sum }_{i=1}^N{\sum }_{j=1}^N𝟙\left(g_i=r,g_j=s\right)},$$

(11)

where $𝟙(·)$ is the indicator function. The denominator and numerator of Equation (11) represent, respectively, the numbers of node pairs $(i,j)$ and edges $(i,j)$ in the original network with node i covered by box $b_r$ and node j covered by box $b_s$.

Finally, $B_{rs}$ is selected as the macroscopic variable, which represents the value of a randomly chosen entry $(r,s)$ in the matrix $\mathbf{B}=\left[B_{rs}\right]$. Without loss of generality, this study assumes that all the edge weights in the original network are positive. Therefore, with Equation (11), $B_{rr}$ represents the density of edges in box $b_r$ of the original network, and $B_{rs}$ is the density of edges between box $b_r$ and $b_s$.

3.3.3. Problem Formulation and Partial Information Decomposition

In this study, the anticipation of technology convergence is formulated as a link-prediction problem. The potential technology convergence relationship is defined as the IPC co-occurrence which is nonexistent before time t but will appear at a future time point $t^{\prime }$. The problem is simplified by restricting $t^{\prime }=t+1$. For each year t, the information contained in $G_t^{\mathrm{hist}}$ is used to predict the links in $G_{t+1}^{\mathrm{hist}}$ that do not exist in $G_t^{\mathrm{hist}}$. These links are referred to as the missing links in year t. The set of missing links in year t is denoted as an $N\times N$ matrix ${\mathbf{A}}_t^{\mathrm{miss}}=\left[a_{ij}^{\mathrm{miss}}\left(t\right)\right]$, where $a_{ij}^{\mathrm{miss}}\left(t\right)$ is formally defined as

$$a_{ij}^{\mathrm{miss}}\left(t\right)=\left\{\begin{array}{cc}& 1,\mathrm{if}{a}_{ij}^{\mathrm{hist}}\left(t\right)=0\mathrm{and}{a}_{ij}^{\mathrm{hist}}(t+1)>0,\hfill \\ \hfill & 0,\mathrm{otherwise}.\hfill \end{array}\right.$$

(12)

This study aims to analyze how the microscopic structure of the original network $s_{ij}$ and its scaled-down replica $B_{g_i{g}_j}$ provide information for the prediction of future technology convergence $a_{ij}^{\mathrm{miss}}$. Therefore, the PID of the joint mutual information $I\left(a_{ij}^{\mathrm{miss}};s_{ij},B_{g_i{g}_j}\right)$ is considered and given as

$$\begin{array}{cc}\hfill I\left(a_{ij}^{\mathrm{miss}};s_{ij},B_{g_i{g}_j}\right)& =I_{\mathrm{red}}\left(a_{ij}^{\mathrm{miss}};s_{ij},B_{g_i{g}_j}\right)+I_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};s_{ij}\mid B_{g_i{g}_j}\right)\hfill \\ \hfill & +I_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\mid s_{ij}\right)+I_{\mathrm{syn}}\left(a_{ij}^{\mathrm{miss}};s_{ij},B_{g_i{g}_j}\right),\hfill \end{array}$$

(13)

where $I_{\mathrm{red}}$, $I_{\mathrm{unq}}$, and $I_{\mathrm{syn}}$ are the redundant information, unique information, and synergistic information, respectively.

First, the redundant information, $I_{\mathrm{red}}$, is calculated based on Equation (4) as

$$\begin{array}{cc}\hfill I_{\mathrm{red}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)& =p(a_{ij}^{\mathrm{miss}}=1)\times min\left(I(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j}),I(a_{ij}^{\mathrm{miss}}=1,s_{ij})\right)\hfill \\ \hfill & +p(a_{ij}^{\mathrm{miss}}=0)\times min\left(I(a_{ij}^{\mathrm{miss}}=0,B_{g_i{g}_j}),I(a_{ij}^{\mathrm{miss}}=0,s_{ij})\right),\hfill \end{array}$$

(14)

where $min(·)$ chooses the smaller of the two types of mutual information, and $I(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j})$ is defined as

$$I(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j})=\sum _{B_{g_i{g}_j}}p(B_{g_i{g}_j}\mid a_{ij}^{\mathrm{miss}}=1){log}_2\frac{p(a_{ij}^{\mathrm{miss}}=1\mid B_{g_i{g}_j})}{p(a_{ij}^{\mathrm{miss}}=1)},$$

(15)

(see Equations (1)–(3) in [83]). The other three types of mutual information in (14) follow similar definitions as (15).

The unique information provided by $B_{g_i{g}_j}$ beyond $s_{ij}$ is given as

$$I_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\mid s_{ij}\right)=I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\right)-I_{\mathrm{red}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right),$$

(16)

where $I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\right)$ denotes the mutual information of $a_{ij}^{\mathrm{miss}}$ and $B_{g_i{g}_j}$,

$$\begin{array}{cc}\hfill I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\right)& =\sum _{B_{g_i{g}_j}}p(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j})log\frac{p(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j})}{p(a_{ij}^{\mathrm{miss}}=1)p\left(B_{g_i{g}_j}\right)}\hfill \\ \hfill & +\sum _{B_{g_i{g}_j}}p(a_{ij}^{\mathrm{miss}}=0,B_{g_i{g}_j})log\frac{p(a_{ij}^{\mathrm{miss}}=0,B_{g_i{g}_j})}{p(a_{ij}^{\mathrm{miss}}=0)p\left(B_{g_i{g}_j}\right)}.\hfill \end{array}$$

(17)

Similarly, the unique information provided by $s_{ij}$ beyond $B_{g_i{g}_j}$ is given as

$$I_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};s_{ij}\mid B_{g_i{g}_j}\right)=I\left(a_{ij}^{\mathrm{miss}};s_{ij}\right)-I_{\mathrm{red}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right).$$

(18)

Furthermore, the synergistic information is calculated as

$$I_{\mathrm{syn}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)=I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)-I\left(a_{ij}^{\mathrm{miss}};s_{ij}\right)-I_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\mid s_{ij}\right),$$

(19)

which quantifies the joint contribution of $B_{g_i{g}_j}$ and $s_{ij}$ to link prediction, and can be further formulated as

$$\begin{array}{cc}\hfill I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)& =\sum _{B_{g_i{g}_j}}\sum _{s_{ij}}p(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j},s_{ij})log\frac{p(a_{ij}^{\mathrm{miss}}=1\mid B_{g_i{g}_j},s_{ij})}{p(a_{ij}^{\mathrm{miss}}=1)}\hfill \\ \hfill & +\sum _{B_{g_i{g}_j}}\sum _{s_{ij}}p(a_{ij}^{\mathrm{miss}}=0,B_{g_i{g}_j},s_{ij})log\frac{p(a_{ij}^{\mathrm{miss}}=0\mid B_{g_i{g}_j},s_{ij})}{p(a_{ij}^{\mathrm{miss}}=0)}.\hfill \end{array}$$

(20)

Since the missing links are rather sparse at each time point t, $p(a_{ij}^{\mathrm{miss}}=1)$ will be very small, and $p(a_{ij}^{\mathrm{miss}}=0)$ will be close to one. As a result, the value of mutual information will be quite small. To highlight the relative contribution of each PID component, this study proposes three relative PID measures, namely, the relative redundant information, the relative unique information, and the relative synergistic information, which are defined as the corresponding PID component normalized by the mutual information $I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)$. The formal definitions of the relative PID measures are given as

$${\rho }_{\mathrm{red}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)=\frac{I_{\mathrm{red}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)}{I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)},$$

(21)

$${\rho }_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\mid s_{ij}\right)=\frac{I_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\mid s_{ij}\right)}{I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)},$$

(22)

$${\rho }_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};s_{ij}\mid B_{g_i{g}_j}\right)=\frac{I_{\mathrm{unq}}\left(a_{ij}^{\mathrm{miss}};s_{ij}\mid B_{g_i{g}_j}\right)}{I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)},$$

(23)

$${\rho }_{\mathrm{syn}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)=\frac{I_{\mathrm{syn}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)}{I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)}.$$

(24)

For convenience of following expression, the above four measures will be simplified to ${\rho }_{\mathrm{red}}$, ${\rho }_{\mathrm{unq}}\left(B\mid s\right)$, ${\rho }_{\mathrm{unq}}\left(s\mid B\right)$, and ${\rho }_{\mathrm{syn}}\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)$, respectively. Notice that ${\rho }_{\mathrm{red}}+{\rho }_{\mathrm{unq}}\left(B\mid s\right)+{\rho }_{\mathrm{unq}}\left(s\mid B\right)+{\rho }_{\mathrm{syn}}=1$.

As the final essential issue in PID, the estimate of mutual information will be discussed. The mutual information is estimated using the most direct binning-based approach in this study. $\mathcal{B}=\left[min\left(\mathbf{B}\right),max\left(\mathbf{B}\right)\right]$ and $\mathcal{S}=\left[min\left(\mathbf{S}\right),max\left(\mathbf{S}\right)\right]$, which denote the ranges of values of $B_{ij}$ and $s_{ij}$, are divided into sets of bins of finite size. All the probabilities used for the calculation of mutual information are estimated by counting the number of samples falling into the bins [120,121]. $n_B^{+}\left(x\right)$ and $n_B^{-}\left(x\right)$ represent the numbers of positive samples (i.e., node pairs $(i,j)$ with $a_{ij}^{\mathrm{miss}}=1$) and negative samples (i.e., node pairs $(i,j)$ with $a_{ij}^{\mathrm{miss}}=0$) falling into the x-th bin of $\mathcal{B}$. Since there are $\frac{N(N-1)}{2}$ node pairs in an undirected network,

$$\begin{array}{cc}\hfill & p(a^{\mathrm{miss}}=1,B_{g_i{g}_j})=\frac{2·n_B^{+}\left(x\right)}{N(N-1)},\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill & p\left(B_{g_i{g}_j}\right)=\frac{2·\left(n_B^{+}\left(x\right)+n_B^{-}\left(x\right)\right)}{N(N-1)},\hfill \end{array}$$

(26)

where the value of $B_{g_i{g}_j}$ falls into the x-th bin of $\mathcal{B}$—i.e., $B_{g_i{g}_j}\in {\mathrm{bin}}_x$.

We could further calculate $p(B_{g_i{g}_j}\mid a^{\mathrm{miss}}=1)$ and $p(a^{\mathrm{miss}}=1\mid B_{g_i{g}_j})$ based on Equations (25) and (). Note that $p(a^{\mathrm{miss}}=1\mid B_{g_i{g}_j})$ is set to 0 when its denominator is zero. Other distributions, such as $p\left(s_{ij}\right)$ and $p(s_{ij},a_{ij}^{\mathrm{miss}}=0)$, could be estimated in the same way, so their detailed expressions are omitted.

The joint distribution is estimated by counting the intersection of samples in the x-th bin of $\mathcal{B}$ and the y-th bin of $\mathcal{S}$. The number of positive samples falling into the intersection of the bins x and y is denoted as $n_{B,s}^{+}\left(x,y\right)$, which is given as

$$n_{B,s}^{+}\left(x,y\right)=\sum _{i=1}^N\sum _{j=1}^N𝟙\left(a_{ij}^{\mathrm{miss}}>0,B_{g_i{g}_j}\in {bin}_x,s_{ij}\in {bin}_y\right).$$

(27)

The number of negative samples in the intersection of x and y is denoted as $n_{B,s}^{-}\left(x,y\right)$. The joint probability $p(a_{ij}^{\mathrm{miss}},B_{g_i{g}_j},s_{ij})$ is estimated as

$$\begin{array}{cc}\hfill & p(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j},s_{ij})=\frac{2·n_{B,s}^{+}\left(x,y\right)}{N(N-1)},\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill & p(a_{ij}^{\mathrm{miss}}=0,B_{g_i{g}_j},s_{ij})=\frac{2·n_{B,s}^{-}\left(x,y\right)}{N(N-1)}.\hfill \end{array}$$

(29)

By plugging the aforementioned distributions into Equations (15), (17) and (20), the mutual information of $I(a_{ij}^{\mathrm{miss}}=1,B_{g_i{g}_j})$, $I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j}\right)$, and $I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)$ is obtained. Other mutual information, such as $I(a_{ij}^{\mathrm{miss}}=0,B_{g_i{g}_j})$ and $I\left(a_{ij}^{\mathrm{miss}};s_{ij}\right)$, can be obtained using a similar approach. Based on the mutual information estimated above, the relative PID measures defined in this research, ${\rho }_{\mathrm{red}}$, ${\rho }_{\mathrm{unq}}\left(B\mid s\right)$, ${\rho }_{\mathrm{unq}}\left(s\mid B\right)$, and ${\rho }_{\mathrm{syn}}$, can finally be calculated.

3.4. Synergistic Link-Prediction Approach for the Anticipation of Technology Convergence

Based on the PID of $I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)$ in Equation (13), it can be observed that if $B_{g_i{g}_j}$ could provide unique information beyond $s_{ij}$ or synergistic information in conjunction with $s_{ij}$, we have

$$I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)>I\left(a_{ij}^{\mathrm{miss}};s_{ij}\right),$$

(30)

which implies that considering both $B_{g_i{g}_j}$ and $s_{ij}$ may achieve better performance than considering $s_{ij}$ alone. In this study, experimental results show that the contribution of $B_{g_i{g}_j}$ mainly takes the form of the synergistic information. Therefore, a link-prediction algorithm is proposed to combine the information provided by $B_{g_i{g}_j}$ and $s_{ij}$, which is named the synergistic link prediction (SLP) approach.

Consider a snapshot $G_t^{\mathrm{hist}}$ at time t. The mutual information could be written as $I\left(a_{ij}^{\mathrm{miss}}\left(t\right);B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right)\right)$. From Equation (20), we discover that $p(a_{ij}^{\mathrm{miss}}\left(t\right)=1\mid B_{g_i{g}_j}\left(t\right)$, $s_{ij}\left(t\right))$ naturally provides a link-prediction score for each node pair. For a node pair $(i,j)$, a higher $p(a_{ij}^{\mathrm{miss}}\left(t\right)=1\mid B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right))$ means that a missing link is more likely to exist.

However, it is worth noting that ${\mathbf{a}}^{\mathrm{miss}}\left(t\right)$ is unknown at time t, since $a_{ij}^{\mathrm{hist}}(t+1)$ can only be observed at time $t+1$ according to the definition in Equation (12). Therefore, the joint probability $p(a_{ij}^{\mathrm{miss}}=1\mid B_{g_i{g}_j},s_{ij})$ cannot be estimated directly at time t.

To solve this problem, we introduce the historically cumulative missing links, which is an $N\times N$ matrix ${\mathbf{A}}_t^{\mathrm{train}}=\left[a_{ij}^{\mathrm{train}}\left(t\right)\right]$ defined as

$$a_{ij}^{\mathrm{train}}\left(t\right)=\left\{\begin{array}{cc}& 1,\mathrm{if}\sum _{\tau =1}^{t-1}{a}_{ij}^{\mathrm{miss}}\left(\tau \right)>0,\hfill \\ \hfill & 0,\mathrm{otherwise}.\hfill \end{array}\right.$$

(31)

The matrix ${\mathbf{A}}_t^{\mathrm{train}}$ is the aggregation of all the historical missing links ${\mathbf{A}}_{\tau }^{\mathrm{miss}}$ from $\tau =1$ to $\tau =t-1$. Since ${\mathbf{A}}_t^{\mathrm{train}}$ does not contain any future information, it could be used as our training labels. Based on the binning-approach described in the previous subsection, $p(a_{ij}^{\mathrm{train}}\left(t\right)=1\mid B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right))$ is estimated as

$$p(a_{ij}^{\mathrm{train}}=1\mid B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right))=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^N𝟙\left(a_{ij}^{\mathrm{train}}\left(t\right)=1,B_{g_i,g_j}\left(t\right)\in {bin}_x,s_{ij}\left(t\right)\in {bin}_y\right)}{{\sum }_{i=1}^N{\sum }_{j=1}^N𝟙\left(B_{g_i,g_j}\left(t\right)\in {bin}_x,s_{ij}\left(t\right)\in {bin}_y\right)}.$$

(32)

We set $p(a_{ij}^{\mathrm{train}}=1\mid B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right))=0$ if the denominator is zero, which indicates that no sample falls into the intersection of the bin x and y.

It seems that an $N\times N$ score matrix could be constructed based on $p(a_{ij}^{\mathrm{train}}=1\mid B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right))$ to quantify the likelihood for the existence of edges between node pairs $(i,j)$. However, such a direct approach may not perform well in practice, since the score matrix may be very sparse. In fact, for two node pairs $(i_1,j_1)$ and $(i_2,j_2)$, if $B_{g_{i_1}{g}_{j_1}}\left(t\right)>B_{g_{i_2}{g}_{j_2}}\left(t\right)$ and $s_{i_1{j}_1}\left(t\right)>s_{i_2{j}_2}\left(t\right)$, the node pair $(i_1,j_1)$ is more likely to be connected than the node pair $(i_2,j_2)$. Such a property is not reflected in Equation (32).

To compensate for the limitation of $p(a_{ij}^{\mathrm{train}}=1\mid B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right))$, this study finally proposes the precise definition of the synergistic link-prediction score as

$$s_{ij}^{\mathrm{SLP}}\left(t\right)=\sum _{B\le B_{g_i{g}_j}\left(t\right)}\sum _{s\le s_{ij}\left(t\right)}p(a_{ij}^{\mathrm{train}}=1\mid B,s),$$

(33)

which could be regarded as the cumulative distribution function (CDF) of $p(a_{ij}^{\mathrm{train}}=1\mid B_{g_i{g}_j}\left(t\right),s_{ij}\left(t\right))$.

4. Experiments

We chose ITS as the technological area for experimental research for three reasons. First, ITS is one of the most cutting-edge branches of transportation, providing innovative and sustainable solutions to notable transportation problems such as traffic congestion, traffic accidents, and high maintenance costs [122]. Second, ITS is an interdisciplinary technology field that involves sensors, communications, algorithms, vehicles, traditional transportation infrastructure, etc. Third, to the best of our knowledge, little work has been done to analyze the ITS development from a technology convergence perspective. This section describes our experiments based on the ITS-related patents, including data description, empirical evidence on self-similar evolution and fractality, PID analysis of the technology-convergence network, and performance evaluation of our proposed synergistic link prediction method.

4.1. Data Description

In this study, the patent data were collected from the Derwent Innovation database, one of the most widely used databases for patent analyses [5,13,116,123,124,125]. The patent data were selected from all the authorities worldwide on the date of 1 July 2022 and searched using the following query: TAB = (intelligent ADJ transportation). This query retrieves patents with the phrase intelligent transportation in the title or abstract.

It is worth noting that the choice of keywords and query rules is a trade-off between accuracy and comprehensiveness. Overly broad query rules may lead to many completely irrelevant patents in the search results. For instance, the topic of a patent with the term “smart traffic” in the title could be “smart traffic scheduling in wireless networks”. On the contrary, excessively rigorous query rules may result in the absence of some relevant patents. There is no clear criterion for this trade-off, since it is rather challenging to strictly define which patents are “relevant” to ITS. This research applied strict search criteria to ensure high accuracy and avoid including utterly irrelevant patents.

The data collected cover 8057 ITS-related patent applications for the period from November 1995 to July 2022. Each patent’s data contain a title, application number, application date, International Patent Classification (IPC), etc. Figure 4 shows the number of patent applications in each year from 1995 to 2022. It can be observed that the number of patent applications is generally increasing annually. Note that the numbers of applications in 2021 and 2022 may not be accurate, since there is a delay between patent application and patent publication, and some patent applications in 2021 and 2022 have not been made publicly available yet.

By scanning all 8057 patents, 306 related technology fields were identified (i.e., IPC subclasses represented by IPC4). Among the 8057 patents, 3763 patents contain two or more IPC subclass codes, based on which the IPC co-occurrence network was constructed.

4.2. Empirical Evidence of the Self-Similar Growth and Fractality of the Technology-Convergence Network

This subsection presents the empirical evidence demonstrating the self-similar growth and fractality of the technology-convergence network. According to the existing literature [28,33,34,35,36,37], the main criterion for the self-similar growth of a time-evolving network is that its size increases moderately over time while its topological property remains in a steady state. In terms of the growth of network sizes, Figure 5 shows that the number of nodes and the average weighted degree of the historically cumulative IPC co-occurrence network ${\mathcal{G}}^{\mathrm{hist}}$ moderately increase over time. In terms of the stability of topological properties, Figure 6 and Figure 7 show that the rescaled global (i.e., degree distributions), local (i.e., clustering coefficients and degree of neighbors), and meso-scale (i.e., community structures) topological properties of ${\mathcal{G}}^{\mathrm{hist}}$ remain in a steady state. Figure 8 demonstrates the fractality of the snapshots of ${\mathcal{G}}^{\mathrm{hist}}$.

Since the IPC co-occurrence network may not be fully connected, Figure 5a presents the evolution of the node number in the maximum connected component per year, denoted as $N_{\mathrm{LCC}}$. In addition, since this study models the IPC co-occurrence network as a weighted graph, Figure 5b illustrates the evolution of the average weighted degree $\overline{d}$ in each snapshot.

It can be observed that $N_{\mathrm{LCC}}$ and $\overline{d}$ increase over time, and both showed high growth rates in 1997–2003 and 2015–2019. During other years, $N_{\mathrm{LCC}}$ and $\overline{d}$ grew relatively slowly and even exhibited flat behavior. This observation is consistent with the history of ITS. The study of ITS dates back to the 1970s, during which, the United States, Europe, and Japan were pioneers [122,126]. Some early representative achievements appeared in the 1990s, such as DEMO’97 in the United States [127] and the Vehicle Information and Communication System (VICS) in Japan [128]. Analogously, ITS-related patent applications began in the 1990s and increased significantly in 1997 and 1998, coinciding with the emergence of DEMO’97 and VICS. The remarkable growth of $N_{\mathrm{LCC}}$ and $\overline{d}$ around 2015 matches the development of connected vehicles and autonomous vehicles [129]. Since snapshots before 2003 are relatively sparse and their growth became stable after 2003, we examined snapshots from 2003 and after.

Figure 6a–c present the complementary cumulative distributions of $d^{\mathrm{res}}$, clustering coefficients over the rescaled degree $\overline{C}\left(d^{\mathrm{res}}\right)$, and normalized average degree of neighbors over the rescaled degree ${\overline{d}}_{\mathrm{nn}}^{\mathrm{res}}\left(d^{\mathrm{res}}\right)$, respectively. Each color in Figure 6 represents a snapshot. The curve overlaps of different snapshots demonstrate the self-similar growth of the IPC co-occurrence network. In addition, Figure 6a shows that the distribution of the rescaled degree $d^{\mathrm{res}}$ approximates a power-law distribution, which could be regarded as an evidence for the fractality of the snapshots.

Figure 7 depicts the self-similarity of network evolution from the perspective of community structure. In the figure, the orange and green bars represent the modularity Q of community partitions in each snapshot and the AMI between the community partitions in the current and prior snapshots. The Louvain method [130] was used to obtain the community partitions. We can observe that the modularity Q remained stable over time, and the community partitions of two consecutive snapshots are highly overlapping. Such observations indicate that the community structure of the network does not change drastically during the evolution of the network.

Finally, Figure 8 provides evidence for the fractality of snapshots in ${\mathcal{G}}^{\mathrm{hist}}$, where each subplot corresponds to five snapshots. For each snapshot, the number of boxes $N_{\mathrm{B}}$ and the average size of boxes $l_{\mathrm{B}}$ obtained by Algorithm 1 are plotted in a logarithmic plane. Clear-cut linear relationships can be observed between $ln\left(N_{\mathrm{B}}\right)$ and $ln\left(l_{\mathrm{B}}\right)$. We further confirmed the linear relationship by performing linear regressions between $ln\left(N_{\mathrm{B}}\right)$ and $ln\left(l_{\mathrm{B}}\right)$ for each snapshot. The coefficient of determination, $R^2$, of the linear regression of each snapshot is presented in Figure 8. With $R^2>0.95$ holding for all the snapshots, the power-law relationship between $l_{\mathrm{B}}$ and $N_{\mathrm{B}}$ is further confirmed, indicating the fractality of snapshots in the time-evolving technology-convergence network.

4.3. Partial Information Decomposition of the Technology-Convergence Network

In this subsection, we investigate how the microscopic structure of the original technology-convergence network and its scaled-down replica provide information for link prediction. The main results are shown in Figure 9. It can be observed that the synergistic information (represented by the red bars for each year) is significant in most cases, which implies that the joint distribution of the two scales could be more informative than the marginal distribution of each individual scale.

For a randomly chosen node pair $(i,j)$, the microscopic structure is represented by the microscopic variable $s_{ij}$, and the scaled-down replica is represented by the macroscopic variable $B_{g_i{g}_j}$. It is worth noting that the calculation of $s_{ij}$ depends on the specific two-hop link predictors. In this study, six two-hop link predictors, as shown in Table 1, were considered. For Figure 9a–f, the microscopic variables $s_{ij}$ were calculated by the link predictors WCN, rWCN, WAA, rWAA, WRA, and rWRA, respectively.

Figure 9 shows the relative PID measures (i.e., ${\rho }_{\mathrm{red}}$, ${\rho }_{\mathrm{unq}}(B\mid s)$, ${\rho }_{\mathrm{unq}}(s\mid B)$, and ${\rho }_{\mathrm{syn}}$) of the joint mutual information $I\left(a_{ij}^{\mathrm{miss}};B_{g_i{g}_j},s_{ij}\right)$, which are colored in blue, orange, green, and red, respectively. The four relative PID measures belonging to the same year are marked by the gray regions. It can be observed that the interplay between $s_{ij}$ and $B_{g_i{g}_j}$ is primarily represented by redundant and synergistic information, whereas $s_{ij}$ could provide unique information beyond $B_{g_i{g}_j}$. In most circumstances, $B_{g_i{g}_j}$ cannot provide unique information beyond $s_{ij}$.

4.4. Performance Evaluation of Synergistic Link Prediction

In this last set of experiments, we evaluated the performance of the SLP algorithm. We implemented six synergy-based link prediction indices by incorporating a community-based index into six local similarity measures via the SLP algorithm. Pairwise comparisons were conducted between the synergy-based index and its corresponding local similarity measure, and the results are shown in

Table 2. Performance evaluation of the synergistic link-prediction methods based on AUC. Six synergy-based indices were implemented in this study, namely, S-WCN, S-WAA, S-WRA, S-rWCN, S-rWAA, and S-rWRA, which aim at enhancing the corresponding local similarity measures (i.e., WCN, WAA, WRA, rWCN, rWAA, and rWRA) by incorporating the community-based index $B_{g_i{g}_j}$. Pairwise comparisons were conducted between the synergy-based index and its corresponding local similarity measure. The one with the better performance is underlined. The best link predictor for each year is marked in bold. It is observed that the synergy-based indices outperformed both $B_{g_i{g}_j}$ and the corresponding local similarity measures in most cases, which could validate the existence and usefulness of the synergistic effect between two scales on link prediction.

Year	${\mathit{B}}_{{\mathit{g}}_{\mathit{i}}{\mathit{g}}_{\mathit{j}}}$	WCN	S-WCN	WAA	S-WAA	WRA	S-WRA	rWCN	S-rWCN	rWAA	S-rWAA	rWRA	S-rWRA
2003	0.7580	0.7192	0.8013	0.7194	0.8028	0.7199	0.8030	0.7192	0.8016	0.7193	0.8021	0.7198	0.8046
2004	0.7513	0.5926	0.7500	0.5926	0.7500	0.5922	0.7489	0.5926	0.7497	0.5926	0.7506	0.5925	0.7501
2005	0.6785	0.6962	0.7693	0.6964	0.7697	0.6968	0.7840	0.6957	0.7683	0.6960	0.7533	0.6963	0.7520
2006	0.7926	0.7145	0.8454	0.7142	0.8233	0.7134	0.8220	0.7148	0.8462	0.7145	0.8238	0.7137	0.8252
2007	0.7698	0.8121	0.8844	0.8130	0.8872	0.8143	0.8892	0.8106	0.8823	0.8112	0.8860	0.8129	0.8874
2008	0.7700	0.7694	0.8844	0.7697	0.8850	0.7697	0.8712	0.7689	0.8849	0.7690	0.8841	0.7692	0.8713
2009	0.6869	0.8813	0.8512	0.8823	0.8554	0.8841	0.8583	0.8797	0.8486	0.8804	0.8526	0.8828	0.8582
2010	0.6926	0.7940	0.8295	0.7936	0.8342	0.7914	0.8072	0.7935	0.8313	0.7933	0.8299	0.7916	0.8304
2011	0.7769	0.6914	0.7658	0.6914	0.7628	0.6904	0.7661	0.6904	0.7646	0.6904	0.7647	0.6899	0.7666
2012	0.6698	0.7068	0.7059	0.7062	0.7115	0.7043	0.6620	0.7069	0.7090	0.7066	0.7075	0.7051	0.7096
2013	0.7049	0.7301	0.7525	0.7312	0.7572	0.7335	0.7606	0.7303	0.7180	0.7310	0.7192	0.7334	0.7582
2014	0.7089	0.7165	0.7476	0.7176	0.7429	0.7199	0.7590	0.7153	0.7450	0.7160	0.7402	0.7181	0.7498
2015	0.6978	0.7045	0.7539	0.7052	0.7474	0.7064	0.7560	0.7036	0.7594	0.7041	0.7559	0.7056	0.7522
2016	0.7336	0.7259	0.7743	0.7282	0.7767	0.7301	0.7774	0.7245	0.7738	0.7260	0.7785	0.7284	0.7787
2017	0.6748	0.7555	0.7635	0.7565	0.7671	0.7553	0.7573	0.7533	0.7485	0.7540	0.7544	0.7541	0.7521
2018	0.7396	0.7678	0.8108	0.7699	0.8148	0.7712	0.8164	0.7662	0.8190	0.7677	0.8082	0.7703	0.8157
2019	0.7144	0.8231	0.8201	0.8248	0.8238	0.8237	0.8181	0.8209	0.7962	0.8221	0.8036	0.8216	0.7889
2020	0.7640	0.8357	0.8612	0.8392	0.8667	0.8421	0.8684	0.8327	0.8379	0.8355	0.8453	0.8398	0.8603
2021	0.5090	0.5506	0.5321	0.5521	0.5399	0.5539	0.5391	0.5483	0.5503	0.5499	0.5508	0.5514	0.5483

. It is observed that the synergy-based indices outperform both the community-based index and the local similarity measures in most cases, which could validate the existence and usefulness of the synergistic effect between two scales on link prediction.

The evaluation metric was the area under the receiver operating characteristic curve (AUC), which is widely adopted for the evaluation of link-prediction algorithms [20,25,111,131]. The AUC for the link predictor f in the snapshot of year t is calculated as

$$\begin{array}{cc}\hfill \mathrm{AUC}(f,t)& =\mathbb{P}\left(s_{i_1,j_1}^f\left(t\right)>s_{i_2,j_2}^f\left(t\right)\mid a_{i_1,j_1}^{\mathrm{miss}}\left(t\right)=1,a_{i_2,j_2}^{\mathrm{miss}}\left(t\right)=0\right)\hfill \\ \hfill & +\frac{1}{2}\mathbb{P}\left(s_{i_1,j_1}^f\left(t\right)=s_{i_2,j_2}^f\left(t\right)\mid a_{i_1,j_1}^{\mathrm{miss}}\left(t\right)=1,a_{i_2,j_2}^{\mathrm{miss}}\left(t\right)=0\right),\hfill \end{array}$$

(34)

where $(i_1,j_1)$ and $(i_2,j_2)$ are two randomly selected node pairs with $a_{i_1,j_1}^{\mathrm{miss}}\left(t\right)=1$ and $a_{i_2,j_2}^{\mathrm{miss}}\left(t\right)=0$. Intuitively, Equation (34) measures the probability that the link predictor f ranks a randomly selected positive sample $(i_1,j_1)$ higher than a randomly selected negative sample $(i_2,j_2)$. See the two surveys [20,111] for additional details of the AUC.

Table 2 shows the performances of 13 link predictors, where the first column displays the AUCs when only macroscopic variables $B_{g_i{g}_j}$ are used as link-prediction scores for node pairs $(i,j)$. From columns 2 to 13, two adjacent columns represent the AUCs of a two-hop link predictor f, and the AUCs of the synergistic link prediction that combines f and $B_{g_i{g}_j}$, respectively. For example, the column “WCN” shows the AUCs for the link-prediction scores $s_{ij}^{\mathrm{WCN}}$, which are calculated by the WCN link predictor as

$$s_{ij}^{\mathrm{WCN}}=\sum _{k\in |\Gamma (i)\cap \Gamma (j\left)\right|}\left(a_{i,k}+a_{j,k}\right),$$

(35)

(see Table 1). The column “S-WCN” represents the AUCs for the synergistic link prediction that combines $s_{ij}^{\mathrm{WCN}}$ and $B_{g_i{g}_j}$. The link-prediction score of S-WCN is calculated based on Equation (33) as

$$s_{ij}^{\mathrm{S}-\mathrm{WCN}}\left(t\right)=\sum _{B\le B_{g_i{g}_j}\left(t\right)}\sum _{s\le s_{ij}^{\mathrm{WCN}}\left(t\right)}p(a_{ij}^{\mathrm{train}}=1\mid B,s).$$

(36)

In Table 2, we first compared the performance of the 13 link predictors and marked the optimal AUC in bold font. It can be observed that synergistic link predictors obtained the best performance in the majority of years (14 out of 19). S-WRA, S-WAA, S-rWCN, and S-rWRA achieved the best AUC in 5, 4, 3, and 2 years, respectively. Exceptions occurred in 2004, 2009, 2011, 2019, and 2021. The best link predictor in 2004 and 2011 was $B_{g_i{g}_j}$. WRA achieved the best performance in 2009 and 2021, and the best link predictor in 2019 was WAA.

We then compared the performance of each two-hop link predictor with that of its synergistic counterpart and underlined the one with better performance. The results show that in most cases, the synergistic link predictor outperformed the corresponding two-hop link predictors. Exceptions occurred in a few years. In 2009 and 2019, all of the two-hop link predictors outperformed their synergistic counterparts. In 2012, WCN and WRA achieved better performance than their synergistic counterparts. In 2013, the performances of rWCN and rWAA exceeded the performances of their synergistic counterparts. In 2017, rWCN and rWRA performed better than their synergistic counterparts. In 2021, four algorithms outperformed their synergistic counterparts, namely, WCN, WAA, WRA, and rWRA.

Such results demonstrate the superiority of synergistic link predictors over the link predictors that consider only microscopic or macroscopic variables. Further, intriguing phenomena can be observed when we compare the link-prediction performance in Table 2 with the PID analysis in Figure 9.

• For 2004, the values of ${\rho }_{\mathrm{unq}}(B\mid s)$ are significantly greater than zero, unlike any other years. Such an observation indicates that the macroscopic variable $B_{g_i{g}_j}$ provides much unique information for link prediction beyond $s_{ij}$. At the same time, the redundant information ${\rho }_{\mathrm{red}}$ approaches (Figure 9a,b) or even exceeds (Figure 9c–f) the synergistic information ${\rho }_{\mathrm{syn}}$. Correspondingly, Table 2 shows that $B_{g_i{g}_j}$ outperformed any two-hop link predictor or synergistic link predictor and achieved the best performance.
• For 2009, Table 2 indicates that for each pair of two-hop and synergistic link predictors, the two-hop link predictors remarkably outperform the synergistic ones. Correspondingly, Figure 9 shows that the values of ${\rho }_{\mathrm{unq}}(s\mid B)$ are very large and clearly exceed the values of ${\rho }_{\mathrm{red}}$ and ${\rho }_{\mathrm{syn}}$. Meanwhile, the values of ${\rho }_{\mathrm{unq}}(B\mid s)$ approach zero for all the six two-hop link predictors.
• The situation for 2011 is similar to that for 2004. $B_{g_i{g}_j}$ outperformed all the two-hop and synergistic link predictors in Table 2 and the redundant information ${\rho }_{\mathrm{red}}$ approaches (Figure 9a,b), and even exceeded (Figure 9c–f) the synergistic information ${\rho }_{\mathrm{syn}}$ in the PID results in Figure 9. In addition, it is shown that the values of ${\rho }_{\mathrm{unq}}(s\mid B)$ are relatively small, and ${\rho }_{\mathrm{unq}}(B\mid s)$ exceeds zero in Figure 9a,b.
• The results for 2012, 2013, and 2017 in Table 2 could be summarized in the same way; that is, there are some two-hop link predictors that outperformed their synergistic counterparts, but the best performance among all the 13 link predictors was still achieved by some kind of synergistic link predictors. Such phenomena could be partially explained by relatively large unique information ${\rho }_{\mathrm{unq}}(s\mid B)$. Meanwhile, it is noteworthy that most of the differences in performance between two-hop link predictors and their synergistic counterparts are not significant for 2012 and 2017. This might be attributed to the poor performance of $B_{g_i{g}_j}$ (AUCs of 0.6698 and 0.6748, respectively), which resulted in incorporating inaccurate information, hence the unsatisfactory performance of some of the synergistic link predictors.
• The link-prediction performance of 2019 is similar to that of 2009 in Table 2, whereas their PID results in Figure 9 are noticeably different. The 2019 PID results show that although ${\rho }_{\mathrm{unq}}(s\mid B)$ is relatively large, it does not exceed ${\rho }_{\mathrm{syn}}$ and ${\rho }_{\mathrm{red}}$ in most cases. Given the fact that the performance of $B_{g_i{g}_j}$ is remarkably weaker than that of the two-hop link predictors, we speculate that the main reason for the unsatisfactory performance of the synergistic link predictors in 2019 may be the inaccurate information brought by $B_{g_i{g}_j}$.
• The year of 2021 is a special case. The experiments of 2021 aimed to predict the new links that appear in 2022 based on the snapshot of 2021, but the data collected in 2022 are incomplete and only cover the first half of the year. Since the AUCs for 2021 in Table 2 are relatively small compared to the AUCs for the year before and after 2021, we suspect that the current experimental results in 2021 could not reflect the general characteristics of link prediction.
• For other years, all the synergistic link predictors outperformed the corresponding two-hop link predictors, and the best performance was also achieved by some synergistic link predictors. Moreover, in some representative years, such as 2014–2016, the synergistic link predictors achieved noticeable performance improvements compared to $B_{g_i{g}_j}$ and two-hop link predictors, and relatively large amounts of synergistic information ${\rho }_{\mathrm{syn}}$ can be observed in the PID results.

Based on the above observations, we could summarize the following three properties of link prediction on technology-convergence networks from the PID perspective. First, combining two different scales (i.e., microscopic and macroscopic) of the network structure commonly yields better link prediction results, mainly because the two scales jointly provide a large amount of synergistic information. Second, jointly considering two scales may not be superior to using a single scale when there is too much redundant or unique information and not enough synergistic information. Finally, when one scale provides much more inaccurate information than the other, jointly considering two scales may not be preferable to focusing just on one scale.

Furthermore, another intriguing observation is that in the early stage of the development of ITS (i.e., 2003 to 2008), a large amount of redundant information existed in the PID results shown in Figure 9. Meanwhile, the AUCs of $B_{g_i{g}_j}$ in Table 2 approach or exceed the AUCs of the two-hop link predictors in most cases. In the middle of the ITS technology’s development (i.e., 2009 to 2013), remarkable unique information ${\rho }_{\mathrm{unq}}(s\mid B)$ existed in the PID results in most instances, and the two-hop link predictors outperformed $B_{g_i{g}_j}$. Synergistic information progressively dominated the PID results in the late stages of ITS technology development (i.e., 2014 and after), for which the synergistic link predictors noticeably outperformed both $B_{g_i{g}_j}$ and two-hop link predictors in most circumstances.

The aforementioned findings may provide key insights into the generation mechanism of technology convergence in ITS. In the early stages of ITS development, technology convergence often occurs between IPC subclasses belonging to the same IPC class. In other words, technologies in similar domains tend to converge. In the middle stages, microscopic structures in technology-convergence networks become the primary driving forces of technology convergence. Finally, in the late stages, the dominant factor in the generation of technology convergence is the synergistic effect between technologies’ similarity and their microscopic structures in the technology-convergence network.

5. Conclusions

This study aimed to provide theoretical underpinnings for previous empirical research on the anticipation of technology convergence using link-prediction algorithms. Existing studies mainly focused on using different link-prediction algorithms to produce accurate predictions of technology convergence in a variety of technology domains, such as ICT, 3D printing, and electric vehicles. However, theoretical explanations for such a prediction paradigm are still relatively lacking: What properties and growth mechanisms of the technology-convergence network guarantee the predictability of future links? To answer that question, we analyzed the ITS-related patent applications between November 1995 and July 2022 and provided empirical evidence to demonstrate that technology-convergence networks are fractal networks with self-similar growth processes. Such findings provide a possible theoretical explanation for the success of link-prediction algorithms in the anticipation of technology convergence, which means that the topological structure of the future snapshot is similar to that of the present snapshot.

Further, the mechanism of how the fractality of snapshots contributes to link prediction was also explored. This study adopted the PID framework to the link-prediction problem and found that the scaled-down replica is attributable to its synergy with microscopic structures. A link-prediction method based on joint probability distributions was then proposed to utilize such synergistic information. Experiments on the ITS-related patent applications demonstrated that our proposed synergistic link-prediction algorithm outperforms benchmark methods in most cases. In addition, the PID results imply that the main driving force of technology convergence varies at different stages of technology development. The results of this study could help researchers better comprehend the generation mechanisms of technology convergence from a network science perspective, which in turn could also assist companies and policymakers in developing more accurate predictive algorithms and R&D policies for potential technology convergence opportunities.

The main limitation of this study is that it overlooks the higher-order interactions of technology areas in technology-convergence networks. For example, for three technology areas (i.e., IPC4 codes) that apply for the same patent, their interactions were modeled as three pairwise connections in this study. However, such a conventional pairwise modeling approach could lead to a loss of information, since the patent could only be created when the three technology areas were combined together. Therefore, future research on technology convergence could benefit from using higher-order networks [132,133,134], in which the links are “simplicial complexes” that can connect more than two nodes [135,136,137].

Moreover, the SLP approach proposed in this study is quite intuitive, lacking a careful design for the formula of the link prediction index. In particular, the sparsity of the links in real-world networks could cause serious issues for the estimation of probabilities, although we tried to tackle this issue, at least partially, via a CDF-like formula. In the future, we would like to extend this study from the perspective of link prediction. First, we would like to examine whether the synergy between community structures and local similarity measures is widespread in real-world networks. Second, a better-designed link/community-based strategy could be proposed to make greater use of the synergistic effect. Third, extensive comparisons with different link/community-based strategies could be conducted in a broad diversity of real-world networks.