Measuring Patent Similarity Based on Text Mining and Image Recognition

Abstract

Patent application is one of the important ways to protect innovation achievements that have great commercial value for enterprises; it is the initial step for enterprises to set the business development track, as well as a powerful means to protect their core competitiveness. The emergence of a large amount of patent data makes the effective detection of patent data difficult, and patent infringement cases occur frequently. Manual measurement in patent detection is slow, costly, and subjective, and can only play an auxiliary role in measuring the validity of patents. Protecting the inventive achievements of patent holders and realizing more accurate and effective patent detection were the issues explored by academics. There are five main methods to measure patent similarity: clustering-based method, vector space model (VSM)-based method, subject–action–object (SAO) structure-based method, deep learning-based method, and patent structure-based method. To solve this problem, this paper proposes a calculation method to fuse the similarity of patent text and image. Firstly, the SAO structure extraction technique is used for the patent text to obtain the effective content of the text, and the SAO structure is compared for similarity; secondly, the patent image information is extracted and compared; finally, the patent similarity is obtained by fusing the two aspects of information. The feasibility and effectiveness of the scheme are proven by studying a large number of patent similarity cases in the field of mechanical structures.

1. Introduction

At present, in the era of economic globalization, every country is constantly innovating and developing. The effective carrier of advanced technology in every country is the patent, which contains rich technical information. It is estimated that 70–90% of patent information is not disclosed elsewhere, so patents have a higher technological content than other technology carriers [1]. At the early stage of national development, domestic attention to patents is still insufficient, and some private enterprises are in a state of insufficient knowledge of patents, some enterprises are in a state of understanding patents but do not pay attention to them, and only a few enterprises recognize the importance of patents. However, with the development of economic globalization and increasing competition in various industries, countries pay more and more attention to patents and establish relevant institutions, the state pays more and more attention to combating piracy and protecting patents, and domestic enterprises also pay attention to patents [2,3]. In recent years, the number of patent applications is increasing, and China in particular became the leading country in patent applications. Since the 21st century, the number of patent applications continued to increase, and according to the number of patent applications in 2022 released by the World Intellectual Property Organization (WIPO), the number of patent applications in 2022 is as high as 278,100, with Asia accounting for 54.7% of the total. China continues to be the largest source of patent cooperation treaty applications with 70,015 applications, and the United States is in second place with 59,056 applications [4]. WIPO’s Global Innovation Index report shows that Switzerland, the United States, and Sweden are in the top three for innovation capacity [5]. International IP filings will remain essentially unchanged in 2022. Despite challenging economic conditions and reduced venture funding, companies are investing in innovation [6].

The issue of patent infringement is a very important legal and commercial issue, especially in the field of intellectual property, which became increasingly prominent due to the continuous advancement of innovation and research and development.

The analysis of patent infringement can be considered from both legal and economic perspectives. From a legal point of view, patent infringement is punishable by law, and patent owners can defend their rights and interests through litigation and other means. In addition, according to the laws of different countries or regions, the compensation standard and the amount of compensation for patent infringement may also vary. Therefore, when analyzing the issue of patent infringement, a thorough study of the relevant laws is required to ensure that the legitimate rights and interests of the patent owner are protected.

From a business perspective, patent infringement can have a negative impact on a company’s business interests, as infringement can lead to problems such as a reduction in the patentee’s market share and damage to its brand reputation. In addition, patent infringement can create financial and commercial risks for the company. Therefore, companies must adequately protect their patents and products to reduce the risk of patent infringement.

The patent application process usually consists of three parts: the patent invention process, the patent application process, and the patent examination process. The patent application process needs to investigate the same type of patent to avoid duplication of patent novelty and innovation, while in the patent examination process, the patent examiner needs to review similar patents to assess the validity of patents. However, the number of patents is increasing, and the efficiency of manual examination can no longer achieve the expected goal, so assessing the validity of patents more intelligently is what academics are exploring. Patent similarity detection is of great importance. First, in terms of technology research, companies need to understand the patent situation in the market to avoid unnecessary legal disputes and develop more accurate market strategies and development paths. Second, patent similarity detection supports innovation. Researchers can learn about existing patents in related fields to avoid duplication of work and provide better research directions. Finally, patent similarity detection helps protect intellectual property rights, reduce the risk of infringement, and protect the legal rights of patent owners. By accurately assessing the similarity between patents, the uniqueness and validity of patents can be ensured, promoting the development of innovation and providing a reliable IP protection mechanism for enterprises and researchers. Achieving a faster and more accurate measurement of patent similarity is the key to evaluating patent validity [7]. At present, patent infringement detection methods (PIDM) are mainly divided into the following: PIDM based on clustering, PIDM based on VSM, PIDM based on SAO structures, PIDM based on deep learning, and PIDM based on patent structures. Existing patent similarity detection methods have various problems. For example, the clustering-based method has the problem of low accuracy, the vector space-based method has a high cost, the SAO structure-based method requires high accuracy of structure extraction, and the deep learning-based method has the problem of poor interpretation.

In order to improve the efficiency of patent similarity and reduce the time and cost, this paper proposes a multimodal patent similarity detection algorithm that combines patent text and images based on existing research. The method extracts the SAO structure and patent image features, uses the SAO structure to prove the patent text similarity, extracts the feature information within the image contour to prove the patent image similarity, and weights the information of the two patents to obtain the patent similarity, which can prove the patent similarity more effectively.

2. Literature Review

2.1. Patent Similarity

The clustering-based PIDM puts all detected patents together to generate one or more clusters, and those clustered with the target patents are more likely to be infringing patents. In particular, Jeong [8] extracted problem solved concept (PSC) terms and constructed a PSC-based map, clustering and evaluating them to explore opportunities for new patent creation. Zhu [9] combined a self-organizing neural network (SOM) with fuzzy C-means (FCM) clustering to obtain a SOM-based FCM algorithm, which improved the quality of clustering, automatically identified patents similar to the patents under investigation and designed a patent infringement detection system. Lee et al. [10] utilized a principal component analysis (PCA) algorithm to cluster and visualize keyword space vectors. Yoon et al. [11] converted each patent document into a vector by extracting keywords, used PCA to reduce dimensionality, and finally performed SOFM with the vector as input to create patent maps for clustering purposes. Lai et al. [12] proposed a method, called the bibliometrics-based patent co-citation approach, by analyzing the co-citations of target patents using clustering methods for cited patents and creating a patent classification system. However, the clustering-based approach can only cluster several different classes, and there will be a large number of patents in the same class as the target patent, which cannot effectively reduce the examination work.

VSM-based PIDM converts text into spatial vectors and feeds into patent similarity by comparing spatial vector similarity. Magerman et al. [13] demonstrated patent similarity using VSM and latent semantic analysis. Yoon et al. [14] used the Doc2Vec model [15] to demonstrate patent similarity and predict the future direction of technology development from the constructed patent network. The Doc2Vec model was improved from the Word2Vec model [16] by replacing the original spatial vector for word detection with the spatial vector for paragraph detection. SAO2Vec [17,18] is an improved spatial vector model based on Doc2Vec. It is easier to construct the vector space model, but the dimensionality of the vectors is positively related to the size of the prediction, and the vectors constructed by large-scale prediction are high-dimensional and sparse, which makes the computation more complicated.

SAO-based PIDM analyzes information such as sentence lexicality and obtains the desired structure using natural language processing (NLP) techniques. Park et al. [19] used the WordNet-based SAO structure to measure patent similarity and used multidimensional scaling to map patent relationships to a two-dimensional space and group patents that could infringe. Li et al. [20] used the SAO structure to prove patent similarity and extended it by using the Sorensen–Dice index [21,22], which has good flexibility and robustness. Yoon [23,24] also used the SAO structure to prove patent similarity and then used similarity to analyze potential competitors and partners. Park et al. [25] proposed a patent infringement map based on SAO semantic similarity to identify patent infringement. The calculation of patent similarity based on SAO structure has a serious dependence on the extracted SAO structure, which requires manual annotation if a higher quality SAO structure is to be obtained.

Deep learning developed rapidly in recent years, with significant achievements in text, image, and radio, and many researchers applied deep learning techniques to the field of patents. Lu et al. [26] proposed a patent citation classification model based on deep learning by selecting convolutional neural networks (CNN) at the document encoding level and introducing multilayer perceptron to gradually compress and extract the most relevant features and adjust the nonlinear relationships. Ma et al. [27] constructed a patent model tree and compared the advantages and disadvantages of CNN, RNN, LSTM, and Siamese LSTM, and established that Siamese LSTM [28,29] has obvious advantages among them. Deep learning PIDM uses neural network models for vectorized representation, although the accuracy rate is high, the model is poorly interpreted, the data for constructing specialized fields are difficult to obtain, and a large amount of manual involvement is required at the initial stage.

The composition of a patent includes several structural components, such as inventor, application number, filing date, IPC classification number, abstract, claims, etc. The last patent-based PIDM considers these structures. Zhang et al. [30] used the IPC classification model and semantic model to evaluate patent similarity by constructing patent terms into different layers of trees, each layer having its own weight value, and equating patent similarity by calculating tree similarity. Fujii et al. [31] used punctuation to segment the claims and Okapi BM25 [32] to obtain paragraph similarity, and then cumulatively obtained overall patent text similarity. Among citation methods [33], Lee et al. [34] proposed a stochastic patent citation analysis method, and Rodriguez et al. [35] proposed a similarity measure in citation networks that exploited direct and indirect co-citation links between patents. Klaans and Boyack [36] compared the accuracy of direct citation-based, bibliographic coupling, and co-citation in representing knowledge classification. In general, the classification in the direct citation classification was better than that in the other classifications. Wu et al. [37] also proposed a method for evaluating patent similarity by considering direct and indirect citations. Cheng et al. [38] used USPC and IPC construction techniques and functional class matrices to demonstrate patent similarity. Similarity based on the patent structure is more relevant, but for patent infringement, each part has a different weight, and manual weighting is resource-intensive and less feasible.

2.2. SAO Semantic Analysis

SAO structure is a construction in which the subject (S) and object (O) of a sentence are related under action (A), and an SAO structure simply reflects the content of a sentence, giving a complete picture of how two things are related or affect each other. For example, in the sentence “The shower spray water”, “shower” is the subject, “spray” is the action, and “water” is the object. Similar to the SAO structure, the subject–predicate–object (SPO) structure, which consists of subject elements, object elements, and the relationships between them, can be considered a semantic network and is widely used for knowledge discovery in biomedical literature [39], while SAO is commonly used for text mining in patent documents [40].

SAO structure is a technical tool for NLP, which is favored by scholars and received wide attention, and the ability of SAO structure extraction became more powerful in the process of the continuous improvement of machine learning algorithms. For example, Kim et al. [41] analyzed the “for” and “to” phrases and verbal forms of object elements to effectively explore the purpose and effect of the technique in depth. Miao et al. [42] used the purpose relationship between the SAO structure and the technology–relationship–technology structure to mine technology solutions and functional information. He et al. [43] proposed a potential technology requirement identification model based on semantic analysis of the SAO structure. They realized the layout and visualization of requirements based on the technology life cycle to guide the direction of technology development and optimize resource allocation. Li et al. [44] used the Unified Medical Language System to evaluate the similarity between SAO structures, which was introduced in the field of medical patents. Yoon [23,24] also used SAO structures to demonstrate patent similarity and then used similarity to analyze potential competitors and partners. Using NLP techniques, rapid mining of SAO structures from text can be achieved.

The structure of the SAO patent triad is extracted from the text, and usually the subject and object are in the form of nouns, representing the performer and the event performed, respectively. The predicates are all used as actions to link the subject and object [45]. A set of SAO structures may be included in a single sentence, or multiple sets of SAO structures may be included. In the patent text, the SAO structure of the patent is summarized as shown in

Table 1. Patent SAO structure.

No.	SAO Structure	Example
1		Wherein the conductive mechanism further comprises a control circuit module; conductive mechanism-comprises-control circuit module
2		The first conductive portion and the second conductive portion are electrically coupled with the control circuit module through conductive line; the first conductive portion-coupled-module the second conductive portion-coupled-module
3		A direction-adjustable showerhead fixing structure includes a showerhead main body and a connecting seat; showerhead-includes-body showerhead-includes-connecting seat
4		The invention reduces and eliminates the above disadvantages; invention-reduces-disadvantages invention-eliminates-disadvantages
5		The control circuit module is disposed on the mount and is located in the first chamber; module-disposed-mount module-located the-first chamber

. The similarity between patents can be translated into the similarity of the SAO set, as shown in Figure 1.

The key point of using the SAO structure applied in the patent field is the quality of the SAO structure, so manual extraction is the most accurate method, but this method is not possible in the presence of a large number of patents, which requires a lot of effort and is very inefficient. However, with the development of NLP, it became possible to extract SAO structures using NLP tools.

2.3. Contour Detection

Contour detection refers to the process of extracting the target contour by ignoring texture and noise interference within the image [46]. Traditional contour detection is broadly classified into three types: pixel-based, edge-based, and region-based contour detection methods. The pixel-based approach is concerned with discontinuity of the image boundary, and the occurrence of sharp changes in pixels around the contour indicates that a regional change is generated. This process introduced linear filtering [47,48,49], such as the Prewitt operator, Sobel operator, and Canny operator. Later, many scholars proposed the use of higher-level features such as luminance, color, and texture gradients [50], and the combination of these features improved robustness. The edge-based approach considers the overall image information and divides the contour extraction process into edge detection and edge grouping [51]. Individual edge points are collected and then formed into a continuum, irrelevant data are eliminated, and the remaining data are rearranged, with each grouping corresponding to a specific object [52]. The early determination of edge elements in the likelihood of being in the same contour was based on empirical statistics, after which Elder [53] added Bayesian inference methods, while Mahamud [54] introduced the concept of contour saliency to identify smooth closed contours. Finally, with regard to region-based approaches, Arbelaez et al. [55,56] proposed the concept of ultrametric contour maps, in which local contrast and regional contribution are involved in the dissimilarity of adjacent regions, and the key to their method lies in the definition of hyperparametric distance. The region-based method is more stable to noise and can adapt to relatively uneven contours.

3. Data Collection

Showerheads are widely used in daily life. With the continuous development of society, people’s demands for showerhead products also increased, and they no longer have only the single function of spraying water, but have added functions, such as disinfection, spraying bath products, and even massage. As a product in the traditional mechanical field, the shower is characterized by a variety of functions, a mature market, and sufficient patent applications. For this reason, the product was chosen as the research object for the experiment. In this paper, the patent database of the United States Patent and Trademark Office was searched with “showerhead” as the keyword, and the handheld shower patents from the past ten years were downloaded for testing; the total number of patents was 131. This paper lists some of the patents, as shown in

Table 2. Partial patent display.

No.	Patent Number	No.	Patent Number
1	US20220105526A1	……	……
2	US20210178409A1	……	……
3	US20210027988A1	124	US20180257090A1
4	US20200384486A1	125	US20180250690A1
5	US20190262849A1	126	US20180065131A1
6	US20190184316A1	127	US20170297039A1
7	US20190143348A1	128	US20170252764A1
8	US20180318860A1	129	US20170189918A1
……	……	130	US20170165682A1
……	……	131	US20170165684A1

.

4. TF-IDF

The term frequency-inverse document frequency (TF-IDF) model is a statistical method that can evaluate the importance of words in a text in the corpus and is a common model for calculating text similarity. The calculation process is shown in Figure 2.

• Calculate word frequency: Word frequency is the number of times a word appears in this article. To make it easier to compare articles of different lengths, word frequency is normalized by dividing the number of occurrences by the total number of words in the article.

$$term frequency=\frac{The number of times a word appears in the article}{Total number of words in the article}$$

(1)

2.

Calculate the inverse document frequency: A corpus is a collection of all articles that simulate the language environment. The more frequent a single word is, the larger the denominator becomes, and the closer the inverse document frequency is to zero. The denominator is added by 1 to prevent the denominator value from being 0 (i.e., all documents do not contain the word); lg means to take the logarithm of the obtained value.

$$Inverse document frequency=\log \left(\frac{Total number of documents in the corpus}{Document containing the word+1}\right)$$

(2)

3.

Calculation of the TF-IDF: As you can see, TF-IDF is proportional to the number of occurrences of a word in the document and is inversely proportional to the number of occurrences of that word in the entire corpus. So, the algorithm for automatic keyword extraction is clear: the TF-IDF value is calculated for each word in the document, and then the top 100 words are taken in descending order. For visualization, words are sorted by TF-IDF value and the top 50 words are captured. Figure 3 shows a heat map of TF-IDF values for these words in some patents.

$$TF-IDF=term frequency\times Inverse document frequency$$

(3)

4.

Build a word frequency list: Build a word frequency matrix; the length of the matrix is the number of texts, the width of the matrix is the number of words, and each group of vectors represents the frequency of words contained in each text.

5.

Calculating the Cosine Similarity: Given two attribute vectors, A and B, the cosine similarity is given by the dot product and the vector length, as shown in Equation (4).

$$cos\left(\theta \right)=\frac{A\bullet B}{\left|\left|A\right|\right|\left|\left|B\right|\right|}=\frac{\sum _{i=1}^n{A_i\times B}_i}{\sqrt{\sum _{i=1}^n{\left(A_i\right)}^2}\times \sqrt{{\sum }_{i=1}^n{\left(B_i\right)}^2}}$$

(4)

5. SAO Structure

5.1. SAO Structure Extraction and Cleaning

In order to better extract the SAO structure, this paper uses a method based on dependent syntactic analysis to extract triples from patents, and the main steps are shown below. The current level of NLP for SAO structure extraction is improving, but it is still impossible to accurately extract all effective SAO structures and there is bound to be some noise, so cleaning the extracted SAO structure is a necessary process. Figure 4 illustrates the SAO extraction process.

• Segmenting the text into independent sentences.
• Dependent syntactic analysis of the sentences.
• Extraction of all SAO structures.
• Clean up the SAO structure and remove the meaningless SAO structure.

The whole text of the patent is divided into sentences and the SAO structure is extracted for each sentence. The Spacy library has certain advantages in execution speed and accuracy, so the text is lexically annotated and dependent syntactic analysis is performed using Spacy to extract the subject, predicate, and object of the text, some of which may contain multiple sets of keywords. The text content of patent US20180318860A1 was subjected to SAO structure extraction, and some of the SAO structures are shown in

Table 3. Part of the SAO structure.

No	S	A	O
1	Invention	Provide	Showerhead
2	Conduit	Taper	Passage
3	Conduit	Taper	Outlet
4	Invention	Have	Application
5	Water	Passing	Showerhead
6	Configuration	Require	Tolerance
……	……	……	……
128	Jet	Include	Passage
129	Jet	Include	Outlet
130	Passage	Include	Ducting
131	Passage	Include	Apertures

. The number of SAO structures extracted by each patent is shown in

Table 4. Number of SAO structures of some patents.

No.	Patent Number	Number of SAO Structure
1	US20220105526A1	334
2	US20210178409A1	611
3	US20200384486A1	712
4	US20190262849A1	199
……	……	……
127	US20170297039A1	327
128	US20170252764A1	116
129	US20170189918A1	466
130	US20170165682A1	117
131	US20170165684A1	84

.

5.2. SAO Structure Semantic Similarity Calculation

Each patent text is represented as a collection of SAO structures, and each SAO structure consists of a subject, a predicate, and an object. The similarity of SAO structures is obtained from the similarity of internal elements, so the similarity between internal elements, i.e., the similarity between words, is measured first.

In this paper, the Word2Vec model is chosen to compute the semantic similarity between words. The Word2Vec model is a language model proposed by Mikolov et al. [16] based on the NNLM model of Bengio et al. [57] and the log-bilinear model of Hinton et al. [58]. A word can be quickly and efficiently trained into a vector form after optimization based on a given valid corpus, providing an effective tool for subsequent word similarity. Word2Vec contains two core architectures, the CBOW model and the Skip-gram model, as shown in Figure 5. The CBOW model predicts the probability of occurrence of the current word w(t) by context, while the Skip-gram model is the exact opposite of the CBOW model, predicting the probability of occurrence of several words before and after the current word w(t). Skip-gram is less efficient but has relatively high accuracy, so this paper chooses to use the Skip-gram model as the training model to ensure the high priority of accuracy.

The Skip-gram model uses a three-layer network structure to train word vectors, including an input layer, a hidden layer, and an output layer. The input layer is the one-hot encoding corresponding to the input words, while the hidden and output layers are the two vector matrices ${\mathit{W}}_1$ and ${\mathit{W}}_2$. The central word matrix ${\mathit{W}}_1$ has the dimension $V\ast N$ and the surrounding word matrix ${\mathit{W}}_2$ has the dimension $N\ast V$, where V is the size of the lexicon and N is the dimensionality of the constructed word vector. Using the one-hot encoding of the input layer multiplied by the matrix ${\mathit{W}}_1$ to obtain the vector of which we want to reduce the dimensionality, this vector can be considered as the central word vector representation of the word. This vector is then multiplied by the surrounding word vector matrix ${\mathit{W}}_2$, which is the influence of the surrounding word on the word; and finally, a word vector of size $1\ast V$ is obtained, which is finally normalized by Sotfmax to obtain the predicted probability value. The difference between the probability value and the true value is actually the loss, and according to these losses, the vector matrices ${\mathit{W}}_1$ and ${\mathit{W}}_2$ are adjusted using the backpropagation algorithm to make the prediction more accurate. The training objective function for this sequence of words is formulated as:

$$\mathrm{l}=\frac{1}{\mathrm{N}}{\sum }_{\mathrm{t}=1}^{\mathrm{N}}{\sum }_{-\mathrm{k}\le \mathrm{c}\le \mathrm{k}}\log \mathrm{P}\left({\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{d}}_{\mathrm{t}+\mathrm{c}}\right|{\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{d}}_{\mathrm{t}})$$

(5)

In this formula, k is the window size, and the larger the window, the more information is captured and the more accurate the result, but the efficiency decreases. After training, each word has its own vector representation, which is finally represented by cosine similarity.

In practice, it is experimentally found that word vectors generated by Word2Vec training are not as accurate as the NNLM model; but given a sufficient corpus, word vectors generated by Word2Vec become more and more accurate. Therefore, it can be trained on English Wikipedia to obtain a highly accurate word vector model.

The S and O are in the extracted SAO structure because there is a singular–plural distinction in the extraction process, and in practice, the singular and plural refer to the same object, so it is important to unify the word forms and eliminate this distinction. Lemminflect is a Python module for reducing the morphology of English words. It uses a dictionary to reduce the morphology of English words, and its accuracy rate is higher than NLTK, spaCy, and Stanford Core NLP. For example, the dictionary has maps from “pipelines” to “pipeline”, “showers” to “shower”, and “plays, played, playing” to “play”, so when you make the change, you can simply consult the dictionary to restore the words.

The SAO structure, where S and O are nouns, can be cross-calculated, and A is a verb and is calculated separately. The specific computation is shown in Figure 6.

The formula for calculating the similarity between two SAO structures is as follows:

$$\begin{array}{l}Sim\left({SAO}_i,{SAO}_j\right)=\\ \left\{\begin{array}{c}\frac{1}{3}\frac{\left[Sim\left(S_{\left(i\right),}{S}_{\left(j\right)}\right)+Sim\left(O_{\left(i\right),}{O}_{\left(j\right)}\right)\right]}{2}+\frac{1}{3}Sim\left(A_{\left(i\right),}{A}_{\left(j\right)}\right),\\ Sim\left(S_{\left(i\right),}{S}_{\left(j\right)}\right)+Sim\left(O_{\left(i\right),}{O}_{\left(j\right)}\right)\ge Sim\left(S_{\left(i\right),}{O}_{\left(j\right)}\right)+Sim\left(O_{\left(i\right),}{S}_{\left(j\right)}\right)\\ \\ \frac{1}{3}\frac{\left[Sim\left(S_{\left(i\right),}{O}_{\left(j\right)}\right)+Sim\left(O_{\left(i\right),}{S}_{\left(j\right)}\right)\right]}{2}+\frac{1}{3}Sim\left(A_{\left(i\right),}{A}_{\left(j\right)}\right),\\ Sim\left(S_{\left(i\right),}{S}_{\left(j\right)}\right)+Sim\left(O_{\left(i\right),}{O}_{\left(j\right)}\right)<Sim\left(S_{\left(i\right),}{O}_{\left(j\right)}\right)+Sim\left(O_{\left(i\right),}{S}_{\left(j\right)}\right)\end{array}\right..\end{array}$$

(6)

5.3. Patent Similarity Calculations

After obtaining the similarity of SAO structures, the Hungarian algorithm is used to find the maximum number of matches for two SAO sets, as shown in Figure 7. The red line represents the matching result. The Hungarian algorithm is a combinatorial optimization algorithm used for solving task assignment problems in polynomial time, and is later used to solve matching problems in graph theory.

In this paper, we set the threshold P. If the similarity of two SAO structures reaches the threshold, it is defined as a match that can be made. $SAO1\left(i\right)\left(1\le i\le n\right)$ represents all the SAO structures in patent 1, and $SAO2\left(j\right)\left(1\le j\le m\right)$ represents all the SAO structures in patent 2. However, it is possible for the SAO structures in one patent to match multiple SAO structures in another patent, and the two foci of matching are:

(1)

The match is the set of edges.

(2)

In this set, any two edges cannot have a common vertex.

Therefore, this paper uses the Hungarian algorithm to achieve maximum matching.

$$si{m}_{text}=\frac{2\ast hungarian }{Num\_SAO \left(p\right)+Num\_SAO \left(pk\right)}$$

(7)

5.4. Determining the Optimal Threshold

In order to distinguish the similarity between relevant patents and targets, this experiment wants the proportion of patents with high patent similarity and patents with zero patent similarity to be as small as possible. High similarity means that patent similarity values are more similar and difficult to distinguish. Smaller repeated similarity values imply subtle differences in similarity between patents. The smaller the proportion of patents with zero similarity, the more detailed the textual content analysis. Before calculating the initial level of patent similarity, a threshold (P) for the SAO structure must be set. A range of thresholds from 0.3 to 1 was set for the search with a step size of 0.01 to determine the optimal threshold setting. Figure 8 shows the proportion of patents with zero similarity and patents with too much similarity at different thresholds.

To reduce the proportion of patents with high similarity and those with 0 similarity, the experiment initially chose a threshold range between 0.6 and 0.8. To cross-check the results, the experiment invited experts to perform manual reading to ensure that the difference between the measured results and the manual understanding was minimized. After reviewing all combinations, it was confirmed that a threshold of 0.8 was chosen.

5.5. Patent Similarity between Target Patents and Related Patents

Using the SAO proof-of-structure method, the target patent US20180318860A1 was compared with other related patents to rank the similarity, and

Table 5. Top 10 most similar patents related to traditional SAO and targets.

No	No. Patent	Similarity
1	7	0.0727
2	14	0.0712
3	2	0.0671
4	17	0.0648
5	9	0.0643
6	23	0.0598
7	8	0.0515
8	5	0.0488
9	19	0.0484
10	20	0.0465

shows the top ten patent serial numbers and the degree of patent similarity.

5.6. Weighted SAO structure

Wang et al. [40] introduced the calculation index of different weighted SAO (DWSAO), extracted the patent SAO structures, and calculated their weight information to measure the patent similarity in robotics. The number of patents contained in the patent set is N. The target patents have m SAO structures, and $SA{O}_i^p$ denotes the i-th SAO structure of patent P. Formula 8 calculates its feature weight DWSAO value:

$$DWSA{O}_i^p=1-\frac{F}{N+1}$$

(8)

where F represents the document frequency of $SA{O}_i^p$, set the initial value to 1, traverse N patents, and add 1 to F if the patent contains SAO similar to $SA{O}_i^p$. It is derived from the formula that the greater the commonality of the SAO structure with other patents, the weaker the ability to represent technical features, and the smaller the DWSAO value. The calculation process is shown in Figure 9.

6. Multimodal Patent Similarity Analysis

The research method in this paper is to compare the target patents with related patents using the analysis method of fused images and SAO structures. In the previous SAO structure, to obtain patent similarity, the degree of similarity between patent texts was obtained only by the similarity of the SAO structure. The abstract of the patent text contains a comprehensive overview of the features of the invention, and the claims contain a detailed overview of the content of the invention; rich in content, the amount of content of the abstract and the claims are large, and the corpus available is numerous, which is suitable for studying patent infringement and patent similarity. In this paper, we choose to combine image information with the SAO structure to accurately promote patent similarity. The specific implementation process is shown in Figure 10.

6.1. Proof of Patent Similarity

• First, the SAO structure in the patent is extracted, and the resulting SAO structure is preprocessed using standard preprocessing. Second, the patent contour is extracted from the drawings attached to the abstract in the patent to preserve internal information. At the end of the process, each patent corresponds to an SAO set and a processed patent image.
• Based on semantic information, the SAO structure similarity and the similarity between the related patents containing the SAO structure set and the target patent are calculated. Each patent contains an SAO structure set, and the similarity of the SAO set is obtained to indicate the similarity of the patent, and the Hungarian algorithm is applied to obtain the corresponding similarity of the SAO structure set.
• Calculate the similarity of image features between related patents and target patents, detect the contour of the patent image, reconstruct the contour map using Fourier descriptors, retain the image within the contour, and calculate image similarity using the mutual information method. Finally, combine the weighting of patent text similarity to obtain the overall patent similarity.
• The TF-IDF method, the SAO structure method, the DWSAO method, and the Sentence Bidirectional Encoder Representations from Transformers (SBERT) method are used to calculate patent similarity between the target patent and related patents and to compare the accuracy of different methods.

6.2. Contour Extraction

After extracting the patented contours in the image, the image is first blurred using median filtering to reduce noise. Median filtering is a nonlinear smoothing technique that replaces the median of the gray values of pixel points in the eight neighborhoods around a point with the gray values of pixels at that point, and the process is shown in Figure 11a, and the grayscale before and after the change is shown in Figure 11b.

Secondly, binarization is performed to facilitate contour extraction, and adaptive thresholding tends to localize the threshold by averaging the pixel value of a pixel point with the pixel value of the region in which the point is located to determine whether the point belongs to 0 or 1.

Finally, for the pre-processed image, the Fourier operator eight-neighborhood detection is performed to extract the contour point coordinates and retain the contour point coordinates with the largest area of the region, after which the sweep profile is reconstructed by the Fourier descriptor to retain valid image information within the contour map before and after treatment, as shown in Figure 12.

Image alignment methods based on mutual information were widely used in the field of image alignment. In this paper, we characterize the similarity between two images by calculating their mutual information.

Table 6. Top 10 related patents that are most similar to the target patent image.

No.	No. Patent	Similarity
1	24	0.3932
2	22	0.3833
3	26	0.3711
4	12	0.3675
5	20	0.3495
6	9	0.3245
7	0	0.3215
8	1	0.3131
9	4	0.2932
10	2	0.2856

shows the top ten patent serial numbers and patent image similarity.

The patent similarity calculation is based on the fusion of the patent image similarity method and the patent text similarity method, as shown in Equation (9).

$$sim=\alpha \ast si{m}_{text}+\beta \ast si{m}_{img}$$

(9)

which satisfies 0 < α < 1, 0 < β < 1 and α + β = 1 in the equation.

6.3. Threshold Selected

We compared several sets of thresholds and determined the appropriate threshold from them, as shown in

Table 7. The value of the ranking change under different thresholds.

No.	α	β	Rank Change Value Sum
1	0.5	0.5	56
2	0.6	0.4	53
3	0.7	0.3	53
4	0.8	0.2	29
5	0.9	0.1	41

.

Patent images are auxiliary to the patent text, giving visual effects to the patent text and thus making it easier for the reader to understand the patent, so the weights of the patent images are lower than the weights of the patent text. The combinations in Table 6 are all confirmed, with combination 4 being the most effective, so α is selected as 0.8 and β as 0.2.

6.4. Patent Similarity between Target Patents and Related Patents

In this paper, the target patent US20180318860A1 is selected as the target patent and compared with other related patents, similarity is ranked using the method proposed in this paper, and

Table 8. Top 10 relevant patents for which the text proposes a method most similar to the target.

No.	No. Patent	Similarity
1	9	0.11574
2	2	0.1084
3	20	0.1021
4	12	0.0967
5	4	0.09376
6	8	0.0921
7	1	0.09184
8	19	0.09048
9	17	0.08746
10	24	0.08592

shows the top ten patent serial numbers and patent similarity.

7. Analysis and Validation of Results

To further verify the effectiveness of the method, the proposed method in this paper was compared with the TF-IDF method, SAO method, DWSAO method, and SBERT method. The purpose of the experiment was to find patents with a high degree of similarity to the target patent. In cooperation with the patent office, the university invited three experts, one of whom was a university professor, another an industry expert with more than five years of experience, and the last was a patent examiner who was practicing for five years, to assess similarity based on the functional features involved in the patent and the technical means used to solve the technical problem. The top ten most similar patents were obtained by ranking them from top to bottom, and

Table 9. Comparison of various methods.

No. Patent	Expert Reading	TF-IDF	SBERT	SAO	DWSAO	SAO-img
9	1	4	2	5	5	1
4	2	19	18	11	13	5
8	3	5	14	7	7	6
2	4	6	13	3	8	2
1	5	14	12	12	9	7
23	6	8	8	6	11	12
7	7	1	7	1	3	11
17	8	12	10	4	4	9
3	9	23	17	14	12	14
16	10	7	5	13	10	13
Rank change value sum	-	62	61	43	43	29
Average ranking change	-	6.2	6.1	4.3	4.3	2.9

shows the results of this analysis. To show the average variation value of each method more graphically, the results are presented in Figure 13.

As shown in the table, the absolute difference between manual reading and TF-IDF is 62, and the mean difference is 6.2. The absolute difference between manual reading and the traditional SAO method is 43, and the mean difference is 4.3. The absolute difference between manual reading and DWSAO is 43, and the mean difference is 4.3. The absolute difference between manual reading and SBERT is 61, and the mean difference is 6.1. The TF-IDF method is less accurate than the SAO method because the SAO structure reflects the structural relationship between engineering components. Park et al. [18] concluded that patent similarity based on the SAO structure is better than a text-based method, which is consistent with their conclusion. The SBERT method, although advanced, does not extract effective patent features, so the results are not excellent. The method used in this paper improves the traditional SAO, and the average ranking change is 2.9, which is better than other methods, which shows the higher accuracy of the method used in this paper.

8. Discussion and Conclusions

The patent specification carries detailed patent information, and companies face significant time and money costs for patent infringement. Blocking the publication of infringing patents at the source of patent publication can effectively mitigate subsequent patent disputes. Before a patent is granted, the patent examiner must go through a series of processes to identify the innovation and novelty of the invention patent, but the number of patent applications is growing too fast and the examination task is becoming increasingly difficult, so new examination techniques need to be improved. Several methods for patent-assisted auditing already exist with good results. In this paper, we also make the following contributions: (1) In this paper, we propose a new method to compute patent similarity by using SAO structure and the Word2Vec model, fusing semantic information with image feature information, and obtaining patent similarity by weight fusion. (2) In the process of patent text similarity calculation, the Hungarian algorithm is used to convert SAO structure similarity into patent similarity. (3) This method has a higher accuracy rate than the traditional methods, and effectively improves the review efficiency of patent examiners.

The method proposed in this paper improved accuracy to a certain extent, but there are still some shortcomings, such as the following: (1) There is still great room for improvement in SAO structure extraction, and future work will integrate deep learning-based information extraction techniques to introduce deeper semantic information and improve the accuracy of structure extraction. (2) Create a patent corpus to supplement technical terms and abbreviations. (3) It is also possible to use machine learning with a collective intelligence [59] or swarm intelligence [60] approach to replace expert judgment thresholds. (4) Extract image information by deep learning and introduce deep image features. (5) Only 131 shower patents applied in USPTO within ten years are used as case study, the quantity of which is low, and is necessary to involve the case in other fields in further research such as from design for mass customization to design for mass personalization [61]. To be sure, the accuracy of patent similarity measurement will be improved by the above-improved methods.