The Efficiency Analysis of National R&D Planning for the Field of Precision Medicine in Korea

Abstract

Precision medicine has received a lot of attention in recent years and we have not yet found any research cases that apply Data Envelopment Analysis (DEA) to investment decision making in this area. The purpose of this study is to analyze the relative efficiency of candidate technology sectors in order to determine priorities for government investment in precision medicine. The results of the efficiency analysis can be used as an important reference for government policy makers to determine the amount of government investment in the next year for each candidate technology sector. The candidate technology for government investment in precision medicine was decided for 23 sectors based on the data analysis and the opinions of expert committees. This study applies the input-oriented DEA in regard to 23 technology sectors, which is widely used to analyze relative efficiency in terms of inputs versus outputs and to enhance efficiency through the propositional reduction of inputs. The input variables include the government’s research and development (R&D) investment and forward and backward industry linkage effects. The output variables are the employment creation effect, value-added effect, number of Korean patents, and number of Korean papers. Our analysis results show that the 23 technology sectors in precision medicine overall have a high efficiency, with the exception of the biobank technology sector. Therefore, since the Biobank technology sector has strong infrastructure characteristics, it seems to require continuous investment. The efficiency of DEA is high in most precision medicine sectors; therefore, overall, investing in these technologies is expected to yield good benefits.

1. Introduction

The Korean government released the adjustment of the budget allocation in the National Research and Development Project for 2019, and announced that the technology fields would drive innovation and future growth for the fourth industrial revolution [1]. Precision medicine was among the eight areas of innovation growth [1]. Precision medicine is strongly linked to artificial intelligence and big data, which are the key technologies characterizing the fourth industrial revolution. Precision medicine was also considered to be a countermeasure for the fourth industrial revolution and finally classified as a field of innovation growth. Specifically, it seems that policy makers judged that precision medicine can strengthen public services and create new industries. Meanwhile, policy officers in the Korean government seem to believe that scientific technologies will lead the nation’s innovation growth and the ‘4th Industrial Revolution’ [1]. The term ‘4th industrial revolution’ was first mentioned at the World economic forum in 2016. However, compared with developed countries, the term ‘4th industrial revolution’ has been used in an exceptional way in Korea. There is one criticism on the other side, which is not relatively accustomed to new technologies related to the 4th industrial revolution represented by artificial intelligence. The fundamental reason for the Korean government’s interest in the fourth industrial revolution is to strengthen the nation’s science and technology competitiveness through innovation for economic growth. From this perspective, we briefly examine advanced research. Yun applied the technique of open innovation to overcome the limits of capitalism growth [2]. Kim et al. suggested that technological innovation is an important source of national competitiveness in the service industry due to the arrival of a knowledge-based society [3]. Based on the stagnation of the world economy in recent years, many experts argue for the need to create new market demands through new technology via openness, convergence, and artificial intelligence to overcome the current economic crisis [4]. In conclusion, the fourth industrial revolution and innovation growth are recognized as important keywords to obtain better economic growth from scientific technologies.

On the other hand, as societies by globalization grow more complex and diversified, it has been necessary to change the planning environment of government policies. In particular, a limitation has been revealed that decisions for policies using the traditional top-down method, the implementation of policies that rely on existing customs, experience, and individual intuition, together with unclear accountability do not explain why the policies should be established and performed. As a result, the importance of establishing policies based on evidence, such as objective information, data, and indicators, has started to be recognized. In other words, evidence-based quantitative decision-making provides clear justifications and rapidity for policy making, and it seems there is the advantage that it more easily resolves conflicts occurring in the process of executing the policies.

Precision medicine is a new industry area that has been highlighted by the state of the Union Address in 2015 [5], in which Barack Obama, former president of the United States, announced plans to invest in the precision medicine sector. The policy decision-makers in the Korean government also selected precision medicine as an area for innovation growth. However, as the policy makers have begun to have the burden of using the expert-opinion method that had been adopted traditionally or investment decision-making in a top-down way, they require a scientific methodology for evidence-based investment decisions. Specifically, after deciding on investment candidates in precision medicine, it is necessary to determine which technology sectors have investment priority. There is a limit to the amount of Korean government research and development (R&D) investment in precision medicine because of the demand for a youth unemployment and welfare increase recently. Therefore, the government’s investment decision-makers need to have an objective basis related to investment priorities for each candidate technology sector.

Therefore, this study proposes an evidence-based Data Envelopment Analysis (DEA) methodology from the viewpoint of R&D decision making. DEA provides the relative efficiency of the output-to-input between decision-making units (DMUs). The results of this study give an evidence-based method for investment decision-making in precision medicine. Specifically, the efficiency of DEA can be used as a proxy to evaluate investment priorities. This methodology also allows government policy makers to have insight into making scientific and evidence-based decisions in a way that escapes from the existing top-down and expert opinion methods. On the other hand, there is a necessity to consider the possibility of creating a new industry or its economic effect in order to determine the efficiency or priority of investment in candidate technology sectors. This is obvious from the viewpoint of government policy makers who want to pull economic growth from scientific technologies. For reflecting on these intentions, key factors, such as forward and backward industry linkage effects, the employment creation effect, and the value-added effect, were used as variables of DEA. Therefore, the results of this study are expected to be used to consider whether a new industry in precision medicine can be created and what economic effects will be made in the future. However, precision medicine is still in an early stage in Korea, is being developed mainly by the public sector, and it is rare for the private sector to have a role. Therefore, most of the input and output variables used in the DEA model were assumed to be from government-level data even though they are from national data, such as number of Korean patents and number of Korean papers. The remainder of this paper is organized as follows. In Section 2, we explore application cases of DEA related to investment efficiency in precision medicine. In Section 3, we introduce the theoretical contents of DEA. Section 4 displays the data used in this study in detail and describes the analysis results. Finally, Section 5 discusses the contributions and limitations of this research and improvement directions for it in the future.

2. Literature Review

2.1. Application of DEA in Precision Medicine and Investment Efficiency Analysis

DEA was developed by Charnes, Coopers, and Rhodes [6] and extended by Banker, Charnes, and Cooper [7] as a “non-parametric programming method for estimating the production frontier and evaluating the relative efficiency of decision-making units (DMUs) with multiple inputs and multiple outputs” [8]. This study adopts the DEA method for analyzing the efficiency between investment candidates in precision medicine. First, the advanced study on the efficiency analysis of investment is briefly as follows. Wu et al. analyzed the efficiency difference in government investment projects [9]. International comparisons of efficiency in the public sector have been analyzed using DEA [10]; the study presents seven socio-economic indicators and 17 sub-indicators that can be measured in public sector performance. DEA has also been used to measure the efficiency of education and technology at the national level [11] and the efficiency of investment in China’s economic development process [12].

It was difficult to find case studies in which DEA was applied to the analysis of investment efficiency in precision medicine. However, there are cases of research using DEA in biomedical or medical fields related to precision medicine. Here are just a few examples. Narimatsu et al. applied DEA to preventive medicine, where it was used to assess an individual’s susceptibility to obesity in establishing an effective risk model for the onset of obesity [13]. In this study, DEA was applied to calculate the efficiency score and to evaluate the usefulness of the risk model. Specifically, the input variables of DEA were exercise (measured by consumed calories) and the reciprocal of food intake (measured by calorie intake), and the output variable was defined as the reciprocal of body mass index (BMI). Salinas-Jiménez used DEA as the most appropriate analytical technique for presenting primary care performance models for patients [14]. The performance of health care should be assessed as an impact on the health outcomes of individual patients, in which DEA was used to develop performance models for primary care. In addition, DEA is used in the biotechnology and medical fields as follows: management towards financial sustainability for private health care [15]; evaluating key factors for success in a multifaceted critical care fellowship [16]; agricultural biotechnology innovation performance [17]; measuring the efficiency of large pharmaceutical companies [18]; assessing the performance of pediatric emergency department physicians [19]; the efficiency of health systems of autonomous communities in Spain [20]; a comparative study of three commonly used methods for hospital efficiency analysis [21]; and evaluation of system efficiency [22].

2.2. Application of Linkage Effect

The term ‘linkage effect’ is used in many fields and appears often in biotechnology and economics texts. Linkage in the biotechnology field often refers to the linkage of genes in the DNA (deoxyribonucleic acid) sequence. Examples of linkage effect studies from the biosector include the following: the variant impact on the linkage effect test (VIOLET) [23]; a genome-wide linkage scan for exercise participation [24]; consideration of the sib-sib correlation, linkage effects, and gene–environment interaction [25]; an electrochemical biosensor based on an enzyme substrate as a linker [26]; linkage effects in inbreeding [27]; electrodermal arousal between participants in a conversation [28]; and linkage effects on the binding affinity and activation of GPR30 [29].

The term ‘linkage’ when used in economics often refers to the linkage of a particular industry to other industries or services. Studies related to the linkage effect in economics include the following: linkage effects in developing countries [30], United Kingdom agriculture in the wider economy [31], linkage effects and environmental impacts from oil consumption industries [32], and direct foreign investment and linkage effects [33].

3. Methodology

As discussed earlier, DEA is a decision-making technique that compares the relative efficiency of outputs versus inputs among DMUs. DEA techniques have also been studied in a variety of different models. The input-based model focuses on reducing inputs with fixed outputs, while the output-based model aims at increasing outputs given fixed inputs. When a researcher chooses a model, he basically considers the realistic aspects of data availability. That is, if it is easy to adjust the input variables, an input-based model can be used, whereas an output-based model is preferred if the output variables are easier to modify. On the other hand, from the viewpoint of returns to scale, if the output increases by the same rate as the input then there is a constant returns to scale (CRS); when the output changes differently from the rate of increase of the input, it is referred to as a variable returns to scale (VRS) model. In the VRS model assumptions, a higher rate of increase in output than the rate of increase in input is called an increasing returns to scale (IRS) and, for the vice versa case, a decreasing returns to scale (DRS). However, in practice, there may be incomplete competition, asymmetrical information, or selective regulations, for example, in which the VRS model is better suited [34]. In this study, we utilize the input-based VRS DEA method.

In DEA, the goal is to improve technical efficiency. Technical efficiency can be expressed as given below [34]:

$$T{E}_k=\frac{{{\displaystyle \sum }}_{n=1}^N{\mu }_n{y}_n^k}{{{\displaystyle \sum }}_{m=1}^M{\upsilon }_m{x}_m^k}$$

(1)

where

TE_k 

is the technical efficiency of entity k using M inputs to obtain N outputs,

$y_n^k$ 

is the quantity of output n obtained by entity k,

$x_m^k$ 

is the quantity of input m consumed by entity k,

${\mu }_n$ 

is the weight of output n,

${\upsilon }_m$ 

is the weight of input m,

N 

is the number of output n, and

M 

is the number of input m.

Equation (1) is called the fractional programming model, because the equation is in a fractional form. However, in general, this fractional model can be transformed into the form shown in Equation (2) in which the solution can be obtained using linear programming techniques.

$$\begin{array}{ll}& {\theta }^{k\ast }={\min }_{\theta ,\lambda }{\theta }^k,\\ \mathrm{s}.\mathrm{t}.& {\theta }^k{x}_m^k\ge {\displaystyle {\displaystyle \sum }_{j=1}^J}{\lambda }^j{x}_m^j\left(m=1,\dots M\right),\\ & y_n^k\le {\displaystyle {\displaystyle \sum }_{j=1}^J}{\lambda }^j{y}_n^j\left(n=1,\dots N\right),\\ & {\lambda }^j\ge 0,\\ & (j=1,\dots k,\dots ,J),\end{array}$$

(2)

where

θ^k 

is the objective function of entity k in linear programming,

θ^k* 

is the real technical efficiency of entity k,

$y_n^k$ 

is the quantity of output n obtained by entity k,

$x_m^k$ 

is the quantity of input m consumed by entity k,

N 

is the number of output n,

M 

is the number of input m,

J 

is the number of entity j, and

λ^j 

is the weight of input $x_m^j$ and output $y_n^j$.

In Equation (2), θ^k represents the concept of efficiency. The actual measured efficiency of entity k is expressed as θ^k*. The conceptual form of computing the efficiency of entity k in Equation (2) can be described as follows. In the first conditional equation, θ^k is reduced starting from 1, and the efficiency of the observed value k becomes θ^k* at the moment when the inequality (≥) of the conditional expression converts to an equality (=). Equation (2) is a typical linear programming problem. Usually, linear programming is conducted in such a way as to maximize or minimize the objective function. The efficiency of observation k effectively becomes the objective function in the linear programming approach. To examine the efficiency calculation process in more detail, the process of calculating the efficiency assuming one input and one output must be examined in more detail as shown in Figure 1 [35].

In Figure 1, the horizontal axis represents the input value, the vertical axis represents the output value, and the black points A, B, C, D, E, and F are observations or entities. In the input-based model, the output value is fixed and the input value is lowered to improve the efficiency. Thus, in the input-based model, each observation moves horizontally to the left in Figure 1, which increases efficiency.

In Figure 1, finding the observation point with the highest ratio of output to input, and connecting this point to the origin, allows for the efficiency frontier to approach 100%. In Figure 1, given that point B has the highest ratio of output to input, the straight line connecting point B and the origin is the efficiency frontier in the CRS model assumptions. The efficiency frontier in the VRS model assumptions is the line connecting the points at the outermost point to the left of the observed values. Using the lines connecting points A, B, and C vertically down from point A and horizontally across from point C outlines the efficiency frontier in the VRS model assumptions. Point B is the point at which the efficiency frontier in the CRS model assumptions meets the efficiency frontier in the VRS model assumptions. From point B, the upper (lower) part of the horizon corresponds to the DRS (IRS). In economics, an assumption of DRS (IRS) implies a smaller (greater) rate of increase in output than the rate of increase in input.

In the VRS model assumptions, the point where the efficiency of point F is 100% is F_VRS-I-NS. However, comparing points F_VRS-I-NS and A, the input values of the two points are the same, but the output value of point A is higher. Thus, to achieve 100% efficiency, point F must move vertically upward from F_VRS-I-NS to point A. The value that an individual entity must further move to achieve 100% efficiency is referred to as ‘slack’.

In Figure 1, point D_CRS-I is the efficiency frontier point, projected point, or target point of point D in the input-based CRS model assumptions. In other words, in the input-based CRS assumptions, point D must move to point D_CRS-I so that the efficiency becomes 100%. Similarly, in the input-based VRS assumptions, point D has to move to point D_VRS-I for 100% efficiency. Thus, the target point of point D differs depending on the assumptions that are used [35].

There are several ways to measure efficiency.

Table 1. Types of efficiency.

Efficiency	Distance Expression
Technical efficiency of point D (TE) (Total efficiency, assuming CRS)	$\frac{{\mathrm{dD}}_{\mathrm{CRS}-\mathrm{I}}}{\mathrm{dD}}$
Point D’s pure technical efficiency (PTE) (VRS assumption)	$\frac{{\mathrm{dD}}_{\mathrm{VRS}-\mathrm{I}}}{\mathrm{dD}}$
Scale efficiency of point D (SE) = TE/PTE	$\frac{\frac{{\mathrm{dD}}_{\mathrm{CRS}-\mathrm{I}}}{\mathrm{dD}}}{\frac{{\mathrm{dD}}_{\mathrm{VRS}-\mathrm{I}}}{\mathrm{dD}}}=\frac{{\mathrm{dD}}_{\mathrm{CRS}-\mathrm{I}}}{{\mathrm{dD}}_{\mathrm{VRS}-\mathrm{I}}}$

shows the types of efficiency represented by distance expressions using Figure 1. For example, in the distance expression in Table 1, dD is the distance between points d and D. Table 1 shows the types of efficiencies represented by distance expressions (Figure 1). However, technical efficiency does not reveal the cause of the inefficiency [36]. Causes of inefficiencies may be management problems or scale problems [36]. The DEA method decomposes technical efficiency into pure technical efficiency and scale efficiency in an attempt to resolve inefficiency issues. Scale efficiency is equal to the total efficiency divided by pure technical efficiency [35].

In addition, pure technical efficiency can be compared using scale and technical efficiencies to characterize inefficiencies more fully.

Table 2. Causes of inefficiency by situation.

Situation	Cause of Inefficiency
Pure technical efficiency > Scale efficiency	Scale
Pure technical efficiency < Scale efficiency	Pure technology
Pure technical efficiency = Scale efficiency	(After comparing technical efficiency and scale efficiency) The lower efficiency factor is defined as the cause of the inefficiency.

summarizes the inefficiencies by situation.

4. Measuring the Efficiency of Precision Medicine Technology Sectors

4.1. Selection of Candidate Technology Sectors in Precision Medicine

We first briefly review the technology sectors that are important in precision medicine based on recent studies from the literature. Im et al. discuss ‘next generation technologies’ related to precision medicine, in which molecular pathology, immunofluorescence, molecular imaging, flow cytometry, and other molecular profiling strategies and their integration into new cutting-edge technologies for functional testing are identified as important technologies [37]. In addition, the techniques of single- and scant-cell molecular analyses, extracellular vesicle analysis, and circulating DNA analysis are emphasized [37]. Hunter proposed nine essential models for predicting the future of precision medicine: (1) proteomics biomarkers bridging research and the clinic; (2) predictive drug safety testing and research; (3) three-dimensional cell culture revolution; (4) bioinformatics and systems biology (e.g., in metabolomics); (5) model-based drug development; (6) pharmacogenomics; (7) wellness monitoring; (8) functional medicine; and (9) new principles of consumer engagement [38].

In this study, DMUs represent the technology sectors and the candidates for the government’s R&D investment in precision medicine. DMUs were selected by combining data-based analysis from a previous research document set and expert knowledge. Specifically, data-based analysis refers to the process of deriving candidates of technology sectors from a global thesis database (e.g., Elsevier’s abstract and citation database, SCOPUS). We examined earlier precision medicine studies from the SCOPUS database going back 5 years from 2012 to 2017. In addition, we applied a document-clustering method based on the similarity of occurrence patterns of technical keywords and reference documents. The clustering results showed that documents with similar occurrence patterns should be grouped in the same sector. For this group of papers, the abbreviated names of the technology sectors were roughly defined. The technology sector candidates were finalized by a committee of experts, composed of doctors, business operators, government officials, and data analysis experts engaged in business or research related to precision medicine. The expert committee first selected six areas of precision medicine application for consideration for government investment. The committee reviewed the technology sectors derived from the data-based analysis, gathered expert opinions, and finally selected 23 candidates for the government’s investment in precision medicine.

Table 3. Technology candidates in precision medicine.

Area	Code	Technology Sectors (Decision-Making Units, DMUs)
Omics and biometric information services	T01	Acquisition of omic data
Omics and biometric information services	T02	Biometric and bioinformatic data analysis
Omics and biometric information services	T03	Biomarker discovery
Omics and biometric information services	T04	Omics-based prediction and diagnosis of diseases
Cohort and clinical information service	T05	Biobank
Cohort and clinical information service	T06	Cohort
Cohort and clinical information service	T07	Clinical data analysis
Cohort and clinical information service	T08	Data-based treatment and prevention service in precision medicine
Lifelog and ICT	T09	Acquisition of lifelog data
Lifelog and ICT	T10	Integrated sensor and mobile healthcare technology
Lifelog and ICT	T11	Development of smart healthcare device
Lifelog and ICT	T12	Smart healthcare based on mobile devices in precision medicine
Precision medicine platform	T13	Data standardization and common model for precision medicine
Precision medicine platform	T14	Health data encryption and security for precision medicine
Precision medicine platform	T15	Data collection and integration for precision medicine
Precision medicine platform	T16	Data storage and process for precision medicine
Precision medicine platform	T17	Data platform for precision medicine
Precision medicine proof	T18	Personalized pharmacogenomics
Precision medicine proof	T19	Disease prediction and diagnosis in precision medicine
Precision medicine proof	T20	Personalized prescription and treatment of disease in precision medicine
Precision medicine proof	T21	Preclinical and clinical trials in precision medicine
Precision medicine service/industrialization	T22	Clinical decision support system
Precision medicine service/industrialization	T23	Personalized medicine services in precision medicine

shows the 23 technology sectors (i.e., DMUs) selected from the six areas.

4.2. Modeling of DEA

To apply the DEA method, the appropriate input and output variables must first be selected. In the analysis of the efficiency of government investment projects, Wu et al. used the following as input variables: financial input in the early stage, financial input from the government, financial input from society, and personnel input from the government; social benefits and economic benefits were used as output variables [9]. Lovre et al. conducted an international comparison of efficiency in public sectors; in this study, socio-economic indicators that affect the growth of public sector performance were categorized into seven areas: administration, education, healthcare, infrastructure, income distribution, stability, and economic performance [10]. These seven areas were subdivided into a total of 17 sub-indicators. Xu et al. applied DEA to measure the efficiency of education and technology; in this research, total R&D expenditure and total R&D personnel nationwide were used as input variables to measure technical efficiency, and scientific articles, patent applications, and patent grants were used as output variables [11]. Kim et al. measured the efficiency of investment in new and renewable energy (NRE) in Korea; here, public R&D expenditure for NRE and subsidy for NRE usage promotion were used as input variables, and number of patents, total volume of power generation, and unit cost of power generation were used as output variables [39]. A study by Zhang et al. examined the efficiency of the Chinese government’s investment; the input variables were fixed-asset investment at the provincial level, net fixed asset value at the industry level, and number of employees at the industry level and at the provincial level, and the output variables were gross domestic product (GDP) and value-added industry at the provincial level in the study [12]. Muangthai et al. conducted an input–output analysis to examine industrial linkage effects, which they describe as follows [40]: “The input–output analysis describes the interconnection of the industries in which the output of an industry will appear as the input of other industries”. In the linkage effect, “the backward linkage measures the impact on the supplier industries of a unit increase in the final demand for a product. The forward linkage represents the increase in sector i needed to supply the input required to produce a unit of the final demand output in sector j”.

In previous research, when the DEA method had been applied to the analysis of government investment efficiency, the input variables tended to be government investment and human resources, and the output variables were usually number of patents, number of papers, social gain, economic gain, GDP, and value-added effect. Notably, input–output analysis and DEA are both efficiency analysis methods. However, input–output analysis, as mentioned above, identifies cases where industry-linkage effects are utilized.

The choice of indicator used for DEA depends on the suitability of the indicator and its data availability; as such, seven DEA indicators were selected based on these two selection criteria as shown in Figure 2.

Figure 2 shows that there are three input variables: forward industry linkage effect, backward industry linkage effect, and government R&D investment. The four variables for output are employment creation effect, value-added creation effect, number of Korean patents, and number of Korean papers. Two indicators, forward industry linkage effect and backward industry linkage effect, may be used as output variables depending on the researcher. However, in this study, these two indicators were used as input variables considering that the variables best represented the situation being examined. Additionally, it is common practice to use government R&D investment as an input variable. All four output variables were judged to be suitable indicators.

Table 4. Seven indicators used for analysis.

Type	Indicators	Data Source	Generation Method	Year
Input	Forward industry linkage effect	The Bank of Korea (Inter-industry relation table)	Technology-Industry classification matching	2014
Input	Backward industry linkage effect	The Bank of Korea (Inter-industry relation table)	Technology-Industry classification matching	2014
Input	Government R&D investment	NTIS	Query/expert	2012–2017
Output	Employment creation effect	The Bank of Korea (Inter-industry relation table)	Technology-Industry classification matching	2014
Output	Value-added creation effect	The Bank of Korea (Inter-industry relation table)	Technology-Industry classification matching	2014
Output	Number of Korean patents	GPASS (LEXISNEXIS)	Query	2010–2017
Output	Number of Korean papers	SCOPUS at KISTI	Query	2012–2017

summarizes the seven indicators used in the analysis and the form in which each indicator value was derived.

The calculation process of the seven indicators is described briefly. First, government R&D investment refers to the amount available for investment after categorizing the government R&D projects by technology sectors. The screening of government R&D projects consists of two processes. First, the potential research projects are identified using queries from the National Science and Technology Information Service, NTIS; these projects are selected as candidates [41]. The search queries in this process are determined through expert consultation. In the second process, research projects are selected by the government’s personnel in charge of the particular area of study (e.g., precision medicine) to best represent the area of study. Through expert consultation, only research projects meeting the study goal are selected. The final research projects are then matched to specific sectors. If a project is matched to many technology sectors in a 1:n relationship, where 1 means a project and n represents the number of related technology sectors, the budget of the project is assigned into 1/n for each technology sector. In other words, after matching projects and technology sectors with 1:n for all research projects, the total budget for each technology sector is used for analysis.

In this study, 2012–2017 data were used. The number of Korean papers was based on retrieved results from SCOPUS using queries [42]; papers were limited to authors from Korea. The number of Korean patents corresponded to patents searched for using queries at LEXISNEXIS (GPASS) [43], filed from 2010 through 2017, in which the applicant’s nationality was specified as Korean. Note that it takes about 2 years for a patent to be released after receiving the application; therefore, the patent search period was extended by two years (2010 through 2017) compared with the thesis time frame. For reference, GPASS is a patent database that processes LEXISNEXIS, which is utilized within the Korea Institute of Science and Technology Information (KISTI). The search expressions, created through collaboration between technical and search experts, included government R&D investment, Korean paper numbers, and Korean patent numbers. The three types of search expressions were similar; however, the words used in the search queries differed slightly depending on the different characteristics of the three databases. The four economic effects, namely the employment creation effect, the value-added creation effect, the forward industry linkage effect, and the backward industry linkage effect, were basically derived from the inter-industry relation table published by the Bank of Korea [44]; however, this table lists the four economic effects by Korean industry classification. Therefore, the industrial classification was matched with the 23 precision medicine technology sectors as shown in the conceptual illustration in Figure 3.

In Figure 3, the technology sector (T01) is represented by the international patent classification (IPC), which is derived from the searched patent documents. It is also possible to calculate the ratio based on the frequency of appearance of the derived patent classifications. In other words, the technology sector (T01) can be defined as the ratio of appearance of the IPC. Four economic effects should be derived for each technology sector; however, the inter-industry relation table provides the values by Korean industry classification. Therefore, we used the Korean Standard Industry Classification-International Patent Classification (KSIC-IPC) matching table provided by the Korean Intellectual Property Office [45] to obtain the four economic effects for each technology sector. If the economic value provided by the industrial classification multiplied by the ratio of the IPC matches the classification, the economic effect value of the technology sector (T01) is obtained. This process was applied equally to all four economic effects. Note that the text and numerical values shown in Figure 3 are conceptual examples and not actual values. On the other hand, the industry association table does not announce the actual value every year. The most recently announced data release was in 2016, corresponding to a base year of 2014.

The four economic effects are described below. In this paper, the employment creation effect refers to the direct employment number required for the relevant industry and the additional employment number indirectly caused by other industries per 1 billion Korean won (KRW) of input, referring to the employment table of the Bank of Korea’s inter-industry relation table. The value-added effect refers to the situation in which the value-added unit directly or indirectly stimulates the entire national economy if the final demand for the domestic product of an industrial sector increases by one unit. That is, it indicates how much the production inducement by the increase of 1 unit of final demand causes the added value to be realized. Given that the products of each industry are used as final goods for consumption and are also used as intermediaries for the production of other industries, there is some correlation among industries, which can be used as an indicator of the industry linkage effect [46].

The industrial linkage effect can be divided into forward and backward industry linkage effects. The forward industry linkage effect represents the degree to which intermediate goods are sold to other industries, and the backward industry linkage effect represents the degree to which intermediate goods are purchased from other industries. The forward industry linkage effect is commonly measured using the sensitivity dispersion index, and the backward industry linkage effect is measured using the power dispersion index [47]. The sensitivity dispersion index can be obtained by dividing the sum of the row entries of the production induction coefficients of an industry by the average of all industries. An industry with a value greater than 1 is an industry that receives higher than average impacts from other industries. The power dispersion index is determined by dividing the sum of the column entries of the production induction coefficients of an industry by the average of all industries. If this value is greater than 1, it means that the production inducement effect on the whole industry is higher than the average. The values of the seven indices derived in the manner described above are shown in

Table 5. Values of the seven indicators used in the analysis.

Code	Forward Industry Linkage Effect	Backward Industry Linkage Effect	Government R&D Investment	Employment Creation Effect	Value-Added Creation Effect	Number of Korean Patents	Number of Korean Papers
T01	0.641	0.990	0.753	9.900	0.979	14	9
T02	0.767	1.026	0.729	10.599	0.983	37	37
T03	0.687	1.069	0.353	9.937	0.980	87	28
T04	0.666	1.005	1.000	10.015	0.979	163	89
T05	0.720	1.016	0.817	9.561	0.977	5	95
T06	0.734	0.979	0.501	10.881	0.981	6	104
T07	0.680	1.042	0.435	9.842	0.979	31	541
T08	1.012	1.239	0.141	9.778	0.978	147	333
T09	0.784	0.975	0.833	11.481	0.985	10	2
T10	0.806	1.043	0.520	11.020	0.983	135	242
T11	0.805	1.071	0.596	10.861	0.983	24	277
T12	0.797	0.965	0.202	11.826	0.984	12	184
T13	0.807	0.953	0.315	11.812	0.984	19	197
T14	0.787	0.984	0.254	11.435	0.984	19	111
T15	0.790	0.987	0.095	11.526	0.984	125	15
T16	0.793	0.963	0.060	11.757	0.984	12	125
T17	0.789	0.950	0.332	11.797	0.984	9	73
T18	0.697	1.050	0.990	9.880	0.979	34	558
T19	0.728	1.059	0.532	9.901	0.979	226	408
T20	0.728	1.049	0.514	9.867	0.979	37	339
T21	0.714	1.014	0.504	10.474	0.980	9	213
T22	0.802	0.865	0.000	12.561	0.984	7	182
T23	0.667	1.010	0.050	10.252	0.980	10	107

; however, government R&D investment was normalized between 0 and 1 using maximum and minimum values, not actual values.

4.3. Efficiency Measurement of Candidate Technology Sectors

In this study, the DEA for the precision medicine field was conducted using the DEAP software [48]. The results are shown in

Table 6. Data envelopment analysis results.

Code	Technical Efficiency (TE)	Pure technical Efficiency (PTE)	Scale Efficiency (SE)	Cause of Inefficiency	Returns to Scale
T01	1	1	1		-
T02	0.947	0.966	0.98	Pure technology	drs
T03	0.989	0.999	0.99	scale	drs
T04	1	1	1		-
T05	0.962	0.966	0.996	Pure technology	irs
T06	0.991	0.992	0.999	Pure technology	irs
T07	1	1	1		-
T08	1	1	1		-
T09	0.982	1	0.982	scale	drs
T10	0.973	1	0.973	scale	drs
T11	0.923	1	0.923	scale	drs
T12	1	1	1		-
T13	1	1	1		-
T14	0.981	0.999	0.982	scale	drs
T15	1	1	1		-
T16	1	1	1		-
T17	1	1	1		-
T18	1	1	1		-
T19	1	1	1		-
T20	0.962	0.963	0.999	Pure technology	irs
T21	0.981	0.982	1	Pure technology	-
T23	1	1	1		-
Average	0.986	0.994	0.992

Note: The T22 technology sector is excluded in the analysis because there is no government R&D investment.

.

Technology sectors where the PTE value is 1 can be interpreted as an area where efficiency is optimized. The following sectors showed optimized efficiency: T01, T04, T07, T08, T09, T10, T11, T12, T13, T15, T16, T17, T18, T19, and T23. DEA showed that the personalized prescription and treatment of disease in precision medicine (T20), biobank (T05), biometric and bioinformatic data analysis (T02), preclinical and clinical trials in precision medicine (T21), and cohort (T6) technology sectors had less than average technical efficiency. Technology sectors where the cause of inefficiency is pure technology should make efforts to improve efficiency by securing the capability to improve the technology; these sectors were as follows: biometric and bioinformatic data analysis (T02), biobank (T05), cohort (T06), personalized prescription and treatment of disease in precision medicine (T20), and preclinical and clinical trials in precision medicine (T21). If the cause of inefficiency is scale and the returns to scale is DRS, efforts should be made to improve the efficiency of the technology sector by reducing the scale; this was the case for the following sectors: biomarker discovery (T03), acquisition of lifelog data (T09), integrated sensor and mobile healthcare technology (T10), development of smart healthcare devices (T11), and health data encryption and security for precision medicine (T14). On the other hand, if the reason for the inefficiency is scale and the returns to scale is IRS, it is necessary to make efforts to improve the technical efficiency of the technology sector by increasing the scale; however, our results did not identify a technology sector with this issue.

Precision medicine requires the medical profiles of individuals as opposed to universal medical care in the traditional way. Biobank and cohort technologies are becoming the infrastructure technology for precision medicine by collecting and managing personal genome, clinical, environmental, and life-log information. Therefore, these technical fields have characteristics that require large-scale resources. Biobank and cohort technologies must enhance their technological capabilities in terms of infrastructure, as the cause of their inefficiencies was identified as pure technology. On the other hand, biomarkers are technologies for measuring changes in the living body. Lifestyle information acquisition and mobile health and device technologies are the basis of smart health, all of which are highly dependent on information technology. In general, precision medicine is considered to be an innovative technology due to its strong bio-information technology convergence; thus, it requires a certain level of economy. The DEA results also showed that the cause of inefficiency of the aforementioned technology sectors is scale, with DRS characteristics. However, from the perspective of cultivating precision medicine as a new industry, it is necessary to consider carefully whether a reduction in size of these technology sectors is a desirable direction.

Input-based DEA attempts to increase efficiency by reducing input resources. In other words, the input resource can be saved by the difference between the projected value where efficiency is 100% and the original value used for DEA analysis. Inefficiency has to do with the reduction rate of input resources as expressed in Equation (3). If the projected value is smaller than the original value, the value of Equation (3) would most likely be negative.

Reduction ratio (inefficiency ratio) = (projected value − original value)/original value

(3)

Table 7. Inefficiencies by input variables.

Code	Technology Sectors (DMUs)	Forward Industry Linkage Effect	Backward Industry Linkage Effect	Government R&D Investment
T02	Biometric and bioinformatic data analysis	−0.034	−0.038	−0.034
T03	Biomarker discovery	−0.001	−0.052	−0.001
T05	Biobank	−0.035	−0.034	−0.267
T06	Cohort	−0.008	−0.008	−0.008
T14	Health data encryption and security for precision medicine	−0.001	−0.013	−0.001
T20	Personalized prescription and treatment of disease in precision medicine	−0.037	−0.036	−0.037
T21	Preclinical and clinical trials in precision medicine	−0.018	−0.018	−0.018

shows the inefficiencies of input resources for the remaining technology sectors, except for the technology sectors with a technical efficiency (VRS TE) of ‘1’.

In terms of the government’s R&D investment, the inefficiency of the Biobank (T05) is −0.267, which is relatively inefficient. This indicates that the biobank sector has a strong infrastructure characteristic; therefore, it is important for society and requires a significant investment. It also indicates that the investment is relatively high compared with that of other technology sectors; however, the low performance is of concern. Treatment of disease in precision medicine (T20), biometric and bioinformatic data analysis (T02), preclinical and clinical trials in precision medicine (T21), cohort (T06), biomarker discovery (T03), and health data encryption and security for precision medicine (T14) were analyzed as technology sectors that should increase efficiency through investment reduction.

On the other hand, if the inefficiency is theoretically set to ‘0’, the efficiency of the technology sector becomes ‘1’. However, DEA researchers argue that inefficiency cannot attain a value of absolute ‘0’. There are cases where it is necessary to assume some inefficiency, considering the characteristics of the analysis unit [36]. Biobank and cohort technologies are all cases of large-scale national projects in that they collect and utilize human resources. The high inefficiency of the biobank (T05) also appears to be the result of a nationwide resource infrastructure project, which has the property that large budgets are injected into the standardization of sample processing and infrastructure for resource management. However, ‘cohort’ is similar to ‘biobank’ in that it has the property of being a resource infrastructure business, but the inefficiency is relatively acceptable. This seems to be due to the fact that the biobank technology sector has been widely used for a long time in traditional biopharmaceutical technology, whereas the cohort technology sector is aimed at a prospective cohort study for a large population. It has also been attributed to the recent emergence of precision medicine. Thus, theoretical inefficiency based on the characteristics of the analytical unit may be interpreted differently depending on the actual industrial viewpoint. As such, the inefficiency of the analytical unit requires careful consideration.

The forward industry linkage effect and the backward industry linkage effect have characteristics that represent the current situation. These indicators are not readily adjustable by the government in a short period of time. In addition, as these two variables increase, the inter-industry linkages become more active and, consequently, help the national economy. As DEA analysts have pointed out, these two variables are not subject to reduction but, rather, are conditional variables that must be accepted.

5. Conclusions

The mode of government investment in R&D is determined through a very complex process. This process reflects the limitations of the budget, the opinions of technical experts and government officials, the needs of private companies, and the general opinion of the public. It also reflects the R&D investment trends of technologically advanced countries’ governments and the announcement of future promising technologies both domestically and abroad. Thus, the R&D status of global companies is also considered. This study focused on an alternative for R&D investment decision-making using DEA to calculate the efficiencies of specific technology sectors using input and output variables, i.e., based on a scientific methodology rather than an expert opinion base. Here, we investigated the investment efficiencies of 23 precision medicine technology sectors that required government R&D investment. Our DEA results identified 15 of the 23 technology sectors as having an especially high pure technical efficiency, with 0.963 as the lowest value. In other words, in the field of precision medicine, the pure technical efficiency is high. Thus, according to the efficiency analysis, it would be sufficient to invest in the detailed technology sectors on an equal footing within the range of the available budget, without discrimination among technology sectors. Our results also revealed weaknesses in the biobank technology sector; however, given its strong infrastructure characteristics, continued investment is recommended.

From the academic point of view, this study contributes to the first application of the DEA technique to analyze investment efficiency in precision medicine. In previous studies, it was difficult to find case studies applying DEA to investment decision-making in precision medicine. Second, this research contributes to the efficiency analysis of the government’s investments by providing a framework that links employment and new industry creation using the industry linkage effect, employment effect, and value-added effect as applied to DEA. In addition, our approach provides a precedent reference for policy makers to decide which areas to invest in first. This is expected to help policy makers in Korea overcome any lack of resources in the midst of slow growth.

This study had several limitations. First, we used an input-based DEA model, which is always concerned with the reduction of input resources; as such, the research methodology and/or practical aspects may be at a disadvantage. For example, in the field of precision medicine, this may not be appropriate if we have a policy of setting sufficient input resources and maximizing performance. Second, even though the input-based model was used, it is not easy to adjust the effects of forward industry linkage and backward industry linkage as input variables. On the other hand, the main purpose of this study was to investigate the efficiency of input versus output from the viewpoint of government investment. Given that precision medicine is a new industry in Korea, it has developed as a government-led endeavor. We applied this aspect to the acquisition of values of seven variables used for DEA analysis, in which six of the seven variables had values in line with the national viewpoint, i.e., government and private values. We assumed that all seven variables were similar to those derived at the government level given the level of government involvement. This is the third disadvantage of this study. However, as the precision medicine industry continues to develop, these variables will need to be calculated separately at the private level and the government level; in the present situation, it is impossible to obtain the seven variables separately. Fourth, the input cash flow is not considered in the four economic effect indicators used in this study. Therefore, in this study, the basic assumption is that the amount of future cash flows that can be obtained from the candidate technology sectors will be the same. In the future, it is expected that more sophisticated efficiency measurement will be possible by estimating the economic life cycle of the candidate technology sectors and the scale of cash flows by the lifetime.

Additional recommendations include supplementing the aforementioned shortcomings described in the limitations, applying the current status of available human resources by technology sector, and developing the means to calculate the input/output variable value based on a nationally recognized or government-specified standard. In addition, it is possible to estimate the economic life cycle and the cash flows by candidate technology sector, thereby measuring more sophisticated efficiency. We are going to address these issues in future research endeavors.