Analysis of the Effect of R&D Planning Support for SMEs Using Latent Growth Modeling

Abstract

Small and medium enterprises (SMEs) in Korea play a pivotal role in the national economy. However, due to scale limitations, the resources available to SMEs for innovation activities are limited. Therefore, as a means to compensate for the market failure of SMEs, which can be caused by resource limitations, and to ensure sustainable growth, the government supports the SMEs for innovation activities. Korea has fulfilled SME R&D planning support projects since 2002 to efficiently support SMEs’ technology commercialization activities through systematic support in the early stages of the technology commercialization cycle. In order to analyze the effectiveness of SME R&D planning support program, this study conducted latent growth modeling analysis using financial data of three years of SMEs participating in the program. According to the analysis results, SMEs receiving R&D planning support funds have an increase in sales compared to those that do not. In particular, it was analyzed that the sales increase effect of companies supported by R&D funds continuously appeared more clearly. The result of this analysis is that policy support for SMEs can help technological innovation activities necessary for R&D planning, which is an early stage of SME technological innovation. This study is meaningful in that it quantitatively analyzes using latent growth model and the contribution of government support at the R&D planning stage to SMEs’ innovation capabilities and their contribution to performance. However, it has a limitation that it does not address the specific impact path of the planning stage support on innovation capacity and innovation performance.

1. Introduction

Innovation is a sustainable process that identifies and captures opportunities in a constantly changing business environment. Therefore, measuring activities and performances that affect innovation is also very important in terms of organizational management for innovation [1,2,3].

Most innovation measures have a structure for evaluating innovation based on the input and output of the innovation process, or there are approaches to assess the process of innovation in addition to input and output [4,5,6,7,8,9]. In addition to SMEs’ own innovation activities and efforts, the innovation process includes allocating and streamlining the resources required for the process through policy support.

Under the RBV (resource-based view) perspective, the policy underpinning the sustainable growth of SMEs is emphasized as a means to compensate for the lack of resources needed for innovation [10]. In particular, it is important to secure resources that can provide a competitive advantage, which includes information. Information is particularly important for SMEs and venture firms, and it is regarded as a learning process that overcomes new responsibilities through information gathering [11]. R&D planning is a process by which companies acquire technology for innovation, allocate resources to produce products, and formulate strategies to approach the market systematically. Recently, discussions on how resources and information affect sustainability in small and medium-sized businesses have been conducted. Some studies have shown that information and resource access have a positive and direct impact on sustainability as well as innovation capacity [12]. Therefore, for the sustainable growth of SMEs, systematic R&D planning based on resources and information is required.

The government’s implementation of supportive policies for SMEs necessitated research on the effects of such governmental support, and there has been a growing demand for in-depth analyses of the performance of individual programs. The Ministry of SMEs and Startups in Korea conducts a descriptive statistical survey of a sample of 3300 SMEs each year and releases analyses on the outcomes of governmental support. However, since this survey covers all technology development activities, it provides very few performance analyses of individual support programs, thus leaving a need for more individualized studies. The government’s investment of R&D funding for SMEs often has indirect effects on the supported firms, but analyses of the performance resulting from such funding have been inadequate, relying mostly on financial data from the beneficiary firms or surveys collecting feedback from employees. Existing studies about governmental R&D support programs remain at the level of introducing major R&D programs in Korea and abroad or analyzing performance only in terms of input cost and budgetary benefits, and many of these studies also rely on surveys of representatives of the beneficiary companies, which may be unreliable due to the subjective nature of the feedback. Furthermore, most of the surveys intended to measure the effects of such programs yield only cross-sectional data for a single period; longitudinal data based on repeated measurements over a longer period remain very scarce. Although it is possible to use the survey method to conduct repeated measurements, this also poses many problems, such as the common method bias that occurs when the same respondent is surveyed for both the independent and dependent variables, and the inconsistencies that may occur due to changes in the respondent representing a firm. The survey method thus limits our ability to accurately measure the efficacy of government support programs and the magnitude of their impact.

This study remedies these problems by performing a more objective analysis of the effect of government support programs by collecting concrete financial performance data from SMEs that received governmental support and comparing this to the data from SMEs that did not receive support. First, we divided the firms that applied to the SME’s R&D planning support program in 2011 into beneficiaries and non-beneficiaries, and compared the outcomes of the program in the three years from 2011 to 2013 by analyzing group homogeneity based on financial performance data (sales revenue). Next, we performed latent growth modeling analysis to determine the impact of the program on the beneficiary firms over time.

One strength of this study is that it improved data reliability when analyzing the effect of support programs for SMEs by using objective financial performance data regarding the beneficiary SMEs. Furthermore, when collecting financial data, we also included the financial performance data of non-beneficiary SMEs, an aspect that was overlooked in previous studies using conventional methods of analysis. This comparative approach to analysis allowed us to determine the effects of the program and the magnitude of the effects in a more objective context. The results of this study will make it possible to perform more objective evaluations of governmental R&D support programs for SMEs, and the feedback from these evaluations will help improve the programs.

This paper is composed of preceding research, research method, results, and discussion. In preceding research section, we review the preceding research on R&D planning and assessments of R&D policy effects, and examine the meaning of the policy of supporting the R&D planning program of SMEs in Korea. The research method section describes the data used in the research on whether the policy support for R&D planning affects the financial performance of SMEs and the research method. The discussion section includes an analysis of the effectiveness of the policy support at the R&D planning stage and will cover what it means for governments and SMEs to select and implement their policy instruments.

2. Theoretical Background and Preceding Research

2.1. R&D Planning for SMEs

R&D planning is the process of planning the tasks and methods to be performed in order to achieve the set purpose of the research project. In other words, it is a dynamic process that prepares a series of decisions about future activities, which is at the forefront of the process of technology commercialization and has a great influence on the whole process. Therefore, various research planning methodologies exist. Many researchers define planning stages in the technology commercialization process, and the scope of planning differs depending on the point of view. Cooper sees the planning phase as idea generation, preliminary evaluation and conceptual design, while Utterback is limited to the formation of ideas [13,14,15]. Briner et al. describe that technology products begin by identifying the target market demands [16]. In addition, it is limited to the feasibility review of the technology.

In general, decision-making on R&D tasks requires the establishment of an R&D strategy based on the business environment, resources for R&D, corporate goals, strengths and weaknesses, and the allocation of resources and various alternatives for a series of processes. It includes procedures such as evaluation.

Skilled professionals in strategic planning are required for R&D planning, but SMEs often do not have expertise in strategic planning due to resource limitations. In particular, early start-ups are often composed of human resources only for research and development, which makes it difficult in systematically planning R&D. Due to the necessity of supplementing SME capacity limitation caused by resource limitations, various countries have been making adjustments through policy instruments.

In the US, SBIR (Small Business Innovation Research Program) has been in operation since 1982 to support SMEs’ technology commercialization. In SBIR stage 1, the R&D planning stage is funded by assessing the feasibility of a research project. This system was similarly introduced in Japan, Korea and the United Kingdom government to support SMEs’ research planning

The research on R&D planning stages mainly includes the establishment of processes or models for planning, planning tools, key indicators of planning, planning systems, and case studies that apply the models to specific domains [17,18,19]. Compared to the recognition of the importance of R&D opportunities, there are few studies on evaluation of R&D planning’s performance contribution and related policies.

2.2. R&D Planning Policy in Korea

South Korea’s policies supporting technological innovation by SMEs correspond with a worldwide policy trend, and existing self-evaluations of these policies indicate that they have strongly contributed to simulating innovative activities to a higher level. Nonetheless, South Korea’s efforts to promote SMEs have proved to have limitations, as indicated by the very low level of success South Korean SMEs have shown in technology commercialization.

South Korea supports SMEs through the Ministry of SMEs and Startups and its subsidiary institutions as well as through public institutions that meet the criterion of conducting research and development at a specified scale (KOSBIR). The governmental policies to support the R&D of SMEs include the efforts of the Ministry of SMEs and Startups to strengthen the technology innovation capabilities of SMEs and to build an infrastructure conducive to research and development. The key areas of priority in these governmental programs are “selection and concentration,” “expansion of the base for technological development,” and building a “technology development infrastructure.” The R&D planning support program analyzed in this paper as a case study of performance analysis falls under the category of “technology development infrastructure.” The responsibility to support SMEs does not belong exclusively to the Ministry of SMEs and Startups. Under the KOSBIR system, all public institutions that have an annual budget for research and development of 30 billion Korean won or more are required to support the R&D of SMEs with 0.1–33.1% of their budget allocated to research and development.

South Korea’s policies on SMEs are regarded as giving less priority to the long-term improvement of technological innovation capabilities through the support of SMEs; instead focusing on making sure that the area of technology being supported is timely and ensuring the efficient management of R&D funding investments. Consequently, compared to other advanced countries, South Korea has achieved a high level of innovative effects when assessed in terms of the cost invested in research and development, the intensity of concentration in research and development, the concentration of research personnel, and patent applications; but Korea has had only limited success in securing performance evaluations that clarify the degree to which these innovative effects ultimately contributed to improving technological competitiveness and generating economic growth. Among the various priorities that could potentially be established for technology development support, Korea’s R&D policies for SMEs tend to focus strongly on managing the transparency and efficiency of funding, and this approach has been criticized for its limitations in addressing the long-term problems in technology development and innovation.

To diversify the technology innovation support policies intended for SMEs and ensure that these policies result in socially beneficial effects, there is a need for continued research to analyze the performance of these policies using multifaceted approaches.

2.3. Assessment of R&D Planning

The budget allocated by the government to supporting R&D by SMEs has continued to increase, from around 2.1 trillion Korean won in 2011 to 2.9 trillion in 2015 and 3.5 trillion in 2018. Governmental programs included 54,031 projects from 2011 to 2015, dispensing support to 73,475 SMEs. Although it is crucial for the government to justify these technology innovation support policies by presenting evidence of their efficiency, there has been a dire lack of empirical studies providing scientific and systematic evaluations of the performance of individual projects, aside from basic statistics and general surveys conducted by the Ministry of SMEs and Startups covering all SMEs that received governmental support. Most of the existing literature remains at the level of introducing some of the major R&D programs in Korea and abroad, and offering performance analyses based on surveys and simple calculations of input cost and budgetary benefits [20,21,22].

Energizing technology innovation among SMEs continues to be a national policy priority, and consequently, there is a continuing need to demonstrate the effects of governmental policies supporting SMEs based on verifiable analyses of input and outcome, encompassing the process of pre-planning for R&D, technological outcomes, and commercial success. Most of the existing studies that analyze the factors that stimulate or impede the technology innovation activities of SMEs are cross-sectional analyses based on surveys at a single point in time, and most performance analyses remain at the level of simply comparing R&D input with innovative performance. There is a regrettable scarcity of empirical corporate panel data analyses using time series data.

SMEs’ technological innovation achievements include rising success rates of technology development, securing intellectual property rights, and strengthening technological competitiveness. Commercial performance can be assessed by the company’s growth potential, profitability, and productivity indicators based on the technical achievements obtained. As an example of a research study between venture firms’ technological innovation performance and industry-academia cooperation, cooperation with the government and related institutions has a positive effect on product innovation capacity, and fair innovation capacity includes process innovation capability/cooperation between the government and related organizations and private companies. Positive relationships have been derived [23]. In addition, in the results of the analysis of the achievements of R&D investment in the science field, the number of publications (the number of articles published in the SCI DB) and the number of citations, which are the result of the research activities, were presented as indicators of the evaluation of the research activities. Through this, the relationship between the input of scientific research (total R&D expenses, total number of researchers) and performance was measured and presented statistically [24].

There has been continuing discussion regarding how to measure economic performance to evaluate innovative performance. There have been empirical case studies using economic indicators including financial performance indicators (such as sales revenue growth rate, profit ratio in relation to sales, etc.) along with non-financial performance indicators (such as job creation, export performance, market share, shareholder value reflected in share prices, market capitalization, Tobin-Q, etc.). An empirical study in South Korea analyzed the relationship between R&D costs and advertisement expenditure among listed companies from 1981 to 1991 and found that the advertisement expenditure of a firm had a positive impact on generating profit [25]. Additionally, a survey-based study of small and medium IT venture companies indicated that strong technological innovation performance (indicated by the number of intellectual property rights acquired over a period of three years) had a positive impact on increasing sales revenue and improving the sales profit margin [26].

Estimates of contributions to the sales revenue of firms are often used to assess economic performance, and these estimates are usually made based on industry data or surveys of firms that reflect the intuition of the research personnel or a small number of experts. Surveys of firms (conducted through in-depth interviews with CEOs and representatives of divisions responsible for business management strategies or expanding performance) are very useful for objectively understanding the effects that projects contribute to sales revenue or for gaining an in-depth knowledge of business conditions.

The survey-based methods hitherto used to estimate business innovation performance have the drawback of introducing a subjective element and may result in inconsistencies caused by changes in the surveyed firm representatives and the discontinuity of data from indicator surveys conducted once each year, which make it difficult to conduct systematic and sustained research. Furthermore, there is a risk that judging performance based only on a comparison of the cost input into a program and the sales revenue figures (operating profit) without analyzing the influences of control variables (or parameters) that impact the performance of support policies for SMEs may result in a lack of objectivity and validity. To redress these problems, we can use analytic methods based on statistical modeling (structural equations, etc.) that help present more persuasive results regarding the influencing factors, taking account of the correlation between independent and dependent variables such as the financial data (performance) of firms in a specified period.

In addition to evaluating the outcomes of the policy, the ultimate goal of policy evaluation for SMEs is to study the characteristics of the demand group of the SME joint research support program in terms of open innovation from the approach of efficiency improvement of policies [27,28,29,30]. Using survey data of SMEs, through decision tree analysis and discriminant analysis methodology, SMEs that prefer a joint research support program have long R&D planning time, high external R&D investment, and lack of available equipment. The policy support of the R&D planning stage and the policy support supporting joint research are closely related, and synergies can be created through the combination of policies.

In policy evaluation, the amount of accessible data is increasing, and as the method of analysis is advanced, an approach to new policy evaluation is taking place. In particular, many researches have been carried out using cross-sectional analysis using time series data. In the case of policy, the effect of policy is often delayed. Therefore, rather than looking at the relationship between variables at a particular point in time, a longitudinal study of how changes in one variable affects changes in other variables may be appropriate. The latent growth model is a model that can explain the differences in the time series of individual subjects.

Therefore, it may be meaningful to perform performance analysis based on mechanical observations (which can minimize subjective elements) and thereby derive data regarding the planning of R&D related government support programs, the validity of the government’s intervention through financial support, successes in commercialization, and contribution to generating improved performance.

3. Materials and Methods

3.1. Analysis Data

South Korea’s Small and Medium Business Administration performed various policy-based measures to promote R&D innovation activities among SMEs. The areas of focus in the Small and Medium Business Administration’s policies can be broadly divided into selection and concentration, expansion of the R&D base, and infrastructure. The Administration implemented technology development support programs with focuses on applied research, development research, and production technology research. These support programs provided policy-based assistance in non-R&D areas as well as governmental funding support in the R&D stage. A leading example is the R&D planning support program for SMEs.

This study researched 538 SMEs that applied to receive benefits through the “R&D planning support program for SMEs” in 2011 and analyzed the effect of this governmental support policy in the R&D planning stage on the financial performance of these SMEs. The SMEs included in this research were classified into firms that applied for the program, the firms selected for the program, and the firms that received further R&D funding through linked projects. This study tracked the financial performance of each of these groups.

3.2. Analysis Methodology

As shown in Figure 1, this study is aimed at companies that supported R&D planning program, and these groups are divided into those selected for R&D planning program (n = 85), those who have been dropped out (n = 93) and those who have been selected for follow-up project (n = 62). In order to identify whether difference of the sales, which are the observational variables of the latent growth model, the homogeneity of each group was analyzed and the latent growth model was analyzed as shown in Figure 2.

The growth curve model was proposed by Potthoff and Roy [31] in an expanded form to analyze repeated measurement data. Since the 1980s, there have been many studies using the latent growth model.

In general, panel data has the advantage of being able to determine the state of each measurement time point and the change of time points by measuring the same object repeatedly over time, having a time series data type and simultaneously measuring the current cross-sectional data. In the past, as the longitudinal data for measuring the same object for a long time, small-scale experimental data in the medical and pharmaceutical fields or time-series data commonly found in the economic field were common, but panel data were released from various institutions to expand research into various fields.

In this paper, we analyzed the group differences between SMEs that fulfilled R&D planning support programs and those who did not, using the financial data of small and medium companies that were repeatedly measured, and the magnitude of the effect using the latent growth model.

We used the IBM SPSS Statistics (version 20.0) to analyze between-group homogeneity. First, among the firms that applied to the planning support project in 2011, we analyzed firms that had trackable data on sales revenues and tested the homogeneity between the group that received support from the R&D planning support program and the group of firms that were rejected after submitting the application by performing the Kolmogorov-Smirnov and Shapiro-Wilk normality tests and then analyzed the data using the t-test or Mann–Whitney U-test.

When testing the homogeneity of two groups, the t-test is more effective than the Mann-Whitney U test because it shows the differences between the two populations more clearly. This is because the t-test utilizes more information from the data. The disadvantage of using the Mann-Whitney U test is that it utilizes rankings rather than actual observations, which means there is a loss of valuable information. Therefore, if the t-test satisfies the postulates, the t-test should be adopted, but if the data is originally provided in the form of rankings or is clearly derived from non-normal distribution, then the Mann-Whitney U test would be the more suitable test method.

Latent growth modeling is an analysis technique that uses longitudinal data consisting of measurements of the same subjects over a period of time, to determine the form of changes in the subject. This research method uses longitudinal data representing at least three repeated measurements [32]. The latent growth model analysis process consists of two stages. The first stage sets up the unconditional model and the second stage sets up the conditional model [33]. First, from the unconditional model, we obtained the intercept, slope and quad of the changes in sales revenue in the period of three years after the firms joined the R&D planning support project, in both the firms that were selected for the project and those that were rejected. Next, in the second stage of the conditional model, we connected the intercept, slope and quad obtained from the first stage to factors we anticipated to affect change, and observed the changes in their values. Our analysis focused on setting up the unconditional model for the first stage.

Preceding studies have offered various types of latent growth model analyses for determining the degree of change in the analyzed subject [34,35,36,37]. For latent growth model analysis, six types of models have been proposed, including the null model, 1-factor free parameter change model, linear change model, second year change model, third year change model, and 2-factor free parameter change model. The linear change model is considered the most basic type of latent growth model and consists of the intercept, slope, and two latent factors. The path coefficients of the intercept factor are all restricted to 1, and the slope factor is presumed to show linear change from the second year and the path coefficients of the slope factor are restricted to (0,1,2,3,4), with the postulate that there are changes each year. The second-year change model restricts the path coefficients of the slope to (0,1,1,1,1), with the assumption that there are changes in the first year and the second year but there are no further changes after the second year. The third-year change model restricts the slope’s path coefficients to (0,0,1,2,3), with the assumption that there are no changes in the first year and second year but there are changes from the second year up to the fifth year.

Generally, the model with the best fit is selected from among these six types of linear models or the null, linear, and secondary models are compared to select the model with the best fit. In this study, we compared the fit analysis results of the six types of linear models shown in Figure 3 (although we also reviewed the secondary model, it did not yield any significant results and therefore it was not included in this study).

4. Results

4.1. Homogeneity of the Group Recipients in the R&D Planning Support Project

Among the firms that applied to the R&D planning support program in 2011, we distinguished the group of firms that were selected and received support from the program and the group of firms that were rejected, and before testing the groups’ homogeneity, we analyzed the descriptive statistics of the annual sales revenues of each group. The results are presented in

Table 1. Sales revenue statistics of SMEs after applying for R&D planning support. (Unit: Thousand Korean won).

Year of Sales	2011		2012		2013
Group	Rejected	Selected	Rejected	Selected	Rejected	Selected
Number of Cases	93	85	93	85	93	85
Mean	8,305,974	10,269,770	9,116,233	10,898,295	8,735,834	10,864,483
95% lower limit	3,821,879	5,636,349	3,733,251	6,275,666	3,817,154	5,865,244
95% upper limit	12,790,069	14,903,192	14,499,215	15,520,925	13,654,514	15,863,723
Standard deviation	21,772,993	21,481,358	26,137,636	21,431,325	23,883,168	23,177,354
Minimum value	38,944	10,000	9207	190,000	29,629	154,900
Maximum value	164,320,278	181,183,301	233,040,286	164,339,013	213,870,763	184,206,105

. According to these results, among the 178 firms analyzed, 93 firms representing 52.4% of the total were rejected, while the remaining 85 firms were selected and received assistance through the planning support program. Observing the firms’ sales revenues in the three years after applying to receive support, we found that the firms that received support had consistently higher mean sales revenues.

To test the difference between the two groups, a normality test must be performed. According to the descriptive statistics, both the group that received planning support and the group that did not were far greater than 30, which led us to anticipate normality to some degree. However, the results of the Kolmogorov-Smirnov test and Shapiro-Wilk test, which we performed to verify normality, showed that the probability of significance for all variables and groups was lower than 0.05. Consequently, the null hypothesis of normal distribution was rejected, and we concluded that normal distribution did not occur in any of the groups and variables. Therefore, in this study, we did not use parametric statistics (t-test) and instead used non-parametric statistics (Mann–Whitney U test) to analyze the between-group homogeneity.

Table 2. Mann-Whitney test of sales revenues for 3 years after application to the R&D planning support project. (Sales revenue unit: Million Korean Won).

Dependent Variables	Application Rejected (n = 93)	Support Received (n = 85)	Z	p (Approximate Probability of Significance) (Two-sided Test)
	Mean ± SD
Sales revenue in 2011	8306 ± 21,773	10,270 ± 21,481	2.149 *	0.032
Sales revenue in 2012	9116 ± 26,138	10.898 ± 21,431	1.782 *	0.075
Sales revenue in 2013	8736 ± 23,883	10,864 ± 23,177	1.532	0.126

Note: * p < 0.1 (one-sided test, p < 0.05).

shows the results of the homogeneity test performed on the group of firms selected after applying to the 2011 R&D planning support program and the group of firms that were rejected. Establishing probability of significance of 0.1 (one-sided test 0.05) based on these results, we found a statistically significant difference between the selected group and rejected group in terms of the sales revenues in 2011 and 2012. The firms selected to receive R&D planning support had statistically significant, higher sales revenues compared to the rejected firms.

4.2. Homogeneity of Groups that Applied for Follow-Up Research and Development Projects after R&D Planning Support

Table 3. Sales revenue statistics of SMEs that received R&D project support following the planning stage (Unit: Thousand Korean won).

Year of Sales	2011		2012		2013
Group	Rejected	Supported	Rejected	Supported	Rejected	Supported
Number of cases	23	62	23	62	23	62
Mean	5,984,700	11,859,393	5,892,786	12,755,178	6,317,363	12,551,318
95% lower limit	2,421,639	5,631,286	2,728,209	6,533,525	2,451,450	5,831,292
95% upper limit	9,547,761	18,087,501	9,057,363	18,976,830	10,183,275	19,271,345
Standard deviation	8,239,578	24,524,699	7,318,084	24,499,282	8,939,922	26,461,750
Minimum value	163,682	10,000	471,812	190,000	260,640	154,900
Maximum value	29,342,747	181,183,301	25,793,815	164,339,013	38,424,020	184,206,105

shows the descriptive statistics analysis results of the annual sales revenues of each group of firms selected for research projects. According to these results, among the 85 analyzed firms that were recipients of R&D planning support in 2011, 62 firms representing 72.9% of the total were selected for inclusion in follow-up support projects, while the remaining 23 firms were rejected and therefore only received support for the initial duration of the planning support program. The sales revenues in the three years after application for support demonstrated that the mean sales revenue among the first group that received follow-up research and development support was nearly twice as high compared to the mean sales revenue among rejected firms.

As above, we used the Mann–Whitney U-test, a non-parametric statistical method, to perform a homogeneity test on the group selected for the projects in follow-up to the 2011 R&D planning support and the group of firms that were rejected, and the results are shown in

Table 4. Mann-Whitney test of the sales revenues of firms that applied to the R&D planning follow-up project (3 years after receiving project support. Unit of sales revenue: Million Korean Won).

Dependent Variables	Rejected from the Follow-up Project (n = 23)	Selected for the Follow-up Project (n = 62)	Z	p (Approximate Probability of Significance) (Two-sided Test)
	Mean ± SD
Sales revenue in 2011	5986 ± 1718	11,859 ± 3115	1.692 *	0.091
Sales revenue in 2012	5893 ± 1526	12.755 ± 3111	1.602	0.109
Sales revenue in 2013	6317 ± 1864	12,551 ± 3360	1.335	0.182

Note: * p < 0.1 (one-sided test, p < 0.05).

. Establishing a probability of significance of 0.1 (one-sided test 0.05) based on these results, we found that only the sales revenue in 2011 exhibited a statistically significant difference between the selected group and the rejected group. It was only at the end of the year in which the firms were selected for the follow-up project after R&D planning support that these selected firms showed statistically significant higher sales revenues compared to the rejected firms.

4.3. Latent Growth Model Analysis to Measure the Effects of the R&D Planning Support Project

As discussed above, although the Mann–Whitney U test can be used to test the homogeneity of a group with non-normality, this test makes it difficult to interpret the difference (i.e., effect), because it uses only data on rankings. Therefore, in this study we performed latent growth model analysis to derive an expected sales estimate model for the groups and used the obtained difference to determine the effects of the project. First, to select a fitting latent growth model, we performed model fit tests on the six types of models described in Figure 3. According to the results, shown in

Table 5. Results of fit index analysis for non-structural basic equation modeling.

	Fit	χ2	CMIN/df	NFI	TLI	CPI	RMSEA
Index Latent Growth Model
model 1		115.433	19.239	0.875	0.941	0.881	0.321
model 2		88.157	22.039	0.905	0.932	0.909	0.345
model 3 (0,1,2)		51.178	17.059	0.945	0.948	0.948	0.301
model 4 (0.1.1)		33.365	11.122	0.964	0.967	0.967	0.239
model 5 (0,0,1)		112.922	37.641	0.878	0.881	0.881	0.455
model 6 (0.1.a)		16.118	8.059	0.983	0.977	0.985	0.200
criteria				<0.9	<0.9	<0.9	<0.1

, all six types of models had a problem of fit. When the RMSEA is 0.05 or below, this indicates a very strong fit, whereas a RMSEA of 0.1 or below is a normal level of fit [38], but all six models tested were found unfit.

To address this problem in the models, as shown in Figure 4, we proposed a new model that takes account of the autocorrelation of the sales revenues. In other words, we considered the temporal relationships among the sales revenues. To select a suitable latent growth model that takes account of autocorrelation, we again performed model fit tests on the six model types as explained in Figure 3 and the results shown in

Table 6. Result of fit index analysis for the additional non-structural latent growth model considering autocorrelation.

	Fit	χ2	CMIN/df	NFI	TLI	CPI	RMSEA
Index Latent Growth Model
model 1		89.684	14.947	0.903	0.955	0.909	0.281
model 2		67.659	16.915	0.927	0.948	0.931	0.300
model 3 (0,1,2)		39.598	13.199	0.957	0.960	0.960	0.263
model 4 (0.1.1)		13.915	4.638	0.985	0.988	0.988	0.143
model 5 (0,0,1)		68.846	22.949	0.926	0.929	0.929	0.352
model 6 (0.1.a)		5.433	2.717	0.994	0.994	0.996	0.098
criteria				<0.9	<0.9	<0.9	<0.1

. According to these results, model 4 and model 6 had strong fit. Model 6, in particular, satisfied all the fit indices.

Based on the results shown in Table 6, we selected model 6 as the suitable model, from among the fit latent growth models that incorporate consideration of autocorrelation. In accordance with the results of our analysis of model 6, we fixed a as 1.15 for 178 cases, which represented all the available cases.

As shown in Figure 4, model 7 (0, 1, 1.15) was developed to determine the parameters for autocorrelation. The fitness of model 7 was confirmed for total program application (n = 178), project rejected (n = 93), project selected (n = 85), and follow-up support project selected (n = 62) as shown in

Table 7. Analysis results of the fit index for the non-structural latent growth model by group (selected / rejected).

	Fit Index	χ2	CMIN/df	NFI	TLI	CPI	RMSEA
Case Explanations
Applied group (n = 178)		37.971	2.712	0.989	0.995	0.993	0.051
Rejected group (n = 93)		37.981	2.532	0.989	0.996	0.993	0.048
Selected group (n = 85)		21.175	1.412	0.994	0.999	0.998	0.025
Additional Supported group (n = 62)		55.321	3.688	0.983	0.993	0.988	0.066
criteria				<0.9	<0.9	<0.9	<0.1

. The results show that the model is very suitable in all cases.

For each group analyzed to be suitable fit models in Table 7, we found that the intercept, slope and autocorrelation coefficient (B1, B2) were all statistically significant and they all differed, as shown in

Table 8. Estimate of factor loading and autocorrelation coefficient by group (selected/rejected).

	Factor Loading	Intercept	Slope	B1	B2
Case Explanations
Applied group (n = 178)		8,601,335 ***	−2,554,840 ***	0.490 ***	0.346 ***
Rejected group (n = 93)		8,600,897 ***	−2,542,102 ***	0.488 ***	0.346 ***
Selected group (n = 85)		8,877,380 ***	−2,410,193 ***	0.259 ***	0.354 ***
Additional supported group (n = 62)		8,984,821 ***	−2,150,945 ***	0.223 ***	0.321 ***

Note: *** p < 0.001.

. In particular, the groups that received planning support or follow-up project support had large intercepts and had relatively high slopes, and the autocorrelation coefficients were relatively small.

Using the model derived in Table 8, we measured the effect of the program based on the differences in expected sales revenues. According to

Table 9. Analysis results on increases in sales revenue among SMEs that received support compared to rejected SMEs. (Unit: Thousand Korean won).

Case Explanations	After 1 year	After 2 years
Selected for program (n = 85)	408,392	428,179
Selected for follow-up project (n = 62)	775,081	833,755

, compared to the SMEs that were rejected from the planning support program, those that received planning support showed an increase in sales revenue of approximately 410 million Korean won after 1 year and 430 million Korean won after 2 years. If a firm was selected for the follow-up support project as well, the sales revenue increased by 780 million Korean won after 1 year and 830 million Korean won after 2 years.

5. Discussion and Concluding Remarks

R&D projects that ensure innovation require a lot of labor, capital and other resources. In terms of securing resources, large companies have no difficulty in securing such resources, but SMEs do not. In addition, imperfect capital markets will make external finance available to only a small subset of startups and other SMEs, a phenomenon well understood [39,40].

In terms of RBV (resource-based view), securing scarce resources for competitive advantage depends on the size of the firm. Therefore, small SMEs are in a relatively unfavorable situation in competition with large companies. As a result, market failure inevitably occurs in the market logic of competition.

The lack of resources can lead to the disruption of SMEs’ innovation activities, providing some justification for policy intervention. The Korean government has also provided policy to support SMEs’ technological innovation. This is meaningful in that it provides the smallest resources to secure the sustainable growth of SMEs, creating a fair competitive environment with existing companies.

Preceding studies that attempted to verify the effects of these governmental support policies were inconclusive in their findings; some arguing that the government’s R&D funding support for SMEs had positive effects on the SMEs’ R&D and business performance, and some concluding that the effect was insignificant [41,42,43,44].

Most of the studies for policy assessment utilized cross-sectional analyses based on financial data or survey data for a single year, obtained from SMEs that received governmental funding. By contrast, this study overcame the limitations of such cross-sectional analyses by analyzing three years of financial data from the SMEs that benefited from supportive R&D funding, and applied a latent growth model to analyze the growth of the beneficiary firms over a period of time, and thus offered more reliable conclusions about the effects of the policies.

There is continuing demand for performance analyses evaluating the effects of governmental intervention, to enhance the efficiency of investments made by government policies and to improve the R&D success rate among SMEs.

Upon analyzing the effects of R&D planning support projects for SMEs using the latent growth model, we found that the R&D planning support program had a significant effect on increasing the sales revenues of the supported firms and furthermore, found that the firms that continued to receive support for the follow-up research and development project in the second year and third year demonstrated even greater increases in revenue. We compared the SMEs that were selected and rejected from the R&D planning support program, and found that the support increased sales revenue by approximately 400 million Korean won or more; and in the case of firms that additionally received follow-up R&D funding, the effect was up to 800 million Korean won, indicating that project had a positive impact on sales revenue. Although we cannot absolutely conclude that the entirety of the increase in sales revenues among the supported firms can be attributed to the effect of the program, our findings are consistent with the results of preceding studies that demonstrated that policies supporting SMEs have a positive effect on increasing the assets of SMEs [40,42]. This study also corroborates the findings of preceding studies indicating that these policies designed to support SMEs result in explorative activities required for technology innovation (technology innovation activities required for searching information on new technologies, learning, and acquisition) [44] and demonstrate that these programs provide SMEs with opportunities to secure the R&D capabilities they need in the long term.

There are various studies, such as direct support such as R&D subsidies as a policy tool, which affect the innovation performance of SMEs, and as an alternative or complementary relationship to private investment [45,46,47]. However, in terms of RBV, it is reasonable to view direct R&D subsidies as achieving innovation outcomes through competencies complemented by the resources. The results of this study can help the SMEs’ competitiveness management and R&D planning capabilities to be supplemented by providing information at the R&D planning stage and improving the planning capacity that can enhance the efficiency of SMEs’ technological innovation.

This study is meaningful in that it analyzes policy performance by applying a latent growth model to time series data collected by a mechanical observation method to solve the problem of policy evaluation by survey. Additionally, in analyzing the impact of policy support at the R&D planning stage on a financial performance of the company, it has an important distinction in that it proposed an alternative policy evaluation to verify the validity of the policy support at the planning stage.

In particular, in Korea, there is a need to solve the problem of low R&D investment efficiency in policy execution and R&D performance. This study presents an input-output analysis in terms of policy input at the R&D planning stage and financial performance using a non-survey method.

When comparing the R&D planning supporting firms with unsupported firms, there is a significant difference in sales, and the amount of increased sales is greater than the cost of R&D planning policy.

In the case of follow-up R&D projects, the sales increase effect is lower than that of the planning stage, but the sales increase effect is greater than the input cost. Therefore, the policy investment efficiency of the program in the R&D planning stage is somewhat secured.

However, in the case of follow-up projects, they show less investment efficiency than R&D planning programs, and it is necessary to consider other linkage alternatives that can increase the efficiency of policy investment.

Alternative follow-up programs may be considered a cooperative R&D program rather than securing an R&D subsidy in terms of complementing their innovation capabilities. This is in line with preceding research findings that the direct effect of government subsidies on early stage ventures regarding sales is short. As government subsidies contribute to greater access to financial and human resources, improving the performance of an R&D planning support program requires an approach that complements the resources and capabilities for innovation [40].

Considering the results of Yoo et al., the support program at the R&D planning stage may be expected to be synergistic in connection with the cooperative research program [27].

In terms of SMEs, sales growth is required for sustainable growth, and government support programs are needed due to the limited resources. Therefore, considering the characteristics of R&D projects owned by SMEs, it is necessary to determine the types of resources that are lacking, and to select government support programs.

However, this study has limitations in that it does not take into account the differences in the firm’s capabilities in terms of input, and does not consider the endogenous effects of the output. Follow-up studies considering this are needed.

On the other hand, one limitation of the model used in this study was the inability to analyze multiple control variables together. Since it is highly probable that the selection process of the R&D planning support program tends to filter for SMEs that already have strong capabilities and growth potential, it is possible that these pre-existing strengths were the reason they demonstrated higher performance compared to the rejected firms. Additionally, our analysis excluded firms if we could not obtain data on their sales revenues after applying to the project, and this exclusion may have limited the level of precision we could achieve in our analysis.