Next Article in Journal
The Study on the Difference in Corporate Performance and Employment Outcomes According to the Results of Equity-Based Crowdfunding Investment
Next Article in Special Issue
Relationship between Organizational Climate and Innovation Capability in New Technology-Based Firms
Previous Article in Journal
Artificial Intelligence Modelling Framework for Financial Automated Advising in the Copper Market
Previous Article in Special Issue
Exploring the Role of Knowledge, Innovation and Technology Management (KNIT) Capabilities that Influence Research and Development
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Comparing National Innovation System among the USA, Japan, and Finland to Improve Korean Deliberation Organization for National Science and Technology Policy

1
Department of Industrial and Management Engineering, Inje University, Gimhae 50834, Korea
2
Division of Data Analysis, R&D Investment Analysis Center, Seoul 08826, Korea
3
Nuclear Safety and Security Commission, Seoul 03154, Korea
4
Department of Industrial and Systems Engineering/ERI, Gyeongsang National University, Jinju 52828, Korea
*
Author to whom correspondence should be addressed.
J. Open Innov. Technol. Mark. Complex. 2019, 5(4), 82; https://doi.org/10.3390/joitmc5040082
Submission received: 7 August 2019 / Revised: 20 September 2019 / Accepted: 1 October 2019 / Published: 9 October 2019
(This article belongs to the Special Issue Technology Driven Innovation, Research Management and Policy Making)

Abstract

:
As the Korean government expands the budget for national research and development, the need for an institute that deliberates, coordinates, and operates research development and its budget has increased. In response to these demands, the National Science and Technology Council (NSTC) was recently established. However, to achieve a creative economy, which is the economic system where value is based on novel imaginative qualities rather than the traditional resources of land, labor, and capital, more efficiently, fundamental research regarding the current state of the Korean national administration system of science and technology in Korea is required. Accordingly, this study first analyzes the function and organizational structure of the NSTC in Korea. Second, it investigates the current state of the NSTC in other countries. Finally, the study derives several implications for improving NSTC operation based on the benchmarking study and suggests an operational improvement plan for NSTC with respect to enhancement of function, operation of organization, human resource management, and improvement of the relationships between other departments. The study contributes to analyze the current state of the NSTC in Korea and science and technology (S&T) Councils in other major countries, systematically and in detail. In addition, based on benchmarking study, this study derived operational improvement of NSTC in Korea with four perspectives, including enhancement of function, operation of organization, human resource management, and improvement of the relationships between other departments.

1. Introduction

In recent years, science and technology (S&T) have become more important and the Korean government has invested a large sum of the budget in national research and development (R&D) [1,2]. In addition, several advanced countries steadily have conducted open innovation activities to improve the capacity of S&T innovation, due to facing the dynamic and complex environment of science and technology [3]. Thus, R&D investments and their plan such as technology roadmap also have become more important to maintain the performance of national innovation activities [4]. Especially, R&D management agencies and policies have been taking on important roles in the national R&D environment [5,6]. In the recent era of open innovation, efficient R&D investment at the national level has a greater impact on the development of national competitiveness [7,8,9].
In response to these changes in circumstance, Korea invests more than 3% of its Gross Domestic Product in R&D, which is high compared to other Organization for Economic Cooperation and Development countries [10]. However, in terms of the establishment of long-term R&D roadmap, it is hard to say that the R&D investment activities in Korea are progressed efficiently [11]. Actually, there are some concerns regarding the performance of R&D investment, which stems from not only diversity of R&D business but also issues concerning demand and budget allocation among research groups. Under such circumstances, establishment of an effective budget allocation system and long-term operating plans in terms of national level are required [12,13,14], in particular, balanced management of national R&D based on sector diversification.
In addition, as the Ministry of Science and Technology and the Ministry of Information and Communication were disbanded in Korea, many experts worried that the competitiveness of science and technology and national Information and Communication Technology (ICT) might be weakened [15]. Reflecting on this issue, in 2013, the Korea government established the National Science and Technology Council (NSTC), an institute responsible for the deliberation, resolution, and adjustment of national R&D. Compared with the National Science and Technology Committee, which was formerly the NSTC, the NSTC is the organization that much focuses more on deliberation [16]. Therefore, the system has been altered to commit itself to its original mission of administrative commission. It is expected that the NSTC will play a role as a new regulation for national science and technology.
Although the NSTC was newly established, there has been a constant need for an in-depth study of the current state of the national administration system of science and technology due to the changing demand to involve development research for pursuit of profit and basic research for public business. In addition, issues related to the effective budget allocation of NSTC has been raised. In this environment, vertical regulations-based regulation, which was successful in the past, is no longer a sustainable alternative to contribute to the development of national science and technology. Therefore, to meet these needs, the current state of the national administration system of science and technology in Korea should be investigated thoroughly.
In response to a paradigm shift in the innovation economy, the administration system of science and technology in Korea should be changed for meeting the requirements of the age of the post-catch-up innovation regime and creative economy. An appropriate approach to derive fruitful implications is to conduct an in-depth study regarding the current state of the national administration system of science and technology in Korea and a comparative analysis of the NSTC between Korea and other countries. However, there is a lack of studies dealing with this issue.
Accordingly, the goal of this study is to contribute to the advancement of the NSTC in Korea in three ways:
(1)
To analyze the function and organizational structure of the NSTC,
(2)
To investigate the current state of the NSTC in other major countries in detail,
(3)
To derive realistic implications for improving the NSTC operations based on the benchmarking study.
The value added to this area of this paper is to suggest an operational improvement plan for the NSTC in Korea based on the benchmarking study. To our knowledge, this is the first study that analyzes the current state of the NSTC in Korea and S&T Councils in other major countries, systematically and in detail. Based on benchmarking study, we derived operational improvement with four perspectives, including enhancement of function, operation of organization, human resource management, and improvement of the relationships between other departments. In addition, since there have been no in-depth comparative studies on the national science and technology management system of other developed countries, there was a lack of references in establishing the policy direction of the Korean government. Therefore, this study can be useful in establishing the policy direction and improving the management system of science and technology in the new government as it has been in-depth analysis of the national science and technology management system.
The remainder of this paper is organized as follows. Section 2 explains the overall research framework of the paper, while Section 3 provides an overview of the operating system of the NSTC in Korea. Section 4 investigates the current state of the NSTC in Japan, the United States (U.S.), and Finland. In addition, implications are derived for improving the NSTC operation in this section. Section 5 suggests an operational improvement plan and, finally, Section 6 concludes with contributions and future research suggestions.

2. Research Framework

The framework of proposed approach is shown in Figure 1. As shown, the process primarily consisted of three parts with respect to overall perspective: (1) analysis of the current system of the NSTC; (2) analysis of the current state of science and technology councils in other major countries; and (3) suggestions for an operational improvement plan for NSTC. In the analysis of the current system of the NSTC in the Korea stage, a brief history of the NSTC is reviewed.
As the second stage, the primary functions and organizational structure of the NSTC are analyzed. In the stage of analysis of the current state of science and technology councils in other major countries, S&T councils in three major countries including Japan, the U.S., and Finland are analyzed as the benchmarking study. Here, we utilize benchmarking study as a methodology because this approach is effective tool for analyzing National Innovation Systems to serve as a starting point for a detailed examination of similarities and differences between benchmarking group and specific group [17]. In addition, for benchmarking study, we selected three leading countries from Europe, Asia, and America. First, in Europe, we selected Finland because Finland was the first country to adopt the concept of a national innovation system (NIS) as a basis for its technology and innovation policy. Second, in Asia, Japan is selected as a benchmarking group because Japan is the country with the most advanced science and technology level in Asia and systematically establishes the direction of national science and technology policy such as determining future R&D priorities and evaluating major R&D projects. Lastly, in America, we selected U.S. because in the U.S., the level of science and technology is very high, and they are constantly striving to strengthen national competitiveness through the development of science and technology manpower and support of basic science. In particular, as the complexity of science and technology policy increases, in the U.S., various organizations are being operated to systematically establish science and technology policy.
As the last stage, in the suggestion of operational improvement plan for NSTC, basic directions for operational improvement with four perspectives, including enhancement of function, operation of organization, human resource management, and improvement of the relationships between other departments.

3. Review of Operating System of NSTC in Korea

3.1. Overview of NSTC

The NSTC was established in 2013 as a national science technology planning tower to achieve a creative economy. The major mission of the NSTC is to establish a vision for national S&T development. In accordance with basic science and basic technology in Korea, the NSTC was officially launched under the Prime Minister’s Office in March 2013 [16].
The leadership of the NSTC consists of the Prime Minister and private sector co-chairmen. In addition, the 13 ministers of related government departments participate as government delegates; private delegates and experts also participate. Table 1 shows the history of each science and technology council.

3.2. Major Functions of the NSTC

The NSTC plays a role in adjusting major science and technology policies, deliberating human resource policies regarding innovation of science, technology, and industrialization, adjusting regional technological innovation policies, and deliberation on the operational R&D budget [18]. Detailed explanations are provided in Table 2.

3.3. Organizational Structure of the NSTC

The NSTC is composed of a plenary session, a steering committee, seven expert committees, two special committees, and two consultation committees, as shown in Figure 2.
Specifically, first, the plenary session deliberates issues related to the establishment and coordination of major policies affecting science technology and efficient operating system of the national R&D business budget. Second, the operating committee provides practical advice regarding issues related to science technology policy and pre-examines the deliberation and decision matters prior to plenary session. Third, expert committee reviews and coordinates the R&D budget in each field and makes decisions on the investment direction and propriety for major R&D businesses, makes connections and coordinates between businesses, and distributes and coordinates the R&D budget, etc. The special committee carries out temporary and intensive discussion on special issues and is composed of the special committee on Civil-Military Technology Cooperation and the special committee for Multi-Department Technology Collaboration. Lastly, the consultation committee consults in each policy field and promoted fundamental research and local science technology.

4. Current State of National Science and Technology Councils in Other Major Countries

4.1. Japan—CSTP (Council for Science and Technology Policy)

4.1.1. Background

Due to the increasing complexity of science policy, the mechanisms for evaluating S&T policy are weak in Japan. In 1995, the Japanese government developed an integrated S&T policy and successive 5-year S&T Basic Plan [19]. For this plan to be successful, Japan needed to construct a strong organization for managing S&T policy and, as the part the task to reform the administration in 2001, the Council for Science and Technology Policy (CSTP) was established [20]. Compared with its previous S&T council, the Science Council of Japan, the CSTP addressed a broad scope of S&T policies and projects and has stronger decision-making power than before. In addition, other S&T organizations cannot intervene in the activities of CSTP because the chairperson has the power to: (1) appoint overall members; (2) determine the direction of the S&T plan; and (3) allocate power to other organizations [21].

4.1.2. Primary Function

The CSTP has the leading role in developing overall S&T policy, including completing the S&T Basic Plan and evaluating other detail agendas related to basic S&T policy. In addition, other members of the Cabinet Office also support the CSTP in formulating important policies and decisions. The main functions of the CSTP can be summarized as follows:
-
Investigate and deliberate basic S&T policies: The CSTP suggests comprehensive strategies for S&T to address the national/social scientific needs. In addition, the CSTP modifies the direction of the primary policies through decisions made by external committees.
-
Evaluate key R&D projects in Japan: The CSTP prioritizes key R&D projects through evaluation with external experts. The CSTP also sets the evaluation criteria and subjects by discussing them with experts. Based on the evaluation, the CSTP determines whether R&D projects are enforced, revised, or rejected.
-
Deliberate the allocation of resources for S&T policies: The CSTP deliberates the allocation of government expenditures and human resources for S&T policies and projects. The Ministry of Finance cooperates with the CSTP when the overall scale of expenditure and other resources is determined.
-
Coordinate between S&T policies: The CSTP conducts coordination efforts between various S&T policies in different departments. The CSTP sets a coordination guideline of basic S&T policy with several ministers. In addition, the CSTP also makes instructions for sub-organizations of the government based on the S&T Basic Plan.

4.1.3. Organization of CSTP and Other Related Organizations

The CSTP is chaired by the Prime Minister and consists of six ministers, seven executive members, and a head of an affiliated organization. Ministers primarily deliver results of council and reflect these results when establishing policies. Executive members, who are eminent persons from various academic or industrial backgrounds, primarily provide S&T policy consultation based on their professional knowledge and opinion. However, only 14 members cannot conduct all of the main functions of the CSTP, therefore, a number of other organizational members support. Figure 3 shows the overall organizations that are related to S&T policy in Japan.
As Figure 3 shows, many organizations and groups participate in the CSTP, even though they are not directly involved in the CSTP. Among them, two groups primarily support the CSTP: First, The Executive Office in the Cabinet Office serves as the CSTP’s secretary. This organization provides additional information regarding S&T policy, both domestic and abroad. For example, it investigates S&T policy trends in developed countries and gathers diverse comments about the new policy from the industry. Second, the Ministry of Education, Culture, Sports, Science and Technology (MEXT) supports the CSTP when the CSTP mediates S&T primary policies and programs because MEXT is able to arbitrate between other government departments related to S&T. MEXT also provides basic direction for the successive S&T Basic Plan and conveys that to other government departments. Third, expert committees are utilized, which consider the policy related to their fields. They also not only offer professional information, such as S&T policy data, but also provide recommendations including evaluation criteria. They meet with the Minister’s CSTP executive members about once a week and consult regarding the direction of S&T policy or other projects. Expert panels can be altered according to circumstance and S&T policy. In 2014, five expert panels were operated by the CSTP. Table 3 shows the objective and roles of the five expert panels.

4.2. U.S.—NSTC (National Science and Technology Council)

4.2.1. Background

In the late 1990s, the U.S. government confronted issues in establishing and deliberating S&T policy. First, S&T policy has become more complex since numerous policies had been established to consider the needs of federal agencies [22]. Federal agencies can support small companies, which can play a role in innovation, so they can influence the development of S&T innovation [23]. Second, coordination between stakeholders such as governments, private sector, and academic fields was required to establish S&T policy because their interconnected networks were needed to operate national innovation systems [24]. To overcome this problem, the National Science and Technology Council (NSTC) transferred to a presidential council. In addition, other S&T organizations such as the Office of Science and Technology Policy (OSTP) and President’s Council of Advisors on Science and Technology (PCAST) also became involved in the new presidential organization, and with the NSTC [25]. Through this change, the U.S. government strengthened the power of NSTC to function as the primary organization for S&T policy. However, unlike Japan and Korea, the NSTC’s power should reflect the opinion from other three powers: administration, legislature, and judiciary officials in its deliberation process.

4.2.2. Primary Function

The NSTC has the leading role in: (1) coordinating S&T policy across Federal and enterprise research; and (2) evaluating S&T policy to strengthen the international competitiveness of S&T. However, unlike Korea and Japan, the NSTC does not determine the budget. Therefore, if the NSTC wanted to change the budget allocation, it would require agreement with other organizations. The primary function of the NSTC can be summarized as follows:
-
Coordinate S&T policy: The NSTC coordinates S&T policy by providing basic direction in S&T development. In addition, it assists in integrating the government and federal agencies, which have different S&T environments.
-
Evaluate S&T policy programs: The NSTC evaluates S&T policy decisions and programs and determines how consistent they are with the President’s stated goals. Even if the programs meet the stated goals, the NSTC still evaluates the performance of S&T policies based on whether the policies are sustainable S&T development.
-
Construct a collaboration system: The NSTC constructs a global collaboration system with other developed countries where the NSTC promotes scientific experts to exchange scientific information and participate in S&T projects with foreign development teams.
-
Set guidelines regarding deliberation: The NSTC publishes a report that contains policy objectives for the coordination and deliberation of S&T policy. Based on this report, the NSTC deliberates R&D programs in administrative departments and federal agencies.

4.2.3. Organization of NSTC and Other Related Organizations

Chaired by the President, the NSTC is composed of the Vice President, the Director of the Office of Science and Technology Policy, 14 Cabinet Secretaries, seven Agency Heads, and four President’s assistants. The number of actual members of the NSTC is larger (30 persons) than the S&T councils in Korea and Japan, but people from various academic or industrial backgrounds do not participate and their opinions are provided indirectly through agency and institute leaders.
To accomplish the functions of the NSTC, a number of expert committees, subcommittees, and working/project groups make up the NSTC’s organization. Figure 4 shows the organization of the NSTC and primary expert committees.
The role of the committees is similar to committees of the NSTC in Korea as they specialized in advising and assisting in the policy-making and evaluation process in their fields. They also support maintaining a strong relation between the CSTP and Federal S&T policies with other cooperated organizations. Committees can be changed or reorganized according to circumstances and S&T policies in the U.S. In 2014, five committees operated to assist in the primary functions of the NSTC. Table 4 shows the objectives and roles of the five committees, which are more focused on fundamental science and education than those in Korea and Japan.
In the U.S., three groups are involved with decision-making regarding S&T policy. In the administration, the OSTP is concerned with evaluating the validation and effectiveness of R&D plans and related works. In addition, it can remonstrate the President’s decisions concerning S&T Basic Plan and detailed programs. Second, the Office of Management and Budget establishes the federal budget for R&D programs and prioritizes them. It also determines the allocation principle regarding program management. The judicial system deals with policy decision-making related to the administration and provide certain judgments as a reference for S&T’s evaluation. While the legislature does not possess S&T-specialized organizations, several subcommittees of legislature participate in evaluating general S&T policy and establish R&D program budgets. In addition, they are concerned with public issues of S&T policy.

4.3. Finland—RIC (The Research and Innovation Council)

4.3.1. Background

Prior to the establishment of the Research and Innovation Council (RIC), the Science and Technology Policy Council (STPC) controlled the S&T innovation policy system in Finland [26]. The STPC influenced overall S&T policy and promoted S&T innovations since 1960s [27]. In the 2000s, the STPC began to promote horizontal innovations that required collaboration between policy sectors [28]. However, Finland’s administration created the innovation-lead strategy for national development and more radical innovations were required. Therefore, it wanted to create a general plan that covered not only S&T developments but also comprehensive successful innovation systems. To create a more comprehensive innovation, the administration suggested that every stakeholder in S&T policy participate in the decision-making process and attempted to allow innovation leaders to affect the entire society. To control innovations, overall stakeholders collaborated with others in S&T innovations and strong S&T policy was required. To do this, The STPC was renamed RIC [16].

4.3.2. Primary Function

The RIC coordinates S&T policy across stakeholders and evaluates S&T policy to promote the development of science and technology. In addition, it created the basic S&T plan for overall S&T areas. In this process, the RIC investigates the relationship between other policy fields and reflects this in plans. Unlike other organizations, the RIC also mediates legal issues with the judiciary and directly manages researcher training programs. However, its evaluation functions are weaker than in other organizations—it only measures the impact and performance of S&T policy. The primary functions of the RIC can be summarized as follows:
-
Coordinate S&T policy: The RIC coordinates the S&T policy-making process. In this step, it provides solutions to the primary issues in S&T policy. In addition, it assists in constructing a basic S&T policy system to promote the participation of stakeholders.
-
Evaluate S&T policy program: The RIC evaluates the performance of S&T policy decisions and programs to identify the impact and possibility of realization. Based on evaluation, the RIC appropriates special budget to public research innovation and allocates it to several R&D projects.
-
Construct internationalization system: The RIC supports issues related to internationalization, such as language barriers. In addition, it controls international collaboration systems that involve several projects conducted on a large scale.
-
Settle the legal issues regarding policy: The RIC mediates between S&T organizations when their conflicts are derived from legal issues related to research, technology development, and science education.
-
Train the researcher: The RIC created training programs to research focused S&T fields. Universities, research institutes, and science education committees recommend the direction of program and support the management of training programs.

4.3.3. Organization of RIC and Other Related Organizations

The membership of RIC, chaired by the President, consists of the two Vice Chairpersons, seven Cabinet Ministry, 10 external experts, and five standing members in the administration. The number of members of the RIC is larger (25 persons) than those of the S&T councils in Korea and Japan. Unlike the U.S., external experts from various academic or industrial backgrounds belong to the RIC through the nomination of the President. Expert committees also consist of the RIC’s organization. Figure 5 shows the organization of the RIC.
Many committees provide opinions regarding the RIC’s decisions. Among the committees, two are supported by the RIC. The Science Education Committee’s purpose is to increase the overall performance of developing new knowledge within the S&T fields. One core function of the RIC is training the researcher, and this committee develops educational activities and programs for researchers. The second committee is the Technology Innovation Committee and its purpose is to improve the overall effectiveness of technology and promote researchers and other stakeholders to participate in the innovation process. This committee also analyzes the economic/social influence of new S&T policy. In addition, it facilitates technology innovation planning that covers the overall S&T fields. In this step, it suggests how to make and maintain innovative competitiveness. Background research is provided by other subcommittees to these two committees.
However, the RIC does not deliberate overall S&T policy levels. Figure 6 shows the S&T policy deliberation system in Finland. At the program level, each program autonomously deliberates its issues and policies.
In addition, the detailed policies of each department are deliberated by the Academy of Finland (AF) and Business Finland. Initially, the AF organization primarily supported fundamental research in universities. However, the AF is also involved in developing advanced research system and environment and maintains a collaboration system between research institutes, universities, and the government. Business Finland was established for the purpose of supporting R&D of companies and universities, providing innovation fund, supporting information on export, investment and tourism, overseas expansion of small and medium companies, and attracting foreign direct investment. Actually, on 1 January 2018, Finpro which is the Finnish trade promotion organization and Tekes which is the Finnish Funding Agency for Innovation were united as Business Finland. Business Finland’s mission is to catalyze new growth and create opportunities for Finland through innovation and international expansion.
Actually, for a long time, Finland’s national innovation system (NIS) had been one of the most successful and effective in the world. However, some years ago, the situation changed. In other words, Finland’s NIS run into system crisis. For example, Finnish RIC has not been functioning well because in previous government there were huge cuts in R&D budget. To overcome this crisis, Finland’s government tries to work out anticrisis policy for innovation by looking for optimal policy-mix to overcome the crisis, demonstrate in details complementary components of the relevant innovation policy. This approach is based on previously accumulated knowledge and all stakeholders of economic development consensus on one issue. Even though there was a crisis, some S&T policies have been doing well in recent years. For instance, the Finland startup program, run by Business Finland, attracts many startups from all over the world based on its active startup support environment, which includes Finland’s growing ecosystem, high quality of life, and open funding opportunities. Based on this S&T policy, the Finnish entrepreneurship ecosystem is expected to continue to grow.

4.4. Comparison of the S&T Councils in Three Countries

The following Table 5 shows the comparison of S&T councils in Japan, the U.S., and Finland.

4.5. Implications Based on Analysis of S&T Councils in Three Countries

In previous sections, the functions and characteristics of S&T councils in three countries (Japan, the U.S., and Finland) are analyzed. This study can derive four implications from the three S&T councils.
First, the overall authority of the three S&T councils is stronger than that of the NSTC. To lead the overall S&T innovation, the role of the S&T control tower becomes more important. According to the analysis, the three S&T councils belong to a presidential advisory body, such as a Cabinet Office in Japan. Therefore, they can review and control the overall national innovation system, even practical cases of R&D in nongovernmental fields. To do this, the three countries construct the legal mechanism for enhancing the authority of the NSTC. For example, the U.S. government legislates on essential particulars for operating the NSTC, and other departments cannot oppose the control of the NSTC. In addition, mediation function of councils also is enhanced by the connection with stronger organizations. The OSTP has a stronger authority by supporting the MEXT, which is one of primary departments in Japan. The MEXT arbitrates between other governmental departments and assist the OSTP in drastic mediation of S&T policy.
Second, the three S&T councils have an effective role sharing system to enhance the efficiency of operation. The three S&T councils have many suborganizations and supported organizations, while the councils deliberate and establish S&T policy. These organizations regularly participate in the decision-making process, as well as provide advice on S&T policy. The OSTP in the U.S. operates and supports the council and is concerned in evaluating the validation of R&D plans and related programs. Moreover, the three S&T councils actively appoint outside specialists, which can contribute to enhancing the capacity of fair decision-making. For example, in Finland, many S&T policies in individual programs rely on various outside specialists when they decide detailed plans and evaluate programs based on its complexity. Specialists set the criteria for evaluating program, and the RIC listens to their feedback when establishing the evaluation criteria of S&T policy.
Third, the three S&T councils strive to secure outstanding experts. They construct international collaboration systems that foreign experts participate. These experts provide information related to foreign technologies and evaluate the competitiveness of global S&T and its policy. In addition, councils such as the OSTP and the RIC regularly rotate the members who are selected experts from each of governmental departments. Experts, who have information related to technologies and their policies, are trained in policy deliberation.
Finally, the three S&T councils have various types of relationships with other fields, such as other governmental departments, academic fields, industrial fields, and citizens. Above all, all three S&T councils attempt to interact with other governmental departments, and various organizations participate in the process of establishing/deliberating on the S&T policies. However, organizations should be followed an S&T policy Basic Plan when they participate in establishing policies. For example, in the U.S., the judicial system and the legislature independently participate in the policy deliberating process and contribute to establishing practical policies. Moreover, there are various types of interactions with nongovernmental organizations. For example, the Japanese government designates independent administrative institutions, one of the academic organizations in Japan. These institutions can autonomously enforce the S&T policy in their programs and provide enforcement to the OSTP. They also provide direction in planning the reflected needs of academic fields. Many subcommittees in councils collect extensive opinions from the industrial field and citizens through industrial panels.

5. Operational Improvement Plan for the NSTC

Based on the analysis to the current state of the S&T councils in Japan, the U.S., and Finland, the following implications providing a desirable direction for the NSTC in Korea can be derived: (1) enhancement of function; (2) operation of organization; (3) human resource management; and (4) improvement of the relationships between other departments.

5.1. Enhancement of Function

Basic directions for enhancement of function can be defined in five areas: (1) total control function; (2) establishment of strategy function; (3) mediation function; (4) planning of open R&D policy; and (5) establishment of a reasonable and fair decision system based on expert opinion. Detailed explanations of the directions for function enhancement are summarized in Table 6. Moreover, the related councils are shown in Table 6 as well.
Our suggestion regarding the enhancement of function has the pros that NSTC can effectively adjust and deliberate policy because if our suggestions are made, there is an institutional mechanism to strengthen the powers needed for deliberations. However, in order to strengthen authority for policy coordination and deliberation functions, the interests of relevant ministries should be considered so it may be hard work.

5.2. Operation of Organization

Basic directions for enhancement of operation of the organization can progress in three areas: (1) the establishment of other supported organizations; (2) enhancement the capability of decision-making process; and (3) variety of information sources. Detailed explanations for basic directions for operation organization can be summarized in Table 7.
Our suggestion regarding the operation of organizations has the contribution in that a balanced system can be established that maintains independence and autonomy among ministries. However, in Korea, there is a difficulty in applying our suggestion because of the change in authority between the various ministries.

5.3. Human Resource Management

Basic directions for enhancement of human resources can progress in two areas: (1) the enhancement for the capability of S&T policy; and (2) the enhancement for the capability of globalization. Detailed explanations for basic directions for enhancement of human resource management are summarized in Table 8.
Basic directions for enhancement of human resource management has the pros that NSTC can effectively utilize experts with expertise in science and technology. However, in order for such human resource management to operate transparently, political intervention should not occur in the selection and training of personnel, but this has often happened. Therefore, this issue should be solved.

5.4. Improvement in the Relationships between Other Departments

Improvements in the relationships between departments can enhance in four areas: (1) government departments, (2) the public, (3) academic field, and (4) industrial field. Detailed explanations for basic directions for relationship improvements are summarized in Table 9.
If our suggestion for improvement in the relationships between other departments is implemented, it can lead to the deliberation of policies with a common purpose, based on active information exchange among ministries or publics. However, our suggestion requires a lot of time and money because the interests of various departments and various industrial fields need to be widely considered.

6. Conclusions

This study contemplates how to improve the NSTC in Korea. To do this, first, the operating system of the NSTC in Korea is reviewed in detail. In this step, the function and organizational structure of the NSTC are analyzed. Second, the National S&T Councils in Japan, the U.S., and Finland are analyzed. According to the analysis, the four councils have a common basic mission and function regarding S&T policy. However, there are some differences between the four councils, such as detailed function, authority, and association with other organizations. Third, this study suggests implications for providing a desirable direction for the NSTC in Korea. Implications include concrete explanations and references derived from the other three councils and four perspectives can be concluded: (1) enhancement of function; (2) operation of organization; (3) human resource management; and (4) improvement in the relationships between other departures.
The primary contribution of this study is to derive implications for the advance of the NSTC in Korea based on the comparative analysis of the current state of the NSTC among major countries. Despite the importance of S&T policy and management, there is a lack of detailed analysis regarding S&T organizations and councils. Therefore, this study analyzes the NSTC in three categories: history, function, and organization. In addition, based on these categories, this study also analyzes the other three S&T councils and identifies difference between the NSTC in Korea and the other councils. These differences can provide a basis for systematical implications for the NSTC in Korea.
Despite the major implications of this study, several limitations exist that suggest a path for future research. First, this study only investigated the National S&T Councils in the U.S., Japan, and Finland, leaving councils in other countries to be investigated thoroughly and derive plentiful implications. Future studies may consider not only developed countries but also developing countries that foster high-tech industries. Many developing countries have S&T strategies based on their economic growth. Therefore, their councils may provide a basis for other implications for the NSTC in Korea. Second, the performance of the NSTC in Korea should be analyzed as this study only analyzed function, organization, and related organizations. The efficiency of councils was not investigated and to overcome this limitation, future studies can identify the NSTC’s performance based on policy items in the national R&D agenda. Performance can be measured by criteria such as the number/scope of S&T policies deliberated by the NSTC and the number of derived S&T programs from S&T policies.

Author Contributions

Investigation, W.J.; project administration, Y.K.; supervision, J.J.; writing—original draft, D.K.

Funding

This research received no external funding.

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2017R1D1A3B03034060). And, this work was also supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIT) (No.2019R1A2C1090655).

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Nagano, M. R&D investment and the government’s R&D policies of electronics industries in Korea and Taiwan. J. Asian Econ. 2006, 17, 653–666. [Google Scholar]
  2. Oh, S.-H.; Lee, K.-J. Governance system of governmental R&D programs: Formation and transformation of the Framework Act on Science and Technology in Korea. Sci. Public Policy 2013, 40, 492–503. [Google Scholar]
  3. Jeon, J.-h.; Kim, S.-k.; Koh, J.-h. Historical review on the patterns of open innovation at the national level: the case of the roman period. J. Open Innov. Technol. Mark. Complex. 2015, 1, 20. [Google Scholar] [CrossRef]
  4. Jun, S.-P.; Seo, J.H.; Son, J.-K. A study of the SME Technology Roadmapping Program to strengthen the R&D planning capability of Korean SMEs. Technol. Forecast. Soc. Chang. 2013, 80, 1002–1014. [Google Scholar]
  5. Hwang, B.Y.; Bae, E.S.; Hong, H.D.; Kim, D.-c. Operational-Efficiency Improvement of Public R and D Management Agencies in South Korea. J. Open Innov. Technol. Mark. Complex. 2019, 5, 13. [Google Scholar] [CrossRef]
  6. Fernández-Sastre, J.; Montalvo-Quizhpi, F. The effect of developing countries’ innovation policies on firms’ decisions to invest in R&D. Technol. Forecast. Soc. Chang. 2019, 143, 214–223. [Google Scholar] [CrossRef]
  7. Song, X.; Zhou, Y.; Jia, W. How do Economic Openness and R&D Investment Affect Green Economic Growth?—Evidence from China. Resour. Conserv. Recycl. 2019, 146, 405–415. [Google Scholar] [CrossRef]
  8. Tou, Y.; Watanabe, C.; Moriya, K.; Naveed, N.; Vurpillat, V.; Neittaanmäki, P. The transformation of R&D into neo open innovation- a new concept in R&D endeavor triggered by amazon. Technol. Soc. 2019, 58, 101141. [Google Scholar] [CrossRef]
  9. Lee, B.; Sohn, E.; Won, D.; Yeo, W.; Kim, K.; Kim, S. The Efficiency Analysis of National R&D Planning for the Field of Precision Medicine in Korea. J. Open Innov. Technol. Mark. Complex. 2018, 4, 39. [Google Scholar] [Green Version]
  10. Lee, I.H.; Marvel, M.R. The moderating effects of home region orientation on R&D investment and international SME performance: Lessons from Korea. Eur. Manag. J. 2009, 27, 316–326. [Google Scholar] [CrossRef]
  11. Chen, C.-P.; Hu, J.-L.; Yang, C.-H. An international comparison of R&D efficiency of multiple innovative outputs: The role of the national innovation system. Innovation 2011, 13, 341–360. [Google Scholar]
  12. Lee, T.-H. Korean information and communications technology policies and future directions: An interview with Korean National IT Promotion Agency (NIPA) President Sooyong Park. J. Inf. Technol. Case Appl. Res. 2015, 17, 108–116. [Google Scholar] [CrossRef]
  13. Park, H.; Lee, J.; Kim, B.-C. Project selection in NIH: A natural experiment from ARRA. Res. Pol. 2015, 44, 1145–1159. [Google Scholar] [CrossRef]
  14. Jang, H. A decision support framework for robust R&D budget allocation using machine learning and optimization. Decis. Support Syst. 2019, 121, 1–12. [Google Scholar] [CrossRef]
  15. Kim, H.; Shin, D.-H.; Lee, D. A socio-technical analysis of software policy in Korea: Towards a central role for building ICT ecosystems. Telecommun. Pol. 2015, 39, 944–956. [Google Scholar] [CrossRef]
  16. Schwaag-Serger, S.; Wise, E.; Arnold, E. National research and innovation councils as an instrument of innovation governance. Vinnova Anal. 2015. [Google Scholar]
  17. Proksch, D.; Busch-Casler, J.; Haberstroh, M.M.; Pinkwart, A. National health innovation systems: Clustering the OECD countries by innovative output in healthcare using a multi indicator approach. Res. Pol. 2019, 48, 169–179. [Google Scholar] [CrossRef]
  18. Freeman, B.; Marginson, S.; Tytler, R. The Age of STEM: Educational Policy and Practice across the World in Science, Technology, Engineering and Mathematics; Routledge: Abington, UK, 2014. [Google Scholar]
  19. Yuko, H. Japanese technology policy: history and a new perspective. Res. Inst. Econ. Trade Ind. 2001, 1–27. [Google Scholar]
  20. Lee, P.-C.; Su, H.-N. Evolution of science, technology and innovation policy in Asia: Case of China, South Korea, Japan and Taiwan. In Proceedings of the 2015 Portland International Conference on Management of Engineering and Technology (PICMET), Portland, OR, USA, 2–6 August 2015; pp. 184–191. [Google Scholar]
  21. Stenberg, L.; Nagano, H. Priority-setting in Japanese research and innovation policy. Vinnova Anal. VA 2009, 23, 1–118. [Google Scholar]
  22. Lane, N. US science and technology: An uncoordinated system that seems to work. Technol. Soc. 2008, 30, 248–263. [Google Scholar] [CrossRef]
  23. Connell, D. Using government procurement to help grow new science and technology companies: Lessons from the US small business innovation research (SBIR) programme. Innovation 2009, 11, 127–134. [Google Scholar] [CrossRef]
  24. Popper, S.W.; Wagner, C.S. New Foundations for Growth: The US Innovation System Today and Tomorrow. An Executive Summary; RAND Corp: Santa Monica, CA, USA, 2001. [Google Scholar]
  25. Ratchford, J.T.; Blanpied, W.A. Paths to the future for science and technology in China, India and the United States. Technol. Soc. 2008, 30, 211–233. [Google Scholar] [CrossRef]
  26. Asheim, B.T.; Moodysson, J.; Tödtling, F. Constructing regional advantage: Towards state-of-the-art regional innovation system policies in Europe? Eur. Plan. Stud. 2011, 19, 1133–1139. [Google Scholar] [CrossRef]
  27. Woiceshyn, J.; Eriksson, P. How innovation systems in Finland and Alberta work: Lessons for policy and practice. Innovation 2014, 16, 19–31. [Google Scholar] [CrossRef]
  28. Pelkonen, A. The problem of integrated innovation policy: Analyzing the governing role of the Science and Technology Policy Council of Finland. Sci. Public Policy 2006, 33, 669–680. [Google Scholar] [CrossRef]
Figure 1. Overall research framework.
Figure 1. Overall research framework.
Joitmc 05 00082 g001
Figure 2. Organizational Structure of the NSTC.
Figure 2. Organizational Structure of the NSTC.
Joitmc 05 00082 g002
Figure 3. Overall organizations related to science and technology (S&T) policy in Japan.
Figure 3. Overall organizations related to science and technology (S&T) policy in Japan.
Joitmc 05 00082 g003
Figure 4. Organization of the NSTC in the U.S.
Figure 4. Organization of the NSTC in the U.S.
Joitmc 05 00082 g004
Figure 5. Organization of the Research and Innovation Council (RIC) in Finland.
Figure 5. Organization of the Research and Innovation Council (RIC) in Finland.
Joitmc 05 00082 g005
Figure 6. S&T policy deliberation system in Finland.
Figure 6. S&T policy deliberation system in Finland.
Joitmc 05 00082 g006
Table 1. History of National Science and Technology Council (NSTC).
Table 1. History of National Science and Technology Council (NSTC).
InformationFourth Generation (Former: 2011–2013)Fifth Generation (Current: 2013~)
Official nameNational Science and Technology CommitteeNational Science and Technology Council
CharacteristicsPermanentNonpermanent
Composition of a commissionSteering committee, special committee, expert committee, consultation committee, intelligence committee, and indirect cost committeeSteering committee, special committee, expert committee, and consultation committee
ChairmanPresidentCo-chairman (Prime minister and private chairman)
Vice chairman--
Assistant administratorSecretary General of National Science and Technology CommitteeMinister of Ministry of Science, ICT, and Future Planning
Competent MinistryNational Science and Technology CommitteeMinistry of Science, ICT, and Future Planning
Legal basisScience and Technology Basic Law (Amendment. 27 December 2010)Science and Technology Basic Law (Amendment. 23 March 2013)
Table 2. Major functions of the NSTC.
Table 2. Major functions of the NSTC.
Major FunctionsExplanation
Adjust major science and technology polices- Establishment and adjustment of major policies and planning of science and technology promotions
- Establishment of basic and general plans for local science and technology promotion
- Establishment of medium- and long-term plan of national R&D
- Investigation, analysis, and evaluation of national R&D
Adjust human resource policies regarding innovation of science and technology and industrialization- Promotion of government-funded research institute in science and technology
- Establishment and adjustment for national growth engine policies
- Adjustment for policies related to the innovation of science and technology in the industries of next-generation growth engines, culture tourist industry, material and component industry, and process innovations area
- Adjustment for policies related to scientific and technical personnel
- Policy support for national standards and intellectual property
Adjust regional technological innovation policies- Building a support system for regional technological innovation policies
Deliberate operational R&D budget- Establishment of budget expansion for science and technology and recommendation of R&D investment
- Deliberations on the operational national R&D budget
- Fund support for technology innovation
Table 3. Five expert panels of Council for Science and Technology Policy (CSTP).
Table 3. Five expert panels of Council for Science and Technology Policy (CSTP).
Expert PanelObjectiveMain Roles
Basic PolicyTo investigate and measure the basic S&T policy - Constructing the S&T Basic Plan
- Promoting the 3rd S&T Basic Plan
Promotion Strategy for Prioritized AreasTo investigate the promotion strategy for four areas (life science, ICT, environmental science, nanotechnology).- Determining the direction of four prioritized area
- Determining priorities in the allocation of resources for four prioritized area
EvaluationTo obtain an effective and efficient allocation of resources for S&T policy- Conducting overall evaluation of government-funded R&D
- Evaluating nationally important R&D
BioethicsTo respond to the development of the life science- Creating guidelines for the utilization of human steam cells, embryo, and other life components
Management of Intellectual PropertiesTo increase international competitiveness through intellectual property- Constructing strategies for the protection and utilization of intellectual property
Table 4. Subcommittees of the NSTC.
Table 4. Subcommittees of the NSTC.
CommitteeObjectiveMain Roles
Environment, Natural Resources, and SustainabilityTo enhance the overall productivity and application of environment and natural resources policy- Creating integrative R&D programs related to natural resources and the environment
- Supporting S&T for sustainability to promote sustainable systems
- Developing metrics to measure sustainability
Homeland and National SecurityTo improve the overall effectiveness of policy that provides a mechanism to promote S&T related national security science- Addressing reviewing technical issues that affect national security
- Identifying and recommending priorities in national security R&D
ScienceTo increase the overall effectiveness related to developing new knowledge in the sciences and mathematics- Coordinating all science policy-making processes
- Facilitating NSTC clearance of documents generated by interagency groups
Science, Technology, Engineering, and Mathematics Education To coordinate Federal programs and activities in support of STEM education- Coordinating with budget, STEM education activities, and programs based on Federal agencies
- Developing participating agencies 5-year STEM education
and strategic plan
TechnologyTo improve the overall effectiveness and productivity of all Federal policy in technology- Facilitating technology planning and communication between stakeholders
- Supporting technology programs and initiatives that enhance national and local competitiveness
Table 5. Comparison of S&T councils in Japan, the U.S., and Finland.
Table 5. Comparison of S&T councils in Japan, the U.S., and Finland.
Nation CriteriaJapanU.S.Finland
Main deliberation councilCSTP (Council of Science and Technology Policy)NSTC (National Science and Technology Council)RIC (The Research and Innovation Council)
Main sub-organizationExecutive Office: about 100 members
Five expert committees
Five expert committees
Several sub-work-groups and projects
10 nominated experts
Two main committees and
Main related organizationsMinistry of Finance: Establishing the budget
Ministry of Science: support partial instruction for education sectors
OSTP: Operating and supporting the council
OMB: Establishing the budget
Committee of legislature: Evaluating the general S&T policy in program level
AF: improving the research environment
Business Finland: creating opportunities for Finland through innovation and international expansion
Characteristics of systemConstructing independent and autonomous system
High control power through connection with other organizations
Emphasized the decentralization and diversification
Deciding policy through the collaboration of three power
Emphasized the participation of various stakeholders
Seeking to coordinate macroscopic S&T policy
Budget planning and coordinationProviding opinion about financial priority of R&D projects
Determining overall scale of expenditure and other resources
Evaluating the R&D project only
Not determining budget and its allocation
Evaluating the R&D performance
Developing money for public research innovation, and allocating it
Settle legal issuesNone (Only suggesting basic reference)None (Only suggesting basic reference)Settling the overall legal problems related R&D and policy
Table 6. Basic directions for enhancement of function.
Table 6. Basic directions for enhancement of function.
AreaExplanation
Enhancement for total control function- Innovation activities for national science and technology should be promoted by performing a conductor role that manages the national innovation system.
The NSTC must reallocate the role of departments and set a control system clearly.
The OSTP investigates and coordinates government departments by assigning experts from each department.
- Legal mechanism for control should be constructed to enhance the authority of the NSTC.
The government can legislate on essential particulars for operating the NSTC.
The RIC legally constructs the cooperation systems with target organizations to enhance the organization.
Enhancement for establishment of strategy function- Four primary strategies are required to increase the efficiency of R&D management
(1) Establishing advanced strategy for R&D diffusion system
(2) Determining core/integrated agenda for R&D management
(3) Enhancing the role and authority of private committee member
(4) Developing integrated R&D strategy for overall fields
RIC and related organizations develop integrated R&D strategy for other fields as well as S&T.
Enhancement for mediation function- The NSTC should enhance the connection with upper S&T departments such as the MISF (Ministry of IT, Science and Future Planning) in Korea
The OSTP has a stronger authority by supporting the MEXT
- Government should provide an institutional strategy for increasing the effectiveness of S&T planning.
The NSTC should be in charge of overall innovation from R&D to identify the needs of each field.
The NSTC in the U.S. is concerned with overall innovation process.
- The NSTC should be more concerned with establishing the budget through the result of evaluation and deliberation.
Before creating a budget, the NSTC should complete the mediation for departments based on Basic Plan, policy analysis, and evaluation
Enhancement for planning of open R&D policy- The NSTC should construct the global collaboration system with S&T organization of other developed countries.
The NSTC in the U.S. constructs global collaboration systems with other developed countries
Enhancement for establishment of reasonable and fair decision system based on expert opinion- Scientific experts and institutions should participate in deliberating S&T policy to secure both fairness and reasonability.
Additional private expert panel—permanent panel (committee) and regular panel—can be considered.
- Related S&T organizations should be enhanced autonomy in partial policy enforcement.
Japan government designates independent administrative institutions that can autonomously enforce the S&T policy in their programs.
Table 7. Basic directions for operation of organization.
Table 7. Basic directions for operation of organization.
AreaExplanation
Establishment of other supported organization- Associated organizations, such as Office of Science and Technology Policy, should enhance to support S&T council.
- The NSTC should continuously interact with associated organizations to construct an integrated system to investigate, analyze, and evaluate S&T policy.
Executive offices in Japan frequently investigates the S&T trends and provide the information for the evaluation of S&T policy
- The NSTC should construct definitive criteria for the evaluation of S&T policy through the information from other fields.
Many developed countries construct a co-evaluation system to capture indirect ripple effects on other fields, such as education and labor
Enhancement for the capability of decision-making process- The NSTC should boost the participation of private experts in decision-making process.
The OSTP gathers opinions from private academic and economic societies such as Japan Science Council in S&T policy decision.
- The NSTC should set the long-term direction of S&T policy and it should match the detailed decisions with the long-term direction.
The NSTC in the U.S. publishes a report that contains guidelines for the decisions of S&T policies and programs.
Enhancement of information sharing and feedback- The NSTC should appoint civilian members who can understand the various fields.
Council members induce information sharing activity to enhance the knowledge of other council members.
The OSTP and RIC listen to the opinion/feedback of executives from various government departments and outside specialists.
Table 8. Basic directions for enhancement of human resource management.
Table 8. Basic directions for enhancement of human resource management.
AreaExplanation
Enhancement for the capability of S&T policy- The NSTC should train council members in three capabilities:
(1) Scientific and technical administration: communication ability, personal management experience, the ability of understanding, and applying the visions of policies.
(2) Scientific and technical policy: overall understanding about the influence of S&T policy on society, politics and economy, basic knowledge of S&T forecasting, and evaluation
(3) Scientific and technical specialty: overall scientific and technical knowledge, understanding about future S&T needs
Subcommittees in Japan, the U.S., and Finland frequently provide scientific and technical knowledge
Executive offices in Japan frequently investigates S&T trends and provide information for the evaluation of S&T policies
Enhancement for the capability of globalization- The committee should appoint experts who can cope with global S&T change.
Experts reflect diversity and specialty derived from global S&T trends in S&T policy.
The NSTC in the U.S. and the RIC cultivate relationships with overseas professionals.
Table 9. Basic directions for the improvement of relationships.
Table 9. Basic directions for the improvement of relationships.
AreaExplanation
Enhancement in the relationships with government departments- The NSTC should develop joint programs and methods that several departments participate in.
Interdepartmental committees and working groups are an example of joining several departments.
The NSTC in the U.S. operates many working groups that share the mission for S&T development.
Enhancement in the relationship with the public- The NSTC should promote cooperation with the public
Improved online webpages can be used to promote suggestions, discussions, and complaints from the public
The Cabinet office in Japan operates public groups to gather the public’s opinion.
Enhancement in the relationships with the academic field- The NSTC should enhance communication with several research societies, laboratories, and universities.
Interactions between research societies and laboratories may enhance the quality of policy advice and S&T policy plans.
The Academy of Finland developed an advanced research system and environment based on the opinions of laboratories and universities
Enhancement in the relationships with the industrial field- The NSTC should set policy direction using the latest information from the industrial world.
Interactions with various industrial fields can induce not only realistic outputs but also innovative outputs.
Council operates regular economic/industrial panels selected by public hearing.

Share and Cite

MDPI and ACS Style

Kang, D.; Jang, W.; Kim, Y.; Jeon, J. Comparing National Innovation System among the USA, Japan, and Finland to Improve Korean Deliberation Organization for National Science and Technology Policy. J. Open Innov. Technol. Mark. Complex. 2019, 5, 82. https://doi.org/10.3390/joitmc5040082

AMA Style

Kang D, Jang W, Kim Y, Jeon J. Comparing National Innovation System among the USA, Japan, and Finland to Improve Korean Deliberation Organization for National Science and Technology Policy. Journal of Open Innovation: Technology, Market, and Complexity. 2019; 5(4):82. https://doi.org/10.3390/joitmc5040082

Chicago/Turabian Style

Kang, Daekook, Wooseok Jang, Yoonjo Kim, and Jeonghwan Jeon. 2019. "Comparing National Innovation System among the USA, Japan, and Finland to Improve Korean Deliberation Organization for National Science and Technology Policy" Journal of Open Innovation: Technology, Market, and Complexity 5, no. 4: 82. https://doi.org/10.3390/joitmc5040082

APA Style

Kang, D., Jang, W., Kim, Y., & Jeon, J. (2019). Comparing National Innovation System among the USA, Japan, and Finland to Improve Korean Deliberation Organization for National Science and Technology Policy. Journal of Open Innovation: Technology, Market, and Complexity, 5(4), 82. https://doi.org/10.3390/joitmc5040082

Article Metrics

Back to TopTop