1. Introduction
Social science is a branch of science that deals with various phenomena of human society with scientific methods (
https://en.wikipedia.org/wiki/Social_science). Furthermore, social scientific research plays a crucial role in the long-term development of a country, and it is a strategic component of the national competitiveness [
1]. Currently, the scientific research of HEIs, as an important engine of social changes, has played a leading role in economic development. Different from natural science which has enterprise markets as its research hotbeds, social scientific research is mostly launched by HEIs where a large number of researchers are gathered producing abundant innovative research achievements. Therefore, some developing countries, including China, have offered considerable support to the social scientific research in HEIs, trying to improve the creativity and sustainability of social science development [
2].
In China, the investment in social science by the governments at all levels is incredibly abundant, taking social science R&D funds of HEIs in Jiangsu Province as an example. In 2014, the funds from governments was 441.25 million yuan, accounting for approximately 60.1% of all sources of funds, which can show that the development of the social science has captured enough attention from the government departments. After investing such a huge amount of money, it is natural and reasonable for the stakeholders, such as policy-makers, administrators of HEIs and scholars to shift their attention to the operation and evaluation of the scientific research supported by those funds. They try to evaluate the performance of the scientific research among HEIs, in order to make resource allocation more efficient, and then to achieve an overall development of scientific research, exerting positive influence on regional innovation, and economic and social development.
In recent years, research evaluation has gradually become a crucial tool for the government to promote the development of the scientific research of HEIs. At the end of 2013, the Ministry of Education issued the “Opinions on Deepening the Reform of Scientific and Technological Evaluation of Colleges and Universities”, saying, for HEIs, that scientific and technological evaluation is not only their main means of scientific and technological management, but also assists with more efficient allocation of resources and macro-management. Hence, deepening the reform of scientific and technological evaluation of HEIs is of great significance in improving HEIs’ innovation ability, promoting the comprehensive reform of higher education, and realizing innovation-driven development. Moreover, the Ministry of Education released more guidance on the evaluation of scientific research of teachers in HEIs at the end of 2016, trying to establish an all-in-one linkage mechanism, encompassing evaluation of teachers, HEIs and disciplines. This guidance is of great significance for evaluation of both basic and applied research, especially the emerging interdisciplinary comprehensive research.
However, scientific research is an intensive process of transforming tacit knowledge into explicit knowledge, and it is difficult to evaluate, let alone do an inter-temporal evaluation. Meanwhile, HEIs as a non-profit organization still retain their certain key characteristics—“the lack of profit motive, goal diversity and uncertainty, diffuse decision-making, and poorly understood production technology,” all of which distinguish them from other types of social organizations [
3]. Although those characteristics have been weakened over time with gradual diversifying roles, for the HEIs’ scientific research, it is hard to give an undisputed production function to judge its performance. With the characteristics of no need to set a production function in advance, a non-parameter method called data envelopment analysis (DEA) [
4] has become a standard tool for evaluating efficiency [
5,
6,
7,
8]. Mostly, DEA is used to evaluate the performance of multiple decision making units (DMUs), which, as an institution, transition a set of input resources into a set of output resources, and then identify the best production frontier among DMUs as the performance benchmarking.
The following DEA model is an input-oriented CCR model [
4], which can be used to measure the relative efficiency of a set of
n DMUs
, with the same production process. The model tries to search the maximum possible reduction of inputs (
) while keeping the outputs (
) at a constant level. The model is:
where
and
are virtual variables and
is a small non-Archimedean number that is used to avoid ignoring any unpredictable factor in the calculating process [
9].
The existing papers focus primarily on natural science rather than social science, and, even in some articles, scientific research is only specific to the research of natural science. This unbalance, which is mainly caused by the difficulty in measuring the output of social science, would prove to be an insurmountable obstacle to the overall development of social scientific research and the industrialization of natural science achievements. Therefore, improving the evaluation of the social scientific research can not only enhance the staying power of social scientific research but also promote the sustainable development of the natural science, even the development of the integrated science field.
Jiangsu province, with a higher education and economic development, enjoys a higher level of economic and social development and hosts the largest number of HEIs and college students in China (
http://www.chinadegrees.cn/xwyyjsjyxx/xwsytjxx/;
http://www.moe.edu.cn/srcsite/A03/moe_634/201606/t20160603_248263.html;
http://www.ec.js.edu.cn/col/col4065/index.html). Obviously, its fast economic and social growth relies heavily on the support from HEIs. The latter’s scientific research achievements have played a crucial role in promoting the economic and social development [
10]. In recent years, governments at all levels have continuously raised their financial support to HEIs, and the project investment continues upward year to year at a relatively stable rate (see
Table 1). In turn, to support the economic and social development, HEIs have produced a huge amount of intellectual achievements, like publishing works, papers, research and consulting reports and other various types of achievements. Although the intellectual achievements also demonstrate a growth trend, their rate fluctuates over this period (see
Table 2).
It is the percentage differences between the investments obtained by HEIs in their research projects at all levels and the intellectual outputs produced by them that drive us to track the performance of scientific research of different HEIs, but how can we accurately evaluate the performance of social scientific research in order to make the resource allocation more efficient? We should be equipped with a reasonable approach and a set of appropriate indicators of performance evaluation. In this paper, we choose HEIs related to the scientific research of projects at all levels as DMUs. Focusing on 18 comprehensive HEIs, this paper, based on the data from Humanities and Social Science Research Management System of the National Ordinary Institutions of Higher Learning (Jiangsu Province), measures and analyzes the efficiency differences of multiple HEIs at different stages and periods, with the help of the multi-period two-stage shared-input DEA model and MPI. Furthermore, we make a ranking of these HEIs in terms of their performance in order to set a benchmarking for the others, which outlines an avenue to a more balanced and sustainable development of the social scientific research of HEIs in Jiangsu province.
The remainder of this paper is structured as follows.
Section 2 defines the general conceptual framework of social scientific research and lists the key elements of the HEIs’ research system. We propose a multi-period two-stage shared-input DEA model and analyze the input and output data in
Section 3. In
Section 4, data are brought into the model for calculation, and the empirical results are demonstrated and analyzed by virtue of online surveys. Besides depending on MPI, we also evaluate the efficiency variance of each DMU at different periods and stages. Finally, a conclusion is drawn in the last section.
2. Key Indicators and Conceptual Framework for Social Science Research
Teaching and scientific research are often regarded as two major tasks of HEIs [
8], but it is difficult to split them apart and measure their performance separately [
11]. Thus, many scholars consider them as a whole, and get the overall performance of the two. However, such a method cannot shed light on how they perform respectively. Hence, this paper attempts to evaluate the performance of the social scientific research by studying the implementation of the HEIs’ research projects, which have little relevance to the teaching activities. Furthermore, the social scientific research of HEIs is a complex process involving multiple factors. To a certain extent, the accuracy of performance evaluation depends upon the choice of key indicators. In selecting the input and output indicators, it is necessary to take into account the linear independence within the input and output indicator groups, and linear correlation between inputs and outputs.
As DEA models enjoy a growing share of the world-class journals, such as OR (Operational Research), EJOR (European Journal of Operational Research), OMEGA, etc., their applications in research and development of HEIs becomes extensive. In the early 1990s, scholars began to measure the efficiency of universities [
12]. Some articles pointed out that R&DE (R&D Expenditure) and R&DP (R&D Personnel) were the key input indicators of scientific activities, while patents and papers were the key output ones [
13]. With the help of DEA, Avkiran [
14] measured the technology and scale efficiencies of Australian universities. Chapple and Lockett [
15] calculated the relative performance of technology transfer units. Beasley [
8] also applied DEA into the measurement of British universities’ teaching and scientific research efficiencies. Additionally, a multi-stage model was proposed by some scholars. Liang and Li [
16] classified the number of SCI papers and patents as the first-stage output, and the research funds obtained as the second-stage output. As for doctoral students, Cook and Green [
17] treated them as the intermediate variable because they were the output of the education stage as well as the input of the scientific research stage. Therefore, based on Cooper’s general provisions of indicators and existing research of other scholars [
18], including Beasley [
8] and Zhu [
19], we argue that the personnel, funds, and equipment related to research are all classified as the key input indicators in the process of HEIs’ scientific research [
20]. Due to the large number of people involved in scientific research, we only choose researchers with a graduate degree and above as inputs, since they are more engaged in scientific research. Meanwhile, the equipment expenditure is mostly regarded as shared resources within HEIs, and it is difficult to specify the annual use weight of each HEI or discipline. In view of the fact that the equipment plays a relatively minor role in social scientific research, we eliminate this variable. With the development of HEIs, researchers are encouraged to complete the existing research projects and apply for new ones, during which a lot of innovative achievements are produced. Hence, the application and completion of research projects are defined as the variables connecting the development stage and innovation stage, i.e., intermediate variables. Though the existing papers jointly regard publications and patents as the final output [
21,
22], this study aims at the social science that hardly produces patents. Instead, the research and consulting reports prove to be very common output, which can be verified by the data structure of the existing management system (Most values of the patents of those 18 HEIs or 22 disciplines are zero.).
The traditional conceptual framework of scientific research focuses primarily on the whole process of research, and this paper tries to open the “black box” of social scientific research to shed light on the respective performance at all stages [
23], for only in this way can the stage of low efficiency be accurately located. Based on the understanding of HEIs’ scientific research and the availability of data, we propose a two-stage conceptual framework (See
Figure 1), dividing the whole research process into two related stages with the dual perspective of development and innovation—the research development stage and the research innovation stage—in order to draw more attention to the internal structure of the stage.
Figure 1 vividly describes the typical process of scientific research and shows its two stages.
Based on previous analysis, the data structure of Humanities and Social Science Research Management System of the National Ordinary Institutions of Higher Learning (Jiangsu Province) and the characteristics of social scientific research, this paper specifies indicators as follows. The researchers involved with a graduate degree and above and research expenditures are defined as inputs and the approved funds of the new applied projects and completed projects are output indicators at the research development stage. At the stage of research innovation, since they still play an important subjective dynamic role, the researchers are considered to be a shared input. All the output of the previous stage turns into the input at this stage, including approved funds of new applied projects and completed projects. The research concerning social science primarily consists of two parts: basic research and applied research, and there are few studies on experiment and development, R&D application, and scientific and technological service (Most values of the fields are zero.). This status quo has a strong guiding and restraining effect on the choice of the final output indicators. Therefore, in terms of innovativeness and support for the development of economy and society, this paper, from various kinds of scientific research achievements, including published works, collation of ancient books, translation works, published translations, electronic publications, published papers, research and consulting reports, etc., chooses published papers, published works and research and consulting reports as output at the innovation stage. Additionally, there is a certain elimination rate of research and consulting reports in reality, and the output in this paper only refers to those finally accepted. All variables and their definitions are shown in
Table 3.
These indicators—input, intermediate output and final output—perform different functions. The input plays a crucial role at the stage of research development; the intermediate output serves as a bridge connecting the two stages; and the final output is produced on the basis of input and intermediate variables and needs to be recognized by the society. Meanwhile, in consideration of the subjective dynamic features of the two stages, this paper regards the researchers as a shared input, for they are indispensable at both stages.
Since the general two-stage DEA model cannot expose the efficiency difference of an HEI at different times, it is necessary to apply the time series analysis to the efficiency evaluation. We need to judge the efficiency variation of the same DMU at different periods and then comprehend a yearly change trend. By adding a dimension of time (from period 1 to period
q) to the framework, and assuming that the scientific research activity that occurs at each period is the same, we come up with a multi-period two-stage shared-input conceptual framework (
Figure 2).
3. The Multi-Period Two-Stage Shared-Input DEA Model
3.1. Data and Time Series
We argue that input variables, intermediate variables and output variables of various projects should well outline the important elements of scientific research activities. Hence, when choosing variables, we try to cover all the key traits related to the production process, based on the basic understanding of social scientific research, current knowledge and the availability of data. Besides the market projects contributing more to promoting the practical development of economy and society with little impact on the scientific research itself, we eliminate the data of these projects.
Based on the scientific research development level and the availability of data, this paper selects 18 representative HEIs (the full names of 18 HEIs are shown in the
Appendix A), with a research cycle of five years from 2010 to 2014. All the data in this paper are obtained from the Humanities and Social Science Research Management System of the National Ordinary Institutions of Higher Learning (Jiangsu Province).
Table 4 lists the total data of input, intermediate output and final output of the 18 HEIs for the five years, respectively.
Additionally, in consideration of the lag characteristic of scientific research results, some papers measure the lag order by using a distributed lag model or manual calculation [
24]. The solution process of the distributed lag model requires data from many years, and this is impossible to obtain because the information system just contains the data from 2010 to 2014. Meanwhile, manual calculation lacks a scientific foundation and reliability. Thus, this paper ignores the lag of the output and argues that social scientific research is a process of continuous overlapping, thus the conversion of its results has no substantive significance. To verify our opinion, we compared the output of lagged one-year with that of lagged two-year, and no great change was found.
3.2. Multi-Period Two-Stage Shared-Input DEA Model
After the preceding analysis of the process of social scientific research as a chain process with two connected stages, this paper then constructs a multi-period two-stage shared-input DEA model based on the multi-period two-stage DEA model proposed by Kao and Hwang [
25] with the consideration of the subjective dynamic characteristics at the two stages—treating the researchers as the shared input. The two-stage DEA model can evaluate both the performance of the whole system and that of respective stages within the system.
Figure 3 is an abstract research system of
Section 3, and it exhibits a multi-period two-stage shared-input research process.
Suppose that there are and each has inputs denoted by , intermediate products denoted by , and outputs denoted by , respectively.
(1) Performance measurement
The relational two-stage model for measuring the system (overall) and stage efficiencies of the kth DMU is to take the operations of the
q-period two-stage into consideration in addition to that of the system defined in the conventional DEA model [
25], and it can measure the system and stage efficiencies at the same time. The time span covers
q periods, the target is to measure the efficiency of a set of
n DMUs, and each DMU endures a two-stage production system. Let
,
, and
denote the inputs, intermediate products, and outputs of DMU
j in period
p, respectively. These two kinds of inputs are denoted as
and shared inputs
, respectively, where
. We set the portions of shared inputs used by the first development stage and second innovation stage as
and
.
,
, and
indicate the total of the inputs, intermediate products, and outputs of DMU j of all periods, respectively. Let
,
and
denote the overall system efficiency, the development efficiency (the stage Ι) and the innovation efficiency (the stage ΙΙ) of
. Let
,
and
denote the system efficiency, the development efficiency and the innovation efficiency of
in the period
p. Additionally, since DEA has a unit-invariant property [
26], the efficiencies measured from any types of data are the same.
The model sets the constraint of the aggregate output to be less than or equal to the aggregate input of each period and all periods:
At optimality, the overall efficiencies of
q periods for the system (
) and two stages (
and
) are:
Similarly, we set the same variables with the same multiplier no matter how it is used, which is the essence of the relational model [
25].
Therefore, the efficiencies of period
p (
) and its two stages (
and
) are:
(2) Performance changes
Pastor and Lovell [
27] proposed a global MPI, which uses the global technology of all
q periods to measure efficiency. All
q periods are used to construct the efficiency frontier for measuring efficiency, and this type of MPI possesses an attractive property of circularity [
28]. Many scholars believe that efficiencies calculated from different frontier facets are not comparable, so we adopt the common-weight global MPI [
25,
29,
30,
31] to do some further analysis.
We then calculate a common-weight global Malmquist productivity index (MPI) between two periods by dividing their efficiencies. The efficiency of period t,
, is calculated via the model (3), we can calculate the efficiency of period h,
, similarly, and the global MPI is
. By the same token, changes in performance for stage one (and two) between period
t and
h can also be measured by a common weight global MPI calculated from the ratio of
(or
) to
(or
):
4. Empirical Results and Analysis
In this section, with the help of MATLAB (MathWorks, Natick, MA, USA), the efficiencies of 18 HEIs are measured. Based on this, we calculate the efficiency differences among DMUs at different periods.
By using the five-year average data and the annual data of each HEI contained in
Table 4, we measure the 18 HEIs’ overall efficiencies and stage efficiencies (see
Table 5) through Model 1 and Model 2.
Table 5 presents the five-year average efficiencies of the 18 HEIs, and also displays a general understanding of the social science development of HEIs in Jiangsu Province, spotlighting the efficiency differences among HEIs. From 2010 to 2014, the average efficiency of 18 HEIs is 0.3044, with the stage 1 of 0.4467 and the stage 2 of 0.3938. Apparently, the efficiency of stage 1, the research development stage, is higher than that of the research innovation stage. Thus, we can infer that the 18 HEIs have done better in the application for new projects and completion of existing projects than producing research achievements.
In addition, it can be seen from
Table 5 that NTU has the highest overall efficiency (0.7465), with the efficiencies of stage 1 and stage 2 being 0.9314 and 0.7733, respectively. As to its good performance, after online surveys to relative HEIs about their social scientific research development (The author of this paper undertakes the writing of the Research and Development Report on the development of the Social Science of HEIs in Jiangsu Province commissioned by the Department of Social science of Jiangsu Province. During the writing process, we have made a thorough investigation and research on some HEIs’ development of social scientific research through email and telephone call, and sorted out the investigation results as the element of the report.), we find the answer. In recent years, based on its existing scientific research foundation, NTU has established a positive partnership with domestic first-class HEIs whose outstanding scientific research strength plays a supportive role in promoting NTU’s scientific research development and application for projects at all levels. Meanwhile, a collective group of disciplines was constructed, including Law, Management, Sociology, etc., in order to give full play to the advantages of its disciplines and realize a collaborated, stable and interdisciplinary development. Furthermore, NTU is active in the conversion transiting of the scientific research achievements in order to contribute to the power of socio-economic development. Furthermore, NTU also serves as the think tank in the regional development, with a number of research and consulting reports well accepted by the society.
Apart from NTU, NJUE, HHU and SCU also enjoy higher rankings and have good performance at each stage. Thus, it can be inferred that these HEIs’ efforts in promoting social scientific research development for the past five years were rewarded. NJUE, with all its majors related to social science, is an HEI characterized by social science. Its social scientific research has been supported by a lot of provincial and ministerial special funds. With some special disciplines, NJUE also enjoys exclusiveness when applying for some research projects according to those disciplines. All these advantages enable it to get a relatively high efficiency. Taking the leading role in natural scientific research, HHU has also progressively developed its social science from 2010 to 2014. Depending on its advantages of natural science, it has greatly promoted social science construction with that positive externality, making a bridge between natural science and social science. On the one hand, social science can ensure the industrialization of natural science achievements. On the other hand, the market responsiveness nurtures the social science development. In the social scientific research, HHU also advocates the interdisciplinary development, making the social scientific research undertaken by various disciplines collaboratively. As to SCU, it focuses primarily on national major projects. By elaborately tailoring research direction and integrating the research team, it has continuously made historic breakthroughs in the field of social scientific research, with research achievements widely accepted by the society. Furthermore, based on its own development characteristics, it has founded a number of scientific research and innovation organizations as think tanks to solve the key and difficult issues of regional economic development.
Compared with these better-performing HEIs, some other HEIs have efficiencies that rank high at a certain stage but have lower overall rankings, due to their inefficiency at the other stage. For example, both NJFU and YZU perform well at the research development stage while NUAA has a higher efficiency at the stage of research innovation. As to NJFU, based on its distinctive research areas, it has made great efforts to develop cutting-edge research projects, equipped with existing professional talents and social resources. Though it has made great progress in the field of social scientific research, there is still room for improving at the research innovation stage. As an engineering college, NUAA performs well at the stage of research innovation. Possibly, under the guidance of more mature methodology, its research achievements are easy to be recognized and accepted by the society. In short, the efficiencies of the first stage of certain HEIs that are better in social science are higher than that of those that have an advantage in natural science, and the results are opposite in the second stage.
In addition, from
Table 4, it can be seen that though it has the largest input and output, NJU gets a lower efficiency. The reason is that, because of its overwhelming advantages and prominent position in the field of social science, it was offered great financial support from the government departments for scientific research, with a large number of research personnel engaged, but, different from the input of funds and personnel, the scientific research achievements cannot present a magnitude growth in a short time. As a result, its efficiency turns out to be relatively lower.
Based on the data of efficiencies in
Table 6 via Model 4, we can draw an efficiency variance (
MPI) of each HEI for these five years (also see
Table 6), specifying which year it reaches the peak and which year it becomes lower. Take NTU as an example. Its overall efficiency is as high as 0.7465 (see
Table 6). From
Table 6, we can find that this high value depends on its relatively good performance in 2010 and 2013 when its efficiency reaches the peak of 1, with its efficiency in 2011 as 0.5556, in 2012 as 0.6596, and in 2014 as 0.6525. As to SEU, its overall efficiency is only 0.2358 (see
Table 5), which, as shown in
Table 6, primarily results from the lower efficiencies of 2011 (0.0935) and 2014 (0.1960) with rather low efficiencies at the research development stage.
According to its overall efficiency and stage efficiencies of each year, every HEI can spotlight its pros and cons to build its own development strategy. It can also establish cooperation with those of higher efficiency in continuously improving its management system of social scientific research. The administration departments of HEIs need to create opportunities to make “economies of scope” of core resources realize optimum spread. Meanwhile, when giving full play of its superiority disciplines, HEI ought to boost the coordinated development of multiple disciplines. Additionally, various performance evaluation systems should be established to carry out a real-time tracking of the social science development, and then some amendments should be applied immediately.
5. Conclusions
HEI is an important base for cultivating high-level innovative talents, and a main force to solve major scientific and technological issues. The social scientific research launched by HEIs is essential to enhance the scope, discover new possibilities, and catalyze change in social and economic development. With a limited number of research personnel and funds, HEIs need to improve their management system and integrate superior resources in order to support the rapid development of social scientific research, giving full play to their role as brain trusts.
After evaluating the performance of these 18 HEIs and online surveys, we determine their inefficiency causes and make a prescription. We argue that, in order to promote its scientific research development, HEIs should strive to establish a scientific and reasonable comprehensive evaluation system, regularly evaluating the development and innovation abilities of the scientific research, like a management and operation mechanism referring to the quality of scientific achievements, etc. The performance evaluation of social scientific research can figure out the stage of low efficiency, which can improve the scientific research and development ability of the entire HEI through resource allocation and a shared interdisciplinary management system. Meanwhile, an open and cooperative mechanism should be established among HEIs, aiming to achieve regional interaction between HEIs and promote knowledge flow and resource sharing. Peers’ advanced management experience can also be learned. Additionally, HEI should establish partnerships with applied research institutions to promote the industrialization of its research achievements, making practical demand nurture scientific research sustainably. In other words, measurement and assessment of sustainability efficiencies are evolving and present an opportunity for future collaboration between institutions and industrial organizations.
Furthermore, the HEI itself needs to strengthen the continuous research of its superiority disciplines and promote the provincial cross-university and interdisciplinary scientific research, constantly seeking and cultivating new growth points of social scientific research. As to natural science and social science, HEIs need to make a bridge between them and form a benign academic ecosystem to improve the overall efficiency of scientific research. Furthermore, various interdisciplinary sharing modes are also encouraged, which will boost the coordinated development among disciplines. Networks and communication tools among disciplines are strategic avenues to rapidly sharing knowledge and resources, in that the disciplines would finally get improved performance. Multi-HEIs and multi-discipline collaborations push the boundaries of sustainable development and inspire change in social research operations.
The relevant administration departments also need to take some measures. They should strive to establish a comprehensive disciplinary environment between natural science and social science. Furthermore, in order to improve the social scientific research abilities in a province or even in the country, the government departments ought to continuously create opportunities to encourage the overflow or sharing of advanced concepts, institutional arrangements, and innovative management practices. The implementation of such solutions will require the best understanding and consensus of HEIs’ development backgrounds in Jiangsu province, the capacity to trigger big improvements in HEIs, and the involvement of administration departments who understand these challenges and can manage efficiency promotion in a holistic manner.
In conclusion, with the joined forces of practical needs, researchers’ innovative abilities and the support of governments at all levels, HEIs’ social scientific research can build a batch of remarkable achievements that not only have important academic influence but also can make a great contribution to social and economic development. We hope that HEIs can actively strive for basic research projects on academic research and unknown areas, and generate a number of innovative academic results and research and consulting reports that can seek the best avenues to a sustainable future.