Evaluating Operation Performance in Higher Education: The Case of Vietnam Public Universities

Abstract

The system of public universities plays an important role in the socioeconomic development of each country. In Vietnam, public universities perform a leading role in the national higher education system’s operation and development. Therefore, public universities are assigned funds, assets, and facilities to implement goals and prioritize investment policies in the country’s education and training. However, to appropriately allocate funding, the state must reconsider the performance of the public education system. This paper presents a methodology to evaluate the operating performance of public higher education in Vietnam. The research design model includes cluster analysis and ANOVA, and Duncan post hoc tests have been used to provide an overview of public universities’ current state in Vietnam and to identify each of the strengths and weaknesses in cluster-specific groups. Based on this study’s results, educational administrators can develop a reasonable financial budget allocation plan for each university cluster.

1. Introduction

The public university system is a highly dynamic higher education center with unique traditions and community relationships. It is said that the academic training process of public universities creates good “products” because the input process has selected good “raw materials” and a modern “production line”, as well as modern machinery and equipment. Besides, the lecturers have good experience and a long history of teaching in different research fields concentrated in these universities [1].

Currently, the network of higher education institutions in Vietnam has initially met the requirements of training highly qualified human resources for the country’s socio-economic development and the people’s learning needs. According to statistics from the Ministry of Education and Training, as of December 2018, there were 236 universities/institutes nationwide (excluding those in the defense/security sector), of which 171 were public universities (accounting for 72.45% of the total number of universities). The above information shows that public universities play an important role in the national education system that improves people’s knowledge, trains human resources, and fosters talents for the country [2].

They recognize that education and training policies, and science and technology policies, are the two national policies that need to be given the highest priority to realize long-term sustainable development goals. The fields of education and training are given priority to invest in extensive resources from the state budget. Annual education budget expenditure in Vietnam is approximately 20%, equivalent to 5% of GDP. This is an extremely high level compared to many countries globally, including countries with much higher economic development levels than Vietnam [3].

In terms of financial resources to ensure public universities’ operation, there are still many limitations so far; this is due to the difficult balancing capacity of the state budget and policies on tuition collection. There are still many shortcomings, such as not ensuring the offset of operating costs [4]. Budget allocation for education remains complex. Defining and evaluating the budget criteria is difficult, and it needs to be considered [5]. Therefore, according to output quality, budgeting public higher education is a trend applied by many countries worldwide, replacing the regular budgeting mechanism according to input indicators or historical data. The main motivation for this new mechanism is to help improve public investment efficiency in higher education. This is especially important for Vietnam in the context of its rapidly expanding higher education, and the maintenance of a full-cost investment mechanism for public higher education is no longer sustainable [6].

To operate output-based budgeting mechanisms, countries worldwide often set up a government-level agency tasked with assessing higher education quality. Evaluating the quality of education at universities is an essential task. However, it takes a long time to get results for all schools and use these results to perfect the grant budget efficient mechanism.

This paper presents a methodology for evaluating the operating performance of public higher education in Vietnam. The assessment method provides insight into training results and can be used as an additional standard for evaluating one’s performance and identifying appropriate training factors. Furthermore, it specifies public universities’ views on the quality of training, and it ensures the practicality of the results and methods applied to the education sector. Based on this study’s results, educational administrators can develop a reasonable financial budget allocation plan for each university cluster.

The authors used data from public universities published in the three publicity reports of two years, 2018–2019 and 2019–2020, and the appropriate tools used for analysis purposes. In this regard, cluster analysis, ANOVA, and Duncan post hoc tests have been used to overview public universities’ current state in Vietnam, and to identify each of the strengths and weaknesses in cluster-specific groups. These tools can enable educational administrators to rationally and efficiently reorganize public university systems and define exact plans for their investment in public universities—sectors such as policy, finance, and infrastructure are our focus. The application of this approach can be modified and, subsequently, extended to other service industries.

The remainder of the paper is structured as follows. Section 2, the literature review, examines the overview and role of public higher education in Vietnam, as well as a general categorization of higher education and state budget investment in higher education. This section also clarifies the definition and meaning of the study method, which is cluster analysis. Section 3, the materials and methods, presents the research model used in the paper, as well as the data collection method and the criteria for selecting analytical factors. Section 4, this section demonstrates our analytical results, and discusses and analyzes these results. The next section, Section 5, offers conclusions by summarizing relevant research contributions. Finally, the section on limitations, contributions, and conclusions provides implications for future research.

2. Literature Review

2.1. Brief Information about Vietnamese Public Universities

2.1.1. The Role of Public Universities in the Higher Education System

Public universities are state-owned training institutions invested in by the state budget to build classrooms, libraries, working rooms, and other grants within each public university’s basic construction scope. All expenses for the operation process (from salaries, allowances, office supplies, purchase of fixed assets) are also mainly derived from state budget allocations. Therefore, the organizational structure, management and service apparatus, salaries, and bonuses of public tertiary institutions must comply with competent state agencies’ principles [7].

The public university system plays an important role, providing benefits outside the economic category. This benefit is not limited to the individual practitioner but is also spread to the entire society. Therefore, the role of the state in building the public higher education system is very important. With the state budget to set up schools, building facilities, and funding for regular activities, the public university system has affirmed the role and responsibility of creating high-quality human resources for the country’s industrialization and modernization. This responsibility is reflected in many aspects, from building regulatory mechanisms to sponsoring. Although the intervention may be different, the nature and form of this intervention always depend on each country’s educational philosophy. The system of public universities and colleges has been established in the provinces to ensure the right to participate in learning and raise state funding qualifications. Simultaneously, through the public education and training system, the state will be able to monitor the quality of training and adjust career structures in line with the development orientation of the national economy [1].

2.1.2. Current Status of Investment Policy in Education and Training

In recent years, the Vietnamese government has always paid great attention to education and training, and they are given priority to invest in large resources from the state budget. Annual education budget expenditure in Vietnam is approximately 20%, equivalent to 5% of GDP. Along with continuous economic and social growth, investment in education from the state budget is always higher than in the previous year. Over the five years from 2016–2020, the state budget for recurrent expenditure on education increased by over 32.2%. In 2016, the state budget allocated for spending on academic training and vocational training was VND 195.6 trillion. By 2020, the estimated expenditure figure is VND 258.7 trillion. Vietnam’s public expenditure on education/GDP is even high compared to many countries in the world (4% in 2019), or even compared to some countries with a higher level of economic development in the same region (Singapore 3.2% in 2010, Thailand 3.8%) [8].

Besides the successes achieved, the Vietnam investment policy in education and training still has limitations and shortcomings. Inappropriate investment structure for education and training is reflected in the tasks’ expenditure structure, levels of education, the spending content in each educational level, and the profession in each education level.

Table 1. State budget expenditure structure for education and training (%).

Kind of Expenditure	Year 2010	Year 2011	Year 2012	Year 2015	Year 2017	Year 2018
Total expenditure	100	100	100	100	100	100
Basic construction expenditure	18.4	18	17.7	18.1	22	23
Frequent expenditure	81.6	82	82.3	81.9	78	77

Source: Ministry of Finance.

shows that recurrent expenditure accounts for about 82% of the total state budget expenditure on education and training. Infrequent expenditure and human expenditure accounts for 80% of the total expenditure. The rest is spent on teaching activities and improving the quality of the curriculum. Spending on basic construction is still low compared to improving school facilities, purchasing teaching equipment, and laboratories [9]. In fact, financial plans of some higher education institutions (public and nonpublic) are not appropriate; in certain periods, several higher education institutions focus heavily on increasing income for employees without actually investing in strengthening facilities, teaching and learning equipment [10].

Financial resources for public higher education in Vietnam are formed from many sources, including the state budget (state budget) and legal revenues from higher education institutions. According to Decree No. 16/2015/ND-CP, the current financial sources of public universities and the mobilization of each source are (i) state budget allocations; (ii) income from public nonbusiness service activities; (iii) fee and charge collection sources following regulations on fees; (iv) other sources of income as prescribed by law; (v) financial sources from financial transactions of public universities by the law (loan capital, mobilized capital, bank deposit interest, etc.); (vi) aid and grants under the law; and (vii) student financial resources.

The state budget for higher education is part of state budget expenditures for public higher education institutions. Besides, a part of the state budget is spent on higher education to implement state policies and regimes for students studying at nonpublic higher education establishments. According to the World Bank, Vietnam allocates 5% of the country’s total GDP to education, however, only 0.33% is invested in higher education (accounting for 6.1% of the total government investment in education) [11]. Meanwhile, in other countries, this rate is much higher (see Figure 1) [12]. In the period 2013–2018, as a statistic, it is estimated that the state budget spent about VND 1,120,355 billion on education and training, reaching about VND 172,905 billion for higher education, including development, investment spending, recurrent expenditures on public universities, and part of the state budget recurrent expenditures to implement state policies and regimes for students studying in nonpublic institutions [13].

In the structure of expenditure on education and training, there is a disproportionate investment in education levels. The state budget for higher education is still limited, with 19% of the total budget for education; only half of the primary education budget. By education level, preschool and general education expenditures account for approximately 70% of total education expenditure. Spending on primary education accounted for nearly 33% of all educational levels’ total state budget expenditure [14]

The state budget spending on higher education has basically ensured 20% of the total state budget expenditure. However, due to many objective factors, such as (i) total state budget expenditure is limited; (ii) the size of general education is large (number of general education institutions (schools, classes), and the number of teachers or students) and the state should spend a large proportion of its budget on general education; (iii) higher education has conditions for higher financial autonomy, so the state budget spending on higher education is more limited and has not been diversified [3]. This leads to difficulties in implementing the financial autonomy mechanism. The state budget still has to cover large and frequent operating expenses for a large number of public universities [15].

2.2. Clustering Method

2.2.1. A Brief Understanding of the Clustering Method

Clustering is considered to be an important method with popular applications. It not only brings analysis benefits but, also, supports other algorithms. The data source that each company collects and exploits today is large, and clustering allows us to understand data quickly without going into analysis, thus helping us to identify patterns. Data samples of internal observation units are nearly the same, exploring data’s laws and potential natural relationships. By clustering in several specific contexts, analysis projects can serve as tools for implementing data understanding, data exploration, and data preparation in data mining [16].

In the simplest way of understanding, clustering is the analytical method through which a data set will be divided into many different clusters/groups. In each cluster/group, data points or observations will be the same or different (the observations in this group differ from observations in other groups). The clustering algorithm is also known as segmentation analysis, because it is applied in marketing, sales, and customer relationship management to identify customer segments to launch effective advertising and sales campaigns [17].

Clustering known as unsupervised classification is a method of unsupervised learning—a method of building analysis models—based on “unlabeled” data sets, which are unsorted data points used to understand and extract valuable information about the characteristics and nature of the internal observations. Unlike supervised learning, clustering does not try to classify, estimate or predict the target variable’s value. The basis of the clustering method that can be deployed, and helps us optimize clustering, is based on the attributes, the variables in the data set, and the characteristics and quality of the data [18].

2.2.2. Applications of Clustering in Some Areas

Similar to classification algorithms, as classification and regression trees, or logistic regression, clustering is widely used in different fields, from economics and medicine to social sciences.

The application of clustering in market segmentation, or customer segmentation, has been mentioned before the 1970s by the analyst, Martin Christopher [19], with cluster analysis and market segmentation. Doyle & Saunders [20] demonstrated it in relation to market segmentation and positioning in specialized industrial markets. The job will be a lot easier if there is clustering. Clustering allows us to group customers in the data set according to demographics, shopping behaviors, and knowledge, as well as experience about factors that attract customers. From there, we can refine the sales and marketing solutions suitable for each group of customers found from clustering. For retail, e-commerce, finance, banking, and communication technology organizations that deploy multichannel, multiplatform sales, technology exploitation and business support techniques are easy to access for different customer data sources and for operating processes, markets, and industry competitor data. As such, the application of data mining, or data analytics and clustering, will bring many opportunities to bring business value. Companies can use clustering for group projects and group products. Besides, they know how each product group can impact company profitability, thereby optimizing its portfolio and managing to sign them better and make the right decisions.

In medicine and health, clustering has been used in psychology to support health improvements and maintenance, and to enhance the healthcare system [21]. Specifically, in health care systems, clustering is used to identify groups of the population that need care services, or those who will benefit from specific health services within the social community [19]. Especially during the Covid-19 pandemic, clustering was also used to prevent epidemics. As in Iran, clustering incorporates geographic information systems (GIS) based on disease situations in different regions, which helps to identify disease-spreading trends and determines the possibility of the virus spreading [22].

In the field of computer vision [23], specifically image recognition, clustering can be used to explore clusters or subclasses [24]. In information technology, namely website lookups, clustering is used to organize search results into different groups. In addition to web searches [25], clustering also clusters documents on the web into different topics commonly used in publishing information on the web, also known as web mining.

In education, GI Mukhamedov et al. [26] discussed how globalization requires education clustering. Besides, they gave extensive comments on the educational cluster’s goals, purposes, principles, and operating directions. Moreover, they describe the organizational significance, practice, and rationale of implementing the pedagogical education cluster based on the scientific research of Western scientists on the analyzed educational cluster. By analyzing neural networks, vector assist machines, decision trees and cluster analysis, Monica Ciolacu et al. estimated student performance in testing to shape potential talent in the next generation for skills in Industry 4.0 [27]. With the same viewpoint, Toshtemirova [28] noted that quality of education is closely linked with educational goals and strategies. To solve these problems, it is advisable to apply a cluster approach to increase educational efficiency. In cluster education, it is first and foremost important to consider the comprehensive reciprocal relationship between elements integrated into a whole. On student performance assessment, Farshid Marbouti, Jale Ulas, and Ching-Ho Wang [29] conducted a cluster analysis method to understand student groups based on academic performance and demographic information. Lee, Recker, Bowers, & Yuan [30] applied data mining and a pattern visualization methodology. Usage patterns clustered using hierarchical cluster analysis presents a form of visual data analytics to help examine and understand patterns of student activity [31]. Perrotta and Williamson draw on material semiotics to examine cluster analysis as a “performativity device” that, to a significant extent, creates the educational entities it claims to objectively represent through the emerging body of knowledge of Learning Analytics (LA). Myers III & Fouts [32] presented a study that provides a test of a theory which suggests that students’ perceptions of their classroom environment affect their attitudes toward science. In this case, a cluster analysis of high school science classroom environments and attitudes toward science were shown. There are also many educational studies using clustering methods.

2.2.3. Important Types of Clustering Analysis

The mechanism required for clustering to work is determining the similarities and differences between observed objects in the data set. In data mining or data analytics, the coefficients and measures used to calculate similarity, or difference, are very diverse; for example, the Jaccard coefficient, the Sorensen–Dice coefficient, or the simple matching coefficient [33]. However, in the clustering method, it is mainly used for distance metrics, such as the Euclidean distance, the Manhattan distance, or the Minkowski distance, of which the Euclidean distance is most commonly used [28]. Similarity coefficients are used to quantitatively describe two data points’ similarity or the similarity of two clusters: the larger the coefficient, the more similar the two data points. The measure and index of distance used to quantify dissimilarity are the opposite: the larger the distance, the less two data points or two clusters are not the same [34].

In terms of the absolute, clustering has two forms: hard clustering and fuzzy clustering. Hard clustering is understood as an observable object, a data point, or an object located in only 1 cluster to consider the difference between clusters at the maximum [35]. When an object is in a cluster, it will automatically be different from other objects in the other cluster. Fuzzy clustering, also known as soft clustering, is the opposite. An observable object, a data point, or an object can reside in 1 or more clusters. Experts often consider fuzzy clustering to be a relaxed form of clustering; results from the clustering process may not be clear, whereas they are differentiated in hard clustering [36].

If based on cluster structure, clustering has two general forms: hierarchical clustering and nonhierarchical clustering. In this research paper, the authors focus on hierarchical clustering, called hierarchy, which is partly due to the name and partly due to the visualization of clustering results. Hierarchical clustering is often illustrated with a dendrogram diagram. It is used effectively when an analyst wants to sort clusters by hierarchy [37].

Hierarchical clustering has two main forms, agglomerative and divisive. With an agglomerative beginning, each observation is a small cluster of its own. Then, in the next steps, the two most recent clusters are aggregated into a new associative cluster. In this way, the number of clusters in the data set is reduced by one at each step. In the end, all clusters are combined into a single large cluster. Divisive clustering starts with all the observations in a large cluster, with the most different observations being recursive. Then, they move into a separate cluster, until each observation represents its cluster [38]. Agglomerative clustering is more commonly used and is integrated into many pieces of analysis software.

When implementing agglomerative clustering, we need to consider group observations into clusters, and cluster each cluster into large clusters to ensure that we follow the distance calculation method with the principle of close distance showing similarity [39]. There are many ways to collect or linkage, such as single-linkage (clustering by the shortest distance), complete-linkage (clustering by the furthest distance), or average-linkage (clustering by average distance) [40]. These clustering methods are according to the graph method. There are also geometrical clustering methods, such as Ward’s, Centroid, and Median (Geometric methods).

The variance method tries to create clusters to minimize variance in clusters. Among the accumulation cluster analysis methods, the Ward procedure has been shown to have better results than the other methods, and it is a commonly used method of variance. In statistics, Ward’s method is a criterion applied in hierarchical cluster analysis [41]. According to Ward’s procedure, we will calculate the average of all variables for each cluster. Then, we calculate the Squared Euclidean distance between the cluster’s elements, with the cluster mean, and sum all of these squared distances. At each accumulation phase, two clusters with the smallest increase in the sum of the cluster’s squared distances are combined. More specifically, in this method, the distance or similarity between the two groups is considered the smallest distance between two points [42].

This study is an application of cluster analysis towards a classification of public universities. The classification of these universities is based on factors related to quality of training and research activities. Quality of training, assurance factors, and universities’ training outputs demonstrating performance are assessed and reported annually, following the Ministry of Education and Training of Vietnam regulations. While cluster analysis is a useful tool for grouping universities, hierarchical clustering is used in classifying universities. After clustering universities, the main purpose is to use the college clusters in benchmarking factors. This study aims to provide insight into universities’ heterogeneity and then compare differences in training quality.

Since clustering is used to group similar objects, we require several measures to assess how similar or different objects are. Measuring the similarity of distances between pairs of subjects is the most popular way. Objects with a smaller distance between them are more similar than objects with a greater distance. There are many approaches to calculate the distance between two objects. The most commonly used measure of similarity is the Euclidean distance or its square. The Euclidean distance is the square root of the sum of the squares of the difference in values for each variable. Other distance measures are also available. The block, or Manhattan, distance between the two features is the sum of the absolute difference in value for each variable. The Chebychev distance between two objects is the maximum absolute value difference for any variable. Using different distance measures can lead to different clustering results [41]. Hence, after applying other measures and comparing the results, we chose a measure of distance or similarity that provides the most obvious results. For our study, we will use Euclidean distances.

In this study, the selected cluster analysis method is the cumulative cluster analysis method, based on variance, as the “Ward procedure” in the hierarchical clustering procedure. Combined with the Euclidean distance, the clustering image was clearly shown.

3. Materials and Methods

3.1. Data Collection

This paper selected 89 public universities (see

Table 2. List of 89 Public Universities.

Name	COD	Name	COD
Thai nguyen University of Sciences	TNUS	Thai Nguyen University of Education	TNUE
Thai Nguyen University of Information and Communication Technology	TNUICT	Thai Nguyen University of Economics & Business Administration	TNUEB
Hung Vuong University	HVU	Thai Nguyen University of Technology	TNUT
Tan Trao University	TTU	Hanoi University of Foreign Languages	HNUFL
Hanoi University of Engineering and Technology	HNUET	Trade Union University	TUU
Hanoi University of Economics	HNUE	Electric Power University	EPU
VNU School of Education	VNUE	Na Noi University of Pharmacy	HUP
Nam Dinh University of Nursing	NDUN	University of Transportation and Communications	UTC
University of Hai Duong	UHD	Ha Noi University	HANU
Hoa Lu University	HLUV	Hai Phong University	HPU
Hanoi Procuratorate University	HNPU	VietNam Maritime University	VIMARU
Hai Duong Medical Technical University	HMTU	University of Economics—Technology for Industries	UNETI
Sao Do University	SAODO	Vietnam National University of Forestry	VNUF
The University of Finance and Business Administration	UFBA	Ha Noi University of Mining and Geology	HUMG
Hue University of Education	HUE	Foreign Trade University	FTU
Hue Univesity of Science	HUS	Hanoi University of Home Affairs	HUHA
Hue University of Agriculture and Foresty	HUAF	Ha Noi University of Education	HNUE
Hue University of Foreign Languages	HUFL	Hue University of Medicine and Pharmacy	HUMP
Hue University of Art	HUA	Hue University of Economics	HCE
Nghe An College of Economics	NAE	Hong Duc University	HDU
Da Nang University of Science and Education	UED	Vinh University	VINHUNI
Da Nang University of Foreign Languages	UFL	Da Nang University of Technology	DUT
Da Nang Univeristy of Technology and Education	UTE	Da Nang University of Economics	DUE
Da Nang University of Medical Technology and Pharmacy	DNUMTP	Nha Trang University	NTU
Pham Van Dong University	PDU	Quy Nhon University	QNU
Phu Yen University	PYU	Da Lat University	DLU
Quang Ngai University of Finance and Accountancy	QNUFA	Tay Nguyen University	TTN
HCM University of Information Technology	UIT	HCM University of Social Sciences and Humanities	HCMUSSH
Ho Chi Minh City University of Culture	HCMUC	International University	HCMIU
Vietnamese-German University	VGU	Ho Chi Minh City University of Transport	UT
Bac Lieu University	BLU	Ho Chi Minh City University of Law	HCMULAW
Can Tho Engineering and Technology	CTEUT	HCM Open University	OU
Tien Giang University	TGU	Ha Noi Open University	HNOU
Mien Tay Construction University	MTU	Banking University of Ho Chi Minh City	BUH
Thai Nguyen University of Agriculture and Forestry	TNUAF	HCM Nong Lam University	HCMUAF
Hanoi University of Science	HNUS	Sai Gon Univeristy	SGU
Hanoi University of Science & Technology	HUST	Ho Chi Minh City Pedagogical University	HCMUE
National Economics Univesity	NEU	University of Finance and Marketing Ho Chi Minh	UFM
Vietnam National University of Agriculture	VNUA	Ho Chi Minh University of Natural Resources and Environment	HCMUNRE
Thuong Mai University	TMU	Thu Dau Mot University	TDMU
Industrial University of HoChiMinh City	HUI	An Giang University	AGU
University of Economics Ho Chi Minh City	UEH	Dong Thap University	DTHU
HCM University of Technology and Education	HCMUTE	Tra Vinh University	TVU
Ton Duc Thang University	TDTU	Can Tho University of Medicine and Pharmacy	CTUMP
Can Tho University	CTU

Note: Follow the code of each university.

) allocated in 7 regions of Vietnam; namely, the Northern mountain area, the Red River delta, the North central region, the Middle South area, the High land area, Southeast area, and the Mekong area. In particular, these universities had fully implemented the three criteria published in the public report under the Ministry of Education and Training regulations—Circular No. 36/2017/TT-BGDĐT, dated 28 December 2017—of the Minister of Education and Training [43]. These three criteria included the following: publicly committed to educational quality and actual educational quality; disclosure of conditions to ensure education quality; publishing of financial revenues and expenditures.

We selected two consecutive school years, 2018–2019 and 2019–2020, to have the right view on changes in operational efficiency. Besides the data extracted from the Three-Public Report, the data on the number of international academic papers were collected from the Scientific Research Department’s statistics on each university’s official website. Those data were combined with data downloaded from the scientific database system, Scopus.

To make the study relevant for research purposes, we selected 10 criteria out of 17 in the data of public reports to the Ministry of Education and Training over two different school years. Specifically, we chose training quality assurance conditions (10 criteria), practical training quality (5 criteria), and effectiveness from financial activities (2 criteria) [43] among the three main categories of report content for analysis. To evaluate the performance of universities, this study selected the criteria that matched the operational model process, namely teaching—research effectiveness and revenue efficiency.

As shown in Figure 2, teaching and research effectiveness estimated the university’s ability to convert teaching and research inputs, including full-time teachers, nonacademic staff, and facilities construction use for training, into scientific research (m²). The five outputs included scientific research and technology transfer activities, international academic papers, number of students, graduated students, and % of students to get a job after one year. In other words, the inputs of full-time teachers, nonacademic staff, and facilities could mean attraction and, ultimately, production into the five listed criteria. In terms of revenue efficiency, we assessed the university’s ability to improve tuition returns and revenue from scientific and technical research. Specifically, the 5 criteria were as follows: scientific research and technology transfer activities, international academics, number of students, graduated students, and the % of students to get a job after one year; these were the inputs of two criteria: revenue from tuition and revenue from science research/technology transfers.

The ten criteria used in this study were defined as follows (see

Table 3. Selected ten criteria in the study.

Names of Items in the Three Public Report for Higher Education Institutions	Criteria Name
Publication of conditions to ensure the quality of education
1. Academic Staff	C1
2. Non-Academic Staff	C2
3. Facilities construction use for training, scientific research (m2)	C3
Publication of the actual quality of education
4. Scientific research and technology transfer activities	C4
5. International academic papers	C5
6. Number of students	C6
7. Graduated students	C7
8. % of Students to get a job after one year	C8
Publication of financial revenues
9. Revenue from tuition	C9
10. Revenue from science research/technology transfer	C10

):

• (C1) Academic staff: This is the full-time lecturers, such as professors, associate professors, experts, and teachers.
• (C2) Nonacademic staff: people who work in departments such as the library, public relations department, admissions office, academic office, office of international affair, student management office, etc.
• (C3) Construction use for training and scientific research (m²): the area used to build lecture halls, classrooms, laboratories, offices, sports grounds, libraries, and halls.
• (C4) Scientific research and technology transfer activities: the activities related to scientific research, technology transfer, and projects, as well as research projects at national, provincial, school, and international partner levels.
• (C5) International academic article: the total number of articles published in international conferences, journals, and projects.
• (C6) Number of students: the number of students currently studying and enrolled in the current year.
• (C7) Graduated students in a recent year: number of students who qualified and graduated in the current academic year.
• (C8) Percentage of students get a job after one year: percentage of students employed one year after graduation. This rate was surveyed by the student organization department and based on the number of previous graduates.
• (C9) Tuition fee: the total amount of tuition fees collected from students currently enrolled at the school.
• (C10) Science research/technology transfer revenue: income from research and technology transfer activities, implemented according to proposed plans and projects.

3.2. Analysis Method and Data Processing

All math and statistical calculations were performed using Excel 2013 (Microsoft Office) software. Analysis of ANOVA and Duncan post hoc tests was processed by SPSS software (PASW Statistics 20). Cluster analysis was processed by Statistica software (version 10). Statistica is known as a portfolio of advanced analytics software developed by StatSoft that provides desktop and enterprise software for statistics, data analysis, data management, data visualization, data mining (predictive analysis), and quality control.

This paper’s analysis was based on a combination of cluster analyses performed with the Ward method, Euclidean distance, ANOVA, and Duncan Post hoc test. Therefore, the research design model (Figure 3) is described as follows.

Step 1: Select universities. Specifically, 89 public universities made full reports according to the Three-Public Report to the Ministry of Education and Training (Table 2).

Step 2: Select the analysis criteria. In this study, 10 criteria for concise expression of training quality were selected (Table 3).

Step 3: The cluster analysis. This was also the first step in the data analysis process. In this step, a cluster analysis of public universities according to assessment criteria was analyzed.

Step 4: ANOVA was applied. The purpose of the application was to present overall performance and demonstrate a significant performance gap between the criteria.

Step 5: The final step was the Duncan Post Hoc test, which was applied to test the criteria precisely, and to determine the cluster’s activity level. Based on data analysis, it was easy to define specific criteria and select the group to become the benchmark for the remaining groups, thereby setting viable targets for the next stage.

4. Results and Discussion

4.1. Clustering Public University by Evaluation Criteria

As mentioned in the previous section, cluster analysis in this research uses Ward’s method as the accumulation algorithm and Euclidean distance as the distance measure. The cluster analysis results for two years (see Figure 4 for 2018–2019 and Figure 5 for 2019–2020) shows that each figure produces multiple clusters, with clusters of different distances. By creating a line as a cut on the charts, at a distance of 1.8 for all years, the authors displayed three public university clusters to be remarked in 2018–2019 and 2019–2020.

This method indicated a higher degree of uniformity among universities within each cluster regarding performance. The other 3 clusters in 2018–2019 were A1, A2, and A3, and the 3 clusters in 2019–2020 were B1, B2, and B3. The cluster features by performance are described as follows:

• The year 2018–2019 (see Figure 4).

Cluster A1: TNUT, TNUEB, TNUS, TNICTU, HVU, TTU, HNUFL, HNUET, HNUE, VNUE, TUU, EPU, NDUN, HUP, UHD, HPU, HLUV, HNPU, HMTU, HUHA, SAODO, UFBA, HUE, HUS, HUAF, HUFL, HCE, HUA, HDU, NAE, DUE, UED, UFL, UTE, DNUMTP, PDU, PYU, QNU, QNUFA, DLU, UIT, HCMULAW, BUH, HCMUNRE, HCMUC, VGU, BLU, DTHU, CTEUT, TGU, and MTU (51 universities). The group had the lowest performance of the 3 clusters. However, the group strengthens the conditions ensuring quality of training and, in particular, of maintaining academic and nonacademic staff.

Cluster A2: TNUAF, HNUS, HUST, NEU, VNUA, HNUE, VINHUNI, DUT, HUI, UEH, HCMUAF, HCMUTE, TDTU, CTU, and TVU (15 universities). This cluster had outstanding performance in terms of realistic training quality, condition training quality assurance, and financial performance.

Cluster A3: UTC, HANU, VIMARU, UNETI, VNUF, HUMG, FTU, TMU, HUMP, NTU, TTN, HCMUSSH, HCMIU, UT, OU, HNOU, SGU, HCMUE, UFM, TDMU, AGU, TNUE, and CTUMP (23 universities). The group had an average performance in training quality, scientific research ability, and facilities. However, this training cluster continues to improve and develop to enhance educational efficiency.

• The year 2019–2020 (see Figure 5).

Cluster B1: TNUS, TNUICT, HVU, TTU, HNUET, HNUE, VNUE, NDUN, UHD, HLUV, HNPU, HMTU, SAODO, UFBA, HUE, HUS, HUAF, HUFL, HUA, NAE, UED, UFL, UTE, DNUMTP, PDU, PYU, QNUFA, UIT, HCMUC, VGU, BLU, CTEUT, TGU, and MTU (34 universities). Over time, the group’s performance increased steadily. However, compared to the remaining clusters, the cluster’s operation was only 30% efficient. This is a university cluster to which attention should be paid in order to improve training activities.

Cluster B2: TNUAF, HNUS, HUST, NEU, VNUA, TMU, HUI, UEH, HCMUTE, TDTU, and CTU (11 universities). During the academic year, the group had the highest performance, with the highest academic staff producing quality international research. Strong facilities ensured students’ teaching and learning (with the highest number of students in the cluster). The cluster has a remarkable record in research and technology transfer activities in national, provincial, and city projects.

Cluster B3: TNUE, TNUT, TNUEB, HNUFL, TUU, EPU, HUP, UTC, HANU, HPU, VIMARU, UNETI, VNUF, HUMG, FTU, HUHA, HNUE, HUMP, HUE, HDU, VINHUNI, DUT, DUE, NTU, QNU, DLU, TTN, HCMUSSH, HCMIU, UT, HCMULAW, OU, HNOU, BUH, HCMUAF, SGU, HCMUE, UFM, HCMUNRE, TDMU, AGU, DTHU, TVU, and CTUMP (44 universities). The group always maintains an average performance, with the cluster’s criteria reaching 50% efficiency in 2019–2020. It is necessary to try to increase the quality of training regularly.

In general, the number of clusters remained unchanged for the selected two years. However, membership in clusters has slightly changed from 2018–2019 to 2019–2020. The A1 cluster of 51 members in 2018–2019 only had 34 members in 2019–2020; 17 members of A1 in the following school year operated more effectively and moved to cluster B3. Cluster A2 moved from 15 members to only 11 members; the number of members changed in this cluster and some moved to cluster B3. We might say that some universities may have slightly reduced performance.

4.2. Presenting a Significant Performance Gap between Criteria: General Performance

According to certain evaluation criteria, six (6) clusters were formed after applying cluster analysis to group universities. In the next step of this section, ANOVA was exerted to determine which criteria exist that are statistically significant differences in the quality of training. The criteria were identified as significant in performance level, which will be applied in a Duncan Post hoc test to discover which clusters perform better at the same level. The method of ANOVA and Duncan Post hoc test results are demonstrated in

Table 4. ANOVA and Duncan post hoc test results for the year 2018–2019.

Criteria	Cluster Means in Each Criterion	ANOVA Result	Duncan Post Hoc Test
Full-time lecturers	A1 = 246.63 A2 = 770.27 A3 = 514.13	F = 72.458 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1 < A3 < A2
Nonacademic staff	A1 = 111.33 A2 = 279.40 A3 = 198.39	F = 66.629 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1 < A3 < A2
Facilities construction use for training, scientific research (m2)	A1 = 29,842.61 A2 = 118,139.32 A3 = 65,139.30	F = 9.159 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1 < A3 < A2
Scientific research and technology transfer activities	A1 = 32.49 A2 = 104 A3 = 60.48	F = 28.992 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1 < A3 < A2
International academic papers	A1 = 34.73 A2 = 195.87 A3 = 42.87	F = 10.191 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1, A3 < A2
Number of students	A1 = 5439.06 A2 = 22,818.73 A3 = 13,965.35	F = 66.555 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1 < A3 < A2
Graduated students in most recent year	A1 = 957.53 A2 = 3206 A3 = 2433.57	F = 32.570 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1 < A3 < A2
% of students to get a job after one year	A1 = 86.82 A2 = 88.17 A3 = 86.84	F = 0.088 < FCrit (2, 86, 0.05) = 3.102 Sig = 0.915	Non-significant differences
Revenue from tuition	A1 = 48.576 A2 = 327.62 A3 = 188.81	F = 45.945 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1 < A3 < A2
Revenue from science research/technology transfers	A1 = 1.67 A2 = 20.93 A3 = 2.31	F = 20.929 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	A1, A3 < A2

Cluster 1 = A1; cluster 2 = A2; cluster 3 = A3; *** p < 0.001.

and

Table 5. ANOVA and Duncan post hoc test results for the year 2019–2020.

Criteria	Cluster Means in Each Criterion	ANOVA Result	Duncan Post Hoc Results
Full-time lecturers	B1 = 195.82 B2 = 749.36 B3 = 487.89	F = 49.392 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1< B3< B2
Nonacademic staff	B1 = 99.68 B2 = 82 B3 = 270.82	F = 49.016 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1< B3< B2
Facilities construction use for training, scientific research (m2)	B1 = 23,557.06 B2 = 178,995.99 B3 = 61,714.85	F = 20.685 > FCrit (3, 41, 0.05)s = 2.833 Sig = 0.000 ***	B1, B3 < B2
Scientific research and technology transfer activities	B1 = 31.59 B2 = 83.36 B3 = 64.43	F = 8.977 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1 < B3, B2
International academic papers	B1 = 31.50 B2 = 355.91 B3 = 56.09	F = 7.301 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1, B3 < B2
Number of students	B1 = 3652.76 B2 = 24,064.73 B3 = 12,603.50	F = 52.951 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1< B3< B2
Graduated students in most recent year	B1 = 733.29 B2 = 3997.91 B3 = 1949.73	F = 42.474 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1 < B3 < B2
% of Students to get a job after 1 year	B1 = 87.79 B2 = 90.49 B3 = 87.44	F = 0.388 < FCrit (2, 86, 0.05) = 3.102 Sig = 0.679	Non-significant differences
Revenue from tuition	B1 = 32.68 B2 = 443.84 B3 = 152.45	F = 65.743 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1 < B3 < B2
Revenue from science research/technology transfers	B1 = 0.91 B2 = 21.46 B3 = 4.98	F = 15.158 > FCrit (2, 86, 0.05) = 3.102 Sig = 0.000 ***	B1, B3 < B2

Cluster 1 = B1; cluster 2 = B2; cluster 3 = B3; *** p < 0.001.

.

• Nonsignificant Differences Criteria.

In the two years 2018–2019 and 2019–2020, the criteria of a percent (%) of students to get a job after one year, which were showed by ANOVA result (F = 0.088, p = 0.915), (F = 0.388, p = 0.679) (see Table 4 and Table 5), respectively, is a nonsignificant difference. These results mean that, as in the above criterion, in terms of statistical significance, all clusters provided similar quality levels for the two years, due to their similar scores.

• Significant Differences Criteria.

The ANOVA results revealed significant differences for the nine (9) remaining criteria: full-time lecturers (F = 72.458, p < 0.001); nonacademic staff (F = 66.629, p < 0.001); facilities construction use for training-scientific research (m²) (F = 9.159, p < 0.001); scientific research and technology transfer activities (F = 28.992, p < 0.001); international academic papers (F = 10.191, p < 0.001); number of students (F = 66.555, p < 0.001); graduated students in the most recent year (F = 32.570, p < 0.001); revenue from tuition (F = 45.945, p < 0.001); and revenue from science research-technology transfers (F = 20.929, p < 0.001) (see Table 4).

Continuing with the results in Table 5, the following demonstrate that the quality of education offered by private universities has changed over the two year period: full-time lecturers (F = 49.392, p < 0.001); nonacademic staff (F = 49.016, p < 0.001); facilities construction use for training-scientific research (m²) (F = 20.685, p < 0.001); scientific research and technology transfer activities (F = 8.977, p < 0.001); international academic papers (F = 7.301, p < 0.001); number of students (F = 52.951, p < 0.001); graduated students in the most recent year (F = 42.474, p < 0.001); revenue from tuition (F = 65.743, p < 0.001); and revenue from science research-technology transfer (F = 15.158, p < 0.001) (see Table 5) (all at 1% alpha level).

In the year 2018–2019, according to the three clusters grouped, nine criteria have significant differences. In other words, a significant performance gap exists between the three clusters. In comparison, in the remaining period of 2019–2020, there are statistically significant differences in the remaining nine criteria. Thus, there are different performance values between the two selected years. The input criteria, such as conditions to ensure training quality, have changed, leading to a change in output value, such as financial performance and actual training quality. Therefore, focusing on the inputs and ensuring quality of education should be considered for performance to be run effectively.

Theoretically, the ANOVA method supports statistically significant differences for at least one pair of clusters; but in terms of specific locations between clusters, ANOVA does not indicate differences. In the next step, a Duncan Post hoc test is applied to determine the cluster performance difference for each criterion. Thereby, groups of activities that have common or different, good or poor, performances, will be presented.

4.3. Deciding the Levels of Cluster Performance: Point Out the Standards

The purpose of the ANOVA in-depth analysis is to see if there is a statistically significant difference between groups, and which group on a given issue is significant. Therefore, a Duncan post hoc test is used to evaluate the actual cluster level. The overall assessment of the criteria’s quality over the two academic years is shown in Table 4 and Table 5. Tissue-detailed difference descriptions and cluster comparisons between the nine criteria are presented as follows:

• Full-time lecturers: This is a team to improve students’ knowledge and one of the essential conditions for ensuring quality of training. Moreover, this is the team that produces research activities, with a contribution to the sufficient revenue of the university. In this criterion, cluster A1 is considered to have the worst performance. A3 is better than A1. Cluster 1 achieved the highest performance among collections. Thus, A2 becomes the standard for A3, while A3 is considered as the benchmark for A1.

In 2019–2020 there is a change in the position of the cluster. Some member fields of cluster A1 and cluster A2 became members of cluster B3. Cluster B2 reached the highest efficiency between clusters. Cluster B3 achieves the average value of the academic staff in terms of quality assurance for the university. Obviously, B1 needs improvement, as it is among the most underperforming clusters. Thus, B2 becomes the benchmark for B3. Likewise, B3 become the benchmark that B1 needs to perform.

• Nonacademic Staff: Staff do not directly contribute to training, but they are required to support to the academic team and students. Thanks to this team, the operation between departments in the school becomes integrated, and the activities to support teachers and students are in better operation. It is similar to the first criterion, in the 2018–2019 school year; although the A1 cluster is ranked in the same unremarkable position, A1 still has the lowest value. A1 can consider A2 and A1 as the benchmarks to follow. A2 is better than A1, and it has also become the benchmark for A1.

Over the next school years, the clusters had quite similar positions. Reaching highest performance, B2 is the benchmark for B1 and B3. However, the value of cluster B3 is higher than B1, and it is the point of reference for B1 when B4 had the lowest performance in all clusters.

• Facilities construction use for training, scientific research (m²): This criterion is also one of the conditions for ensuring training quality. When the facilities meet faculty and students’ teaching needs, the quality of the school’s training will be developed. The level performance of cluster A2 is outstanding, and it is the benchmark for A1 and A3. The values of A3 are higher than A1, and it became the benchmark for A1. In the 2019–2020 school year, although B1 and B3 are not significantly different, B1 still had lower performance than B3 and B2. In these criteria, B2 has the highest performance, which is the benchmark for B1 and B3.
• Scientific research and technology transfer activities: This criterion demonstrates the effectiveness of both faculty and student research activities in state, ministry, provincial and city projects. Comparing three clusters in the school year, 2018–2019, A1 and A3 had low efficiency, and their performance was less than A1. At the same time, A1 is less than A3. Therefore, A2 becomes the benchmark of A3, and A3 becomes the benchmark for A1.

The following year, there was no significant gap between B2 and B3, but there was a gap between them and B1. Clusters B2 and B3 had good performance, so they become the benchmark for the remaining clusters in B1.

• International academic papers: This criterion is also one of the research activity results that all universities desire to achieve. Many universities maintain and improve the quantity and quality of international publications to enter the prestigious rankings. Results of the year 2018–2019 revealed that the A3 and A1 clusters had nearly the same performance gap. However, they are a considerable distance from the A2 cluster. With the highest performance value, A2 becomes the benchmark for A3 and A1.

Because of the role of this criterion, the university has increased its performance over the years. Cluster B2 had the highest score among clusters, which became the benchmark for B3 and B1 clusters when these clusters do not have a significant performance gap.

• Number of students: This criterion shows the actual performance of the university. The number of students also shows the university’s enrollment attraction when a reputable university can eclipse students’ many choices. In 2018–2019, there is no similarity in the distance between A2, A1, and A3. With much higher efficiency, the A2 cluster becomes the benchmark for A1 and A3, while A3 is the benchmark for A1. The number of students in all clusters changed slightly compared to the previous year. B2 presents the highest score and begins to be the benchmark for B1 and B3. However, B3 outperforms B1 and become the benchmark for B1.
• The graduated students the most recent year: The number of graduates expresses the quality of training. If the rate is low, it is necessary to review the quality of lecturers, teaching, and the students’ learning capacity. At this point, the educational manager provides a solution to education at their university. In this criterion of the year 2018–2019, cluster A1 had the lowest productivity. A3 is better than A1. A2 achieved the most outstanding value compared to the remaining clusters. Thus, A2 came to be the benchmark for A1 and A3. Similar to the above case, cluster B2 reached the cluster’s highest quota and became the benchmark for B3 and B1.
• Revenue from tuition: Tuition fee revenue has also shown efficiency from training activities. It is also achieving an upshot when a large number of students choose to study at the university. Cluster A2 was higher than A3 and A1, and it was the benchmark for A3 and A1. Cluster A1 it is still smaller than A3. Hence, A3 is the standard point for A1. The performance of the schools has changed year by year, so the revenue is also different. In 2019–2020, B2 reached the highest standard, so it was the B3 and B1 benchmark. Clusters B3 and B1 also witnessed a big gap; they were higher than B1 and became the benchmark for B1.
• Revenue from Science Research/technology transfers: One of the vital outputs for training is revenue from science research and technology transfers. It demonstrates the effectiveness of scientific research and technology transfer activities by academic staff and students. However, not all universities can attain big revenue from this criterion. This is reflected in the performance achieved by the fields; in fact, they all demonstrated low a performance. For the 2018–2019 school year, A3 and A1 achieved nearly the same income. Only A2 had the highest yield across the clusters. As such, it became the benchmark for the remaining clusters.

This criterion increased slightly over the years; in the next year there is almost no striking change. B3 and B1 still did not have a significant performance gap and chose B2 as the benchmark to improve upon.

The final results from these tests above can provide educational managers with insight into training quality criteria and performance. For the analyzed criteria, managers will consider the important criteria when assessing the quality of the university’s education that is still satisfied in demonstrating actual training quality, training quality, efficiency assurance, and financial revenue. In terms of quality performance between clusters, administrators will have the advantage of identifying efficient and ineffective clusters. There will be investments and incentives to maintain quality in clusters that perform well. In poor performance clusters, there will be an appropriate strategy for motivating or refining. Furthermore, thanks to clustering, university administrators can compare their own universities’ performance with other universities, then adjust and improve against the benchmarks of high-quality universities.

5. Conclusions

To better understand the public university system’s operational performance in Vietnam, in this study, the authors selected 89 public universities and applied a research model, including cluster analysis, ANOVA, and a Duncan Post hoc test, to examine the following ten criteria: full-time lecturers, nonacademic staff, construction use for training and scientific research, scientific research and technology transfer activities, international academic papers, number of students, graduated students in the most recent year, percentage of students to get a job after one year, tuition revenue, and science research/technology transfer revenue. This analysis aims to support the overall management strategy of public universities for public education administration. This result helps management boards develop specific and reasonable policies for allocating finances to public universities. Besides, the universities also review their training capacity. The weak criteria need to be upgraded and improved to provide optimal human and social development activities.

The study’s method helps classify public universities based on the criteria of training quality, evaluates an overview of performance, presents significant performance gaps between criteria, and, finally, indicates performance levels of the clusters by building up the benchmark. In two consecutive school years, 2018–2019 and 2019–2020, three school clusters were collected through the results of cluster analysis. In the ten criteria mentioned above, the criteria of percentage of students having a job after one year, presented by the ANOVA results, show all clusters provide a level of quality similar in both years. The remaining nine criteria have significant differences between groups. Then, the clusters with better and weaker performance are indicated. The cluster that performed well was suggested as a benchmark for the remaining clusters to follow. Based on this clustering and benchmark formula, educational leaders can identify the public education system’s weaknesses. The university’s practical training activities for other schools in the same field or region has received a more objective view in making a strategy of constant development, or improvement of weak criteria. Furthermore, they may see a clear goal for development in the coming evaluation years.

Through the results of clustering—identifying different levels of quality performance between clusters and criteria that differ significantly—the authors make objective recommendations for educational managers.

Firstly, based on the clusters’ performance efficiency, whether good performance, average performance, or poor performance, education managers have a policy of merging, consolidating or dissolving universities, educational institutions, and training ineffective operations. It is not necessarily that all provinces have universities. Furthermore, it is significant to rearrange, reorganize, and focus on building several key pedagogical schools to train teachers and educational management [12]. Good implementation of tasks will contribute to restructuring the state budget expenditure for higher education for public universities. The state budget will be spent on a restructured number of public universities, focusing on high-quality universities and special universities through training ordering modes. The state budget will avoid spending on inefficient universities or universities that do not need to be maintained.

Secondly, implementing the current autonomy policy is a task that is placed on the public university. However, implementing this state policy is not easy [44]. The high-performing public university clusters are the benchmarks for the remaining clusters selected for piloting, with 100% autonomy on frequent operating funding. The implementation of the financial autonomy and self-responsibility mechanism has created conditions for universities to organize professional activities, link the management, and use financial resources, with quality and efficiency in operations. Accordingly, the universities have conditions to mobilize financial resources, contribute to improving the quality of performing professional tasks, and increase income for the university’s faculty and workers.

6. Limitations and Future Research Suggestions

In terms of research limitations, this study was conducted for 89 public universities in Vietnam, while the public university system has 172 universities. However, based on public reports made in full over the years, just over half of the universities with full data were selected. Although the Vietnamese higher education system includes public universities and private universities, the two-type operating mechanisms are somewhat different in finance. As such, the authors chose to analyze each type separately. Future research will have a combination of private and public for an objective comparison. This study verifies university performance by analyzing 10 criteria out of a total of 17 in the data of public reports to the Ministry of Education and Training over two different school years. Specifically: training quality assurance conditions (10 criteria), practical training quality (5 criteria), and effectiveness from financial activities (2 criteria). There is no specific standard scale for these criteria; it is only shown on the data reported by each school. Therefore, future studies may incorporate self-assessment reports. These reports collect current quality assessment reports of universities to find a new scale for objective assessment and more details.