Effectiveness and Sustainability Indicators in Higher Education Management

Abstract

This research aims to analyze the determinant factors of the effectiveness and sustainability indicators of higher education management at the Federal University of Santa Catarina. The research is characterized as a quantitative study of inferential nature, with structural equation modeling. Data were collected from a survey and applied to professors and employees of two teaching centers at the Federal University of Santa Catarina. The theoretical basis is the Baldrige Excellence Model. This research brings together several theoretical and practical implications involving the sustainability of higher education management, because higher education institutions have been under pressure to improve their performance, and many governments have implemented new regulations that seek to professionalize universities in pursuit of excellence. So, this paper discusses one of the models in the academic literature, the Baldrige model, which focuses on the effectiveness of university management, encompassing quality improvement and excellence in management. The results obtained show that the construct leadership, focus on the student, and society, have a direct effect on strategic planning and an indirect effect on the effectiveness of management. In addition, the constructs of transparency and management by competencies have a direct effect on work processes. Universities face a management challenge and demand new quality models.

1. Introduction

The university, due to its interdisciplinary profile, and operating with the teaching, research, and extension tripod, has a unique organizational profile. These institutions interconnect with a large number of segments of society, compared to other organizations. With this, there is also an expectation of society when it comes to the quality of services provided by public universities, which must have as a goal, continuous improvements to achieve good scientific, economic, and social performance for all. In this scenario, the management of universities is on the discussion agenda of both political and academic bodies.

Management and educational area are related to methods, projects, ideology, and politics, but despite both dealing with people, education carries with it the noble mission of, with its practices, informing, training, and transforming those with whom it comes into contact, leading students to be thoughtful and critical.

This research aims to contribute to the constant search to improve the management of higher education institutions. To this end, it is proposed the use of the Baldrige Excellence Model; considered one of the most prominent models to integrate organizational planning, assessment, and improvement [1].

Managing a university involves a wide variety of audiences to be served, which include students, faculty, employees, parents of students, government officials, and boards of directors. Generally, each of these parties has competing interests or expectations, which makes the work of higher education leaders especially challenging [1,2].

There is a demand for research in university sustainability and management that finds an adequate model, which is not necessarily focused on a business model based on profit, but that meets the criteria of excellence for the improvement and effectiveness of higher education management. Thus, the goal of this research is to analyze the determining factors of the effectiveness of higher education management at the Federal University of Santa Catarina. The idea is to create a model with a set of criteria for excellence in the management of universities. It was chosen to use the Baldrige Excellence Model, as the model aims to respond to the need of higher education institutions by combining the results of national development and innovation, through best practices adopted in management [2].

The search for information on strategic management is important in the university scenario, especially in Brazil. First, it is a topic linked to institutional evaluation at higher education institutions. In addition, it manages to capture the institutional movement present in the institution’s structure and contribute to the development process and new management practices [3,4,5]. Additionally, because the educational market in Brazil is undergoing several transformations due to the growth in higher education institutions, implying the demand for an adaptation of the management models of higher education institutions.

The choice of this research topic is because strategic planning positively impacts the effectiveness of university management. Having said that, it is possible to say that management is a necessary practice for the proper functioning of any institution, especially in universities. Using the right strategies, the university can evolve in the educational field, opening new opportunities, lines of research, expanding infrastructure, among other things. Therefore, the goal of this work is to analyze the antecedents of the effectiveness of higher education management considering the Baldrige Excellence Model, and as a consequence to promote an improvement in this sense.

2. Literature Review

Altbach stated that universities present several problems in terms of administration [6]. The author believes that, due to the bureaucratic structure, added to their specific functions, in contrast to the high level of autonomy of their employees, universities deserve special attention in terms of management.

In the 1960s, great transformations began, which led the university to become the complex institution it is today. The concept of institutional “organism” contrasted with that “mechanism”—a series of processes that produce different results, a pluralist organization with different purposes, different centers of power, and distinct clientele. Little by little, universities have adopted structures that resemble conglomerate institutions, in which the units have a life of their own and are practically independent of each other [7]. The mission of universities—such as research, teaching, extension, and in some cases, even medical clinics—encompass a wide range of stakeholders.

In addition, the context of work in higher education has undergone significant changes in recent decades, resulting in economic, social, and technological changes, which lead the various actors involved in this field to undertake structural and behavioral changes [8]. Canterle and Favaretto state that dynamic scenarios set challenges for universities, requiring them to develop new management, efficiency, and financial alternatives to provide services of value to society [9].

In the context of higher education, teaching has undergone revisions and changes. There were changes in the professor’s career, expanding the activities commonly associated with these professionals, notably teaching, research, and extension, due to the deepening of the performance culture, which denotes the overvaluation of knowledge as a new form of capital accumulation [10].

The new scenario in which higher education institutions operate has created the need for the constant analysis of the organizational environment and adopting competitive positioning that provides these companies a leadership position. Leadership can be achieved when creating a competitive advantage, for example, by reducing costs or by developing a differentiation. To achieve this goal, it is still necessary to develop the ability to predict the future of the sector. In other words, an organization has no possibility of success without an articulated vision of the opportunities and challenges it will face. Higher education institutions need to manage educational policies for scholarships, internationalization, and evaluation [11].

Badri et al. found in their work that much research is carried out in the educational field, on teaching mechanisms and quality; however, there is still a need to produce works that demonstrate the effectiveness of quality programs and processes in the organizational results of an educational institution [12].

To develop this strategic vision, the managers of these institutions must develop the skill called information competency. It is this competence that will allow them to feel the need, and perceived availability, obtain and process information, and, from there, practice in the environment in which they work. A manager with competence to deal with information has a greater ability to understand the environment in which it operates and the variables that affect it, thus being able to trace the most appropriate paths to be followed by institutions, through a strategic action plan. In a manner, it is important to emphasize that strategic management premises flexibility, emphasis on information, knowledge as a critical resource, and integration of processes, people, and resources.

In the context of adapting to the environmental instability that characterizes the current scenario, it is assumed that managers can present proactive initiatives and articulate actions that value flexibility and managerial innovation—recommendations somewhat trivialized by the managerial literature, yet attributes not always found in organizations [7]. The organizational process consists of setting goals, monitoring them, and using this information to prepare and execute new organizational practices [1]. There is a need for organizational evaluation, to value excellence in higher education to help educational institutions improve their performance [13]; using the Baldrige method as the most prominent and well-regarded method for achieving this excellence. The Baldrige Prize is not a quick-fix program; it is a transformational journey that requires dedication and ongoing implication from upper management [14]. From the moment an organization executes the Baldrige self-assessment, to understanding and knowing its current state of performance excellence, to reaching a world-class organization and winning the Baldrige award, it takes many years of work.

The political model for managing universities was formulated by Baldrige [2], on the assumption that they are complex organizations that can be studied as political microsystems and that have many of the characteristics of organized anarchy.

The Baldrige Performance Excellence Program has the advantage of being a complete structural transformation, which is easy to understand and follow. It can be used to improve operational resources, and verify new flow capabilities needed to help organizations innovate and sustain their business, thus increasing their competitive advantage.

Table 1. Baldrige model criteria.

Criteria	Detailing
Leadership	How upper management leads the organization
Strategy	How the organization establishes and plans to implement strategies
Customers	How the organization builds and maintains good relationships with customers
Measurement, analysis, and knowledge management	How the organization uses data to support processes and improve performance
Workforce	How does the organization prepare workers
Operations	How the organization designs, manages, and improves its processes
Results	How the organization performs final products or services and how it compares to competitors.

Source: own elaboration.

presents the criteria of the Baldrige model.

With the elucidation of the Baldrige criteria and their relationships, it is possible to build the research hypotheses of this work, to verify and develop a model of excellence for a higher education institution. It is noteworthy to mention that for the originality of this research, the criteria of transparency and management by competencies were included, in place of the criteria of measurement, analysis, and knowledge management; and workforce focus, respectively.

Research Hypotheses

The Baldrige Excellence Model takes into account that the Baldrige excellence criteria are divided into seven categories and must be analyzed separately [2]. This work seeks to answer how they influence the effectiveness of higher education institutions.

Top management leadership is seen as a primary element of quality performance [1,15,16]. The leadership criterion evaluates the practices of organizational leaders and the interpretation of these practices so that the strategic goals of an institution are achieved [2,17]. This must be verified through Hypothesis 1.

Hypothesis 1 (H1).

Leadership positively influences strategic planning.

The analysis of strategic planning is considered the development of the preparation of higher education institutions. For this, plans are evaluated as a key element, and goals and adaptations to environmental changes are also controlled, measuring the competence to adapt to changes [2,12]. This should be verified through Hypothesis 2.

Hypothesis 2 (H2).

Strategic planning positively influences the effectiveness of higher education institutions management.

The focus on students and society is defined by the search for satisfaction and quality, promoting changes and development [14,16,17]. The increased probability of evasion in higher education is also one of the effects of the COVID-19 pandemic. Therefore, the focus on students and society highlights the need to seek solutions for university management. Higher education sustainability demands adjustments to their development and management plans, following the transformations that universities have been going through [13,18]. The focus on students is verified through Hypothesis 3.

Hypothesis 3 (H3).

Focusing on students and society positively influences strategic planning.

The transparency criterion was adapted and chosen to be done this way, due to the growing importance of information on this topic in public management. Since the end of the 14th century, public management in Brazil is undergoing major changes, encouraged by social demands that demand greater efficiency, transparency, and effectiveness when applying public resources [11,13,19]. As this is a study in a public university, it is convenient to verify if this topic of discussion is being verified by the management and if it contributes to its effectiveness. This must be identified through Hypothesis 4.

Hypothesis 4 (H4).

Transparency positively influences the work processes of higher education institutions.

The focus on management by competencies is a touchstone that relates to the development of knowledge, abilities, and attitudes of the people working in the higher education institution. This being said, measures are sought on how to create a conducive work environment to develop better performance teams and ways in which these reverberations will directly influence the results [11,13,19,20]. Competency management includes individuals and teamwork, which can generate results for the effectiveness of management. Therefore, competency management is proposed in Hypothesis 5.

Hypothesis 5 (H5).

The focus on management by Competencies positively influences the effectiveness of the management of higher education institutions.

Focus on operation is about the strategy in the possible ways in which the organization will operationalize the plans to respond to the demands in pursuit of the goals. This model criterion, in addition to analyzing the organization of current work processes and projects, seeks to innovate existing processes, predicting possible risks; with this, it intends to achieve the efficiency and sustainability of operations, to deal with emergencies too [2,16,17]. These processes are analyzed through Hypothesis 6.

Hypothesis 6 (H6).

The focus on operation positively influences the management effectiveness of highereEducation institutions.

Lastly, the Baldrige Excellence Model works with the last hypothesis, verifying if all the previous criteria impact the effectiveness of the management of the body analyzed [16,17]. Then, Hypothesis 7 is created:

Hypothesis 7 (H7).

There is a correlation between the constructs of the structural model and the effectiveness of the management of higher education institutions.

Table 2. Constructs and observed variables.

Constructs	Observed Variables	Authors
Leadership	Inspiration, communication, ethics, training for team members	[1,2,15,16,17]
Strategic planning	Identification with the organization, engagement with the organization, support of the organization, differentials of the organization, adaptation to changes	[12,16,17]
Focus on students and society	Ombudsman, student involvement, loyalty, attracting new students	[14,16,17,18]
Transparency	Data collection and monitoring, trust, decision making, organization strategy	[11,13,19]
Management by competencies	Skills, compensation, performance, training, turnover	[11,13,19,20]
Focus on Operation	Freedom to change, productivity, rework, operations in atypical situations, suppliers	[2,16,17]
Management effectiveness	Monitoring results, satisfaction, achieving goals, synchrony with planning	[16,17]

Source: own elaboration.

presents the constructs and their observed variables.

3. Research Method

The research is marked as quantitative, based on the multivariate statistical method called structural equation modeling [21]. The time horizon was transversal, in a cross-section analysis.

The study was executed through a survey, based on a questionnaire applied to all employees of the Socioeconomic Center and Legal Sciences Center of the Federal University of Santa Catarina. These two centers are very representative in terms of size, number of professors and technical-administrative staff. Technical administrative and teaching staff were interviewed. UFSC was founded in 1960, and currently has 11 teaching centers on Campus Trindade in Florianópolis, and four more campuses throughout the state, with one teaching center each. It was decided to carry out this research at the Socioeconomic Center and Legal Sciences Center, as this is the place where the researcher studies as well as works.

To prepare the questionnaire applied through a survey, the description of the variables is presented in

Table 3. Presentation model of the selected variables.

Variable Abbreviation	Type of Variable in the Hypothesis Testing Model	Description of the Variable	Description of the Variable	Data Collection Source
CO	Independent variable	Position held in the institution	Position held in the institution	Questionnaire
QF	Dependent variables	Questions formulated	Questions formulated	Questionnaire

Source: own elaboration.

.

The application of the questionnaire was based on the Baldrige Excellence Model, to verify the effectiveness of management at UFSC, specifically in the Centers for Socioeconomic Education and Legal Sciences. The sample was characterized as non-probabilistic convenience (adherence), through the online collection (e-mail) for all servers of these centers, as well as a personal approach to collect the responses.

For models with few variables, as in the present study, using regression analysis through the structural equation model, a sample of between 100 and 200 individuals on average is the minimum recommended [22]. The research sample consisted of 117 respondents.

The questions were replicated from the article Baldrige excellence criteria in the effectiveness of the management of higher education institutions [23]. The questions of this research were prepared based on the Baldrige Excellence Model [2] using as support a theoretical framework.

The questionnaire was designed with responses on a Likert scale, with a score ranging from 1 (strongly disagree) to 5 (strongly agree), and consisted of 50 questions. The research instrument is presented in the appendix of this work.

To calculate the sample size, the GPower software was used. To do so, the construct or latent variable that obtains the highest number of arrows or has the highest number of predictors must be evaluated. In this research, the construct effectiveness in top management receives two arrows (it has two predictors), so it is the construct selected. For the calculation (prior collecting data) it should be noted that there are two parameters: the power of the test (Power = 1 − βerror prob. II) and the size of the effect (f2). The literature recommends using power as 0.80, median f² = 0.15 [24].

For the analysis of the power of the test, the following input parameters were considered: (a) an effect size f² of 0.15; (b) an alpha error (α) of 0.05; (c) a power (Power = 1 − β error prob. II) of 0.95; (d) two predictors; (e) f test (Fisher and Snedecor probability distribution); (f) statistical test for multiple linear regression, with fixed effects model, and R2 deviation from zero; and (g) type of power of a priori analysis, considering the required sample size calculation based on alpha, power and effect size. The result obtained the power of 0.9518556 or a probability β (1 − β) of incurring a Type II error. For this calculation, two parameters were observed: the power of the test (Power = 1 − βerror prob. II) and the size of the effect (f²). It is recommended [24] to use the test power equal to 0.80, and a median f² = 0.15. In this case, the minimum sample calculated was 107 cases. The value of 117 validated questionnaires exceeds the minimum sample for a statistical power of 0.95 and an effect size of the independent variables of 0.15 [24].

In addition, a non-centrality parameter γ of 16.05, a critical F value of 3.0837, 2 degrees of freedom, and a final test power of 0.9518556 were obtained as results.

After completing the field research, the data were charted in an Excel spreadsheet and analyzed using the structural equation modeling (SEM) method with partial least squares (PLS). Considering the profile of the participants, 57 technical-administrative employees (48%) and 60 professors (52%) were interviewed.

In this research, the method of modeling structural equations by partial least squares was applied. It is a multivariate analysis technique that combines multiple linear regression and common factor analysis [24]. Unlike other regression techniques, this method allows representing of multiple relationships simultaneously and specifies the causal direction. Structural equation modeling can present non-observable concepts in complex models, which makes it an advanced multivariate analysis tool. The PLS method of path modeling was developed as a sequence of regressions in terms of weighting vectors.

4. Analysis of Results

The proposed initial model, with all the questions of all the constructs, is presented in Figure 1. For the elaboration of this model, all the variables (item coding, represented by the rectangles), which belong to their respective latent variables or constructs (represented by the circles), were considered.

This model was tested and submitted to a confirmatory factor analysis for the composition of the measurement model. The model adjustment was performed through tests of convergent and discriminant validity [22,24].

First, the existence of outliers was verified from the analysis of the squared Mahalanobis distance. As there were no significant observations to be excluded, a conservative outliers exclusion procedure was adopted. This procedure meets one of the hypotheses required by the structural equation model (SEM): the absence of outliers.

To calculate the partial least squares (PLS) algorithm, the PLS path modeling method [8] was applied. The PLS algorithm is a sequence of regressions in terms of weighting vectors.

4.1. Weighting Schemes

The PLS-SEM algorithm allows the researcher to execute three possible weighting arrangements of the structural model: (a) centroid-based weighting scheme (centroid); (b) factor-based weighting scheme (factor); (c) path-based weighting scheme (path). For this research, it was chosen to apply the weighted model based on paths. The reason is, while the results differ little for the different alternatives of weighting schemes, path-based weighting is the approach recommended by the literature. Furthermore, this weighting system provides a higher R² value for endogenous latent variables and applies to all types of specifications of the PLS path model and estimations.

4.2. The Criterion of the Maximum Number of Iterations

This parameter represents the maximum number of iterations that are applied to calculate the partial least squares results. This number must be great enough. In this research, 300 iterations were applied. When verifying the PLS-SEM result, it was necessary to certify that the algorithm did not stop due to the maximum number of iterations reached, but because of the stopping criterion. The value 0 (zero) for the maximum number of iterations would allow obtaining the results of the sum of scores approach, which uses the average value of criteria to determine the score of the latent variable.

4.3. The Stopping Criterion

The partial least squares (PLS) ends when the change in the outer weights between two consecutive iterations is less than the stopping criterion value. This value ought to be small enough (for example, between 10⁻⁵ and 10⁻⁷). In this research, the value of 10⁻⁷ was adopted as a stopping criterion, considered a very low and efficient value.

The first model, therefore, is shown in Figure 2. In the model, the observed variable (VO), also called the measured variable or indicator, is represented by the rectangle. The construct or latent variable (VL) is represented by the circle. The arrow represents the correlation between the latent variable (circle) and the observed variable (rectangle). The arrow that starts from a latent variable to another latent variable, that is, between two circles, represents a causal relationship, which implies a path coefficient between the dependent variable (Y) and the independent latent variable (X).

After creating the first structural equation model (SEM), it is necessary to evaluate the measurement models and after their adjustments, evaluate the path model [20]. Following this, the first aspect to be observed of the measurement models is the convergent validity, obtained by observations of the average variances extracted (Average Variance Extracted—AVEs). For this, the Fornell and Larcker criterion will be used next [24], the values of AVEs must be greater than 0.50 (AVE > 0.50).

4.4. Reliability and Validity Assessment

For the analysis of the reliability of the construct, the scientific literature recommends a Cronbach’s alpha greater than 0.7, a Rho greater than 0.7, a composite reliability greater than 0.7, and an AVE greater than 0.5.

Table 4. Reliability and validity of the construct for the first structural equation model.

	Cronbach’s Alpha	Rho	Composite Reliability	Average Variance Extracted (AVE)
Management effectiveness	0.902	0.910	0.920	0.562
Focus on student and society	0.756	0.805	0.828	0.454
Competency management	0.733	0.744	0.810	0.354
Leadership	0.793	0.825	0.855	0.505
Strategic planning	0.740	0.767	0.815	0.394
Work processes	0.750	0.807	0.828	0.460
Transparency	0.902	0.906	0.923	0.634

Source: own elaboration.

presents the results of the first analysis performed for the reliability indicators.

From Table 4, it can be noticed that concerning Cronbach’s alpha, all constructs presented values greater than 0.7. As for the Rho calculation, all constructs presented values greater than 0.7. In terms of composite reliability, the constructs presented values greater than 0.7. The exception occurred with the calculation of the AVE, as it was observed that the following constructs presented values below 0.5: focus on the student and society (0.454), management by competencies (0.354), strategic planning (0.394).

As in this initial analysis of the model, three constructs presented values below 0.5, for situations like this the scientific literature recommends eliminating variables from the constructs that present an AVE < 0.50. Thus, indicator III.4 was initially excluded and a new PLS model was generated. Consequently, the AVE of the construct focus on the student and society improved, reaching 0.502. Then, for the competency management construct, it was excluded indicator V.5 (which had the lowest correlation with the construct, corresponding to 0.432), indicator V.7 (second lowest correlation, which corresponded to 0.524), the V.8 indicator, the V.2 indicator, and lastly, the V.3 indicator. For each indicator exclusion, it was necessary to run a new calculation using the Partial Least Squares algorithm. Then, for the strategic planning construct, indicator II.1 was initially excluded (lowest correlation, 0.415). Following the event, indicator II.3 was excluded. At last, indicator II.5 was excluded. For the construct entitled work processes, variable VI.1 was excluded (since it had the lowest correlation, corresponding to 0.405). By excluding these indicators, it was possible to come up with a table with excellent reliability results (see

Table 5. Reliability and validity of constructs for the final model of structural equation modeling.

	Cronbach’s Alpha	Rho	Composite Reliability	Average Variance Extracted (AVE)
Management effectiveness	0.902	0.91	0.92	0.563
Focus on students and society	0.747	0.788	0.83	0.502
Competency management	0.706	0.735	0.833	0.625
Leadership	0.793	0.839	0.854	0.504
Strategic planning	0.708	0.716	0.82	0.534
Work processes	0.765	0.801	0.843	0.526
Transparency	0.902	0.906	0.923	0.634

Source: own elaboration.

).

After ensuring the convergent validity by analyzing the average variance extracted (AVE), the observation of values of internal consistency (Cronbach’s alpha) and composite reliability (CC) (Rho de Dillon-Goldstein) was analyzed. Cronbach’s Alpha (AC) is considered the traditional indicator and is based on intercorrelations of variables. Table 5 demonstrates that Cronbach’s alpha and de composite reliability values are adequate.

Figure 3 shows the structural equation model with seven variables and which presented factor loadings of lower values. To clarify, AVE is the average of the squared factors loading. Thus, to increase the value of AVE, variables with factor loadings (correlations) of lower value must be eliminated.

The next step of the analysis was the evaluation of the discriminant validity (DV) of the structural equation model, which indicates if the constructs or latent variables are independent of each other [24]. There are two ways to check this question: (a) look at cross loading—indicators with higher factor loadings in their respective LV (or constructs) than in others; and (b) the criterion of Fornell and Larcker: the square roots of the AVE value of each construct is compared with the correlations (of Pearson) between the constructs (or latent variables). The square roots of the AVEs must be greater than the correlations between the constructs.

Table 6. Discriminating validity by Fornell and Larcker criteria.

	Management Effectiveness	Focus on the Student and Society	Competency Management	Leadership	Strategic Planning	Work Processes	Transparency
Management effectiveness	0.751
Focus on the student and society	0.682	0.708
Competency management	0.653	0.521	0.791
Leadership	0.584	0.562	0.465	0.710
Strategic planning	0.633	0.752	0.529	0.601	0.731
Work processes	0.708	0.630	0.559	0.738	0.651	0.725
Transparency	0.732	0.698	0.566	0.688	0.697	0.802	0.797

Source: own elaboration.

shows the results obtained from the appropriate model. Table 6 shows that the square root of the AVEs of the latent variables is superior to the correlations between them. So, it is concluded that the adjusted model has discriminant validity at the level of latent variables.

Analyzing the results of the calculations of cross-loadings, according to Chin’s criterion, it was noticed that the factor loadings of the observed variables (Vos) in the original constructs (VLs) are always higher than in others. This means that the model has discriminant validity by the criterion of Chin.

For the analysis of discriminant validity, a correlation matrix was prepared by using the Heterotrait-Monotrait ratio criterion, the result is shown in

Table 7. Discriminant validity by the Heterotrait-Monotrait ratio criterion.

	Management Effectiveness	Focus on the Student and Society	Competency Management	Leader- Ship	Strategic Planning	Work Processes
Focus on the student and society	0.812
Competency management	0.783	0.699
Leadership	0.685	0.683	0.633
Strategic planning	0.767	0.991	0.738	0.761
Work processes	0.834	0.787	0.748	0.948	0.869
Transparency	0.804	0.840	0.695	0.812	0.857	0.945

Source: own elaboration.

.

4.5. Structural Equation Model analysis

With the guarantee of discriminant validity, the adjustments are finished and now it starts the analysis of the structural equation model. The first analysis of this second moment is the evaluation of Pearson’s coefficients of determination (R²). The purpose of calculating the coefficient of determination is to evaluate the portion of the variance of endogenous variables, which is explained by the structural model. In other words, it indicates the quality of the fitted model. For the area of social and behavioral sciences, Hair et al. (2014) suggest that an R² equal to 2% is classified as a small effect, an R2 equal to 13% as a medium effect, and an R² equal to 26% as a large effect. In this research, as shown in

Table 8. R-squared and adjusted R-squared.

	R Squared	Adjusted R-Squared
Management effectiveness	0.553	0.545
Strategic planning	0.612	0.605
Work processes	0.659	0.653

Source: own elaboration.

, large effects were found for the constructs of management effectiveness, strategic planning, and work processes. This management effectiveness in universities contributes to local development [25].

Continuing, as this research deals with correlations and linear regressions, we must evaluate whether these relationships are significant (p-value less than or equal to 0.05), since for the cases of correlation it is established the null hypothesis (H0) as r = 0 and for regression cases, it is established with H0: Γ = 0 (path coefficient = 0). If p > 0.05, the H0 is not rejected and the inclusion of VLs or VOs in the structural equation model must be reconsidered. With the support of the SmartPLS software, Student’s t-tests were calculated between the original data values and those obtained by the resampling technique, for each VO—VL correlation relationship and for each VL—VL relationship. SmartPLS displays the t-test values and not the p-values. Thus, it should be interpreted that for high degrees of freedom, values above 1.96 correspond to p-values ≤ 0.05 (between −1.96 and +1.96 corresponds to a probability of 95% and outside this range of 5%, in a normal distribution). To test the significance of the relationships mentioned, “Bootstrapping” (resampling technique) was applied. Summarily, bootstrapping is used to obtain the p-values of the correlations between the VL and the factor loadings. It is a non-parametric procedure that allows testing the statistical significance of various results from PLS-SEM models, such as path coefficients, Cronbach’s alpha values, HTMT and R².

4.6. Bootstrapping

In this research, a complete bootstrapping was performed, with a two-tailed test, with corrected and accelerated bias. (bias-corrected and accelerated bootstrap—BCa). In bootstrapping, subsamples are created with observations taken randomly (with replacement) from the original data. To ensure the consistency of results, the number of subsamples must be large. To obtain the final results, a large number of subsamples (e.g., 5000) must be used. For this research, bootstrapping was applied with 10,000 subsamples.

From bootstrapping with 10,000 repetitions, p-values were estimated (see

Table 9. Structural model results.

	Hypothesis	VIF	F2	Coef. Structural	Standard Deviation	t-Value	p-Value	Adjusted R2
Leadership → Strategic Planning	H1(+)	1.461	0.121	0.262	0.063	4.122	0.000	0.605
Strategic planning → Management effectiveness	H2(+)	1.734	0.115	0.299	0.091	3.294	0.001	0.545
Focus on the student and society → Strategic planning	H3(+)	1.461	0.645	0.605	0.061	9.863	0.000	0.605
Transparency → Work processes	H4(+)	1.471	1.101	0.714	0.053	13.57	0.000	0.653
Management by competences → Work processes	H5(+)	1.471	0.048	0.156	0.071	2.192	0.028	0.653
Work processes → Management effectiveness	H6(+)	1.734	0.339	0.513	0.085	6.035	0.000	0.545

Source: own elaboration.

). The highest value of VIF (variance inflation factor) was equal to 1.734, corresponding to Hypothesis 2 of this research. As all VIF values were below 5, it is concluded that the model does not present multicollinearity problems. If the VIF values were greater than 5, the literature would recommend considering the exclusion of predictors or grouping them into VL of second-order [24].

The f² column in Table 9 corresponds to the results for each research hypothesis concerning the size of the Cohen effect. The effect size was evaluated according to Cohen’s criterion: f² = 0.02 = small; f² = 0.15 = average; f² = 0.35 = large. It was concluded that only hypothesis H5 had a small effect size. Hypotheses H1 and H2 had a medium effect size. And hypotheses H3, H4, and H6 had a large effect size.

The structural coefficients presented in Table 9 are the regression betas. From the R-square analysis, it was possible to evaluate the explained variance of the endogenous variables. According to Hair et al. [24]: R² = 2% = small; R² = 13% = average; R² = 26% = large. Thus, it was concluded that all hypotheses presented a large adjusted R-squared.

4.7. Blindfolding

Then, the blindfolding procedure was applied. Blindfolding is a sample reuse technique that allows calculating the value of Stone-Geisser’s Q², which represents a criterion for cross-evaluation of the predictive relevance of the PLS path model. The systematic procedure of eliminating data points and predicting blindfolding depends on the omission distance (D). Academic literature [24] recommends selecting a value for D when performing a blindfolding procedure, with a suggested value for the omission distance (D) between 5 and 12. For this research, the omission distance D equals 7.

An omission distance of 7 implies that every seventh data point of the target construct indicator is eliminated in each iteration of the blindfolding process. Since the blindfolding procedure has to omit and predict all the points of the indicators used in the measurement model of a given latent data variable, it consists of seven blindfolding iterations. The number of iterations is always equal to the omission distance (D).

The goal of the blindfolding procedure (

Table 10. Blindfolding.

	SSO	SSE	Q2 (=1 − SSE/SSO)
Management effectiveness	1053.00	764.96	0.274
Focus on the student and society	585.00	585.00
Competency management	351.00	351.00
Leadership	702.00	702.00
Strategic planning	468.00	328.00	0.299
Work processes	585.00	399.55	0.317
Transparency	819.00	819.00

Source: own elaboration.

) is to use all observations for prediction and as consequence to exclude complete observations in each blindfolding iteration. For this reason, the number of observations used in the original dataset divided by the skip distance (D) should not be a whole number.

Relevance or predictive validity (Q²), also known as the Stone-Geisser indicator, assesses how close the model is to what was expected of it. It would be the equivalent of an analysis of the quality of the model prediction or the adjusted model’s accuracy. As an evaluation criterion for the value of Q², values greater than zero must be obtained. A perfect model would have Q² = 1, and this would show that the model fully reflects reality, without errors [24]. In this research, Q² values of 0.274, 0.299, and 0.317 were obtained. All these values are greater than zero, and demonstrate that the model is close to what was expected of it.

This research presented a structural equation model that explicitly tests the relationships between leadership [1,2,15,16,17] and student focus and society [14,16,17,18] in strategic planning, and the relationship between strategic planning and management effectiveness [17]. In addition, this model that was presented analyzed the causal relationship between transparency [11,13,19] and management by competencies [11,13,19,20] in work processes, as well as the relationship between work processes [2,16,17] and management effectiveness. All coefficients found for the constructs showed a positive relationship. That is, all coefficients in the model have a positive impact. Thus, this research presents a synthesis of pillars to be observed in university management, in order to improve effectiveness and sustainability in higher education management. In addition, the model shows which of these variables most impact the effectiveness of university management [17].

It was observed how much the focus on the student and society [14,16,17,18] impacts on strategic planning, bringing to the manager the information that it is necessary to know what the student wants to better outline the work plan. The indicator, management by competences [11,13,19,20], also stood out, having a strong impact on work processes [2,16,17]. The manager then has the information that allocating people according to their skills, bringing together those who best complement each other to achieve the objectives proposed by the planning, is relevant for the good performance of the work routine. The work process is strategic planning in action [17]. It is up to the manager to have this vision of getting to know the student and society that are impacted by the university, as well as getting to know its servants, so that they, in action, can meet the planning elaborated by the institution. It is interesting to note the importance of the work process linked to strategic planning, as previously discussed by authors [2,16,17].

As demonstrated by the observed variables, through the applied questions, it was possible to observe that the responding servers believe they have a good performance and perceive quality when it depends solely on their work [16,17]; unlike when it is something that depends on the team or leadership. This is an important point to be worked on by the manager, as these are issues that can be resolved with training, resizing the team, improving communication, such as more frequent meetings and encouraging team conversations. This point is important to be observed, as it involves one of the indicators related to the new public management, originally addressed in this thesis: management by competencies. According to the literature [11,13,19,20], competence is realized in action, and the professional is not competent alone. It becomes necessary to know with whom and with what a person is competent. So, it would be this idea of allocating people, according to their competences, their skills, and developing them individually, to then achieve the development of the organization.

The survey data also show that there is a perception that the information collected is not analyzed and used to improve management. In fact, it is not enough just to collect information and make it available on transparency portals, one must seek to use this data to aggregate when making decisions. For example, not leaving them just to demonstrate how something was used, but using this information for future events. Transparency is not an end in itself, but an instrument to achieve management efficiency [16,17].

5. Final Considerations

This research aimed to analyze the determining factors of the effectiveness of higher education management at the Federal University of Santa Catarina. The results indicate that strategic planning and work processes are important for the effectiveness of the management of the Federal University of Santa Catarina. According to the model analyzed, constructs leadership and focus on the student and society are relevant for the determination of effective strategic planning; both showed good correlations, causality, and significant p-values. The ways of applying the strategy to achieve goals are essential for the effectiveness of management.

The constructs of leadership, focus on the students and society, transparency, and management by competencies, presented in this research an indirect impact on the effectiveness of management at the Federal University of Santa Catarina. The constructs of leadership and focus on the student and society had a positive influence on strategic planning, which had a positive and significant influence on the effectiveness of management. Transparency and management by competencies constructs had a positive and statistically significant influence on the work processes construct, which had a positive influence on the effectiveness of management. Therefore, obtaining results with these constructs allows the creation of new research agendas on the theme of management effectiveness in higher education institutions.

In this research, it was also shown that leadership, as well as the perception of focus on the student and society, are important factors and must be considered in the elaboration of strategic planning of a higher education institution. In conclusion, as a strategy to be put into action and contemplated in educational operations, the transparency construct has an important role in capturing, organizing, and supporting decision-making.

This research contributes to the practical and academic perspective, based on the observation that strategic planning, as a decision-making tool for the organization, and focus on processes, as a distinct planning action that puts strategy into practice, are the elements most representative of the effectiveness of management at the Federal University of Santa Catarina. A direct influence of these two constructs was found in this research. Specifically, concerning the theory, where there is progress in the Baldrige Excellence Model.

Furthermore, from a methodological perspective, it is indicated that the Baldrige Excellence Model should be contextualized to the attributes of Brazilian public higher education institutions. From a pragmatic point of view, this research contributes to the management of higher education institutions and institutional evaluation, because it demonstrates from the strategic perspective with a focus on results that the activities of the administration must focus on interested parties, external or internal, to provide an economic return for the shareholders, as educational operations, retention, and student motivation are associated with the qualification and training of the administrative and teaching staff of higher education institutions.

The main limitation of the study is the fact of testing a new model of structural equation modeling. This aspect, which can be, at first, understood as a limitation, signals to other researchers the possibility and the need to look for alternative paths, without, nevertheless, losing the methodological consistency. Therefore, the limitation is reported as a potentiality to be considered in scientific investigations. In such a manner, the research shows results relevant to the context of Brazilian higher education institutions, as it allows analysis derivative from the literature on the Baldrige and effectiveness criteria of excellence, which contribute to the management of the higher education institution, in addition to meeting the obligations for education that are provided for by law in Brazil.

As a suggestion for future research, it is believed that the study can be performed in other public higher education institutions, from other regions, to investigate different regional cultures and perceptions, including in public higher education institutions.

Another suggestion is to apply it in private higher education institutions, and later compare the results, identifying similarities and differences. It is also suggested the presence of an alternative model to the theoretical model, which could show how organizational effectiveness is seen from the viewpoint of the public and private sectors, which would permit the comparison of the results obtained through the Baldrige Excellence Model between public and private higher education institutions. It is also suggested the application of the contextualized model with companies to analyze how the graduates of higher education institutions contribute to the results of the organizations, because they absorb the workforce formed by the higher education institutions. Thus, it would be possible to measure the effectiveness of management on the perception of other actors in the area of higher education.

It is possible to indicate, in the model proposed by this research, that the hypotheses of this research directly influence the effectiveness of management at the Federal University of Santa Catarina. The execution of this research created unprecedented data and deepened the theory of effectiveness and criteria for excellence in university management, based on a robust multivariate statistical method. The proposed model, generated from structural equation modeling, can help the management of higher education institutions verify the constructs of the excellence model that have more impact on the effectiveness of management.