PBL Impact on Learning Outcomes in Computer Engineering: A 12-Year Analysis

Abstract

This article describes an evaluation of the Project-Based Learning (PBL) methodology implemented in the STEAM School of Universidad Europea in the Computer Engineering degree. The study raises research questions related to the perception of technical and soft skills acquired by students who used or considered PBL as their primary learning methodology compared to those who did not. Students’ motivation and adaptability to work after graduation have also been examined. The sample of students includes graduates from the last 20 years and therefore analyzes both the period in which the methodology was already implemented (from 2012) and some previous years. The study concludes that students who have identified or experienced PBL as their main learning methodology perceive a better acquisition of technical competencies and some soft skills, as well as better motivation and adaptability to the work environment.

1. Introduction

In this section the Project-based-learning methodology implementation at the School is described, and the analysis for the present research contextualized.

1.1. Why PBL 12 Years Ago?

Even before the implementation of the European Higher Education Area (EHEA), early analyses recognized the need for career guidance programs to ease students’ transition to the labor market [1] and looked for ways to increase students’ motivation with their future professions [2]. To pursue both objectives, it was necessary to find a learning methodology that could develop the professional competencies of graduates and provide enough motivation during the process to reduce dropout rates [3].

In Europe, the EHEA forced a redefinition of the competencies to be developed during engineering studies, identifying the learning outcomes that every graduate should achieve. These results were described in terms of their depth using Bloom’s taxonomy [4], later revised to include assessment in addition to learning [5]

In addition, the experiential learning models described by [6] laid the groundwork for a new teaching approach called Project-Based Learning (PBL) [7]. Since then, PBL as a teaching methodology [8] has been widely adopted by many middle and higher education institutions. Some, such as Aalborg University, were among the pioneers in using it in their classrooms [9] and have continued to do so with the necessary evolutions [10]. From the vanguard of the PBL approach, it is promoted that project-based education is inherently multidisciplinary and produces better results than other methodologies thanks to the active participation of students.

At the STEAM School of the European University of Madrid (UEM), experiential learning has been a means to develop a more meaningful learning experience for our students, which meets the requirements of fostering deep learning and motivation. Research, innovation, and practical implementation of new teaching and learning methods using PBL are evident in numerous publications over the years [11,12] and even the impact of PBL on student behavior, especially their motivation, has been analyzed [13]

PBL has been applied and assessed in various fields, subjects, and curricula. In [14], the authors propose PBL to approach problems, relating it to the design thinking method. In [15], PBL is suggested as a method for learning “computational thinking”, a structured and methodical way of approaching complex problem solving.

Hence, in 2012 at the Universidad Europea de Madrid (UEM), after several years of implementation of the EHEA and considering the good results that PBL had given in other institutions, it was decided to start some PBL experiences in certain degrees, including Computer Engineering [16].

1.2. PBL History in the School and in the Degree

This implementation of PBL at the UEM aimed to enhance motivation; deepen learning competencies required for an engineering profession, both specific competencies and soft skills; and establish a link between the classroom and the real engineering world.

Over the past twelve years, the implementation of PBL in the school has gone through three main phases: an initial phase, where projects were introduced as learning activities spanning multiple subjects; a second phase, where specific project-related subjects were introduced in some curricula; and a final phase, aimed at reinforcing the methodology by systematizing and ensuring its effective use.

1.2.1. Phase 1: Let’s Use Projects

The first phase has already been described in detail by some researchers at the university [17]. That article explained how each academic year, students participated in comprehensive projects that covered parts of the content of multiple subjects, with several professors contributing to each project. The integrative projects were designed exclusively with subjects from the same year (horizontal integration) but could belong to different terms. The results of this phase were quite promising, although not without some unwanted effects.

Both teachers and students experienced an increase in motivation. Students perceived project integration as a valuable tool to foster deep and lasting knowledge of specific skills. The opportunity to participate in projects with external companies was highly appreciated by the students, further increasing satisfaction with their professors.

However, there was an excessive specialization of the members of the working group within the projects, which was not conducive to the acquisition of the learning outcomes by all students equally. It was also noted that good results depended mostly on the commitment of teachers, and that the use of the methodology varied significantly between different majors, as it was not a mandatory requirement in the design of the curriculum.

1.2.2. Phase 2: Project-Centered Subjects

As a result of the initial analysis, run in 2015 and published in [17], the STEAM School decided to update the curricula to include specific project-related subjects. For example, in the Computer Engineering degree, five subjects titled “Project in…” were added, allowing students to apply the knowledge acquired in other subjects within the same year on a project tailored to their level.

In the first year, students encountered a subject called “Engineering Project”, where they put into practice knowledge based on competencies in areas such as mathematics, physics, programming, and business. In the second year, computer engineering skills, including advanced programming, software engineering, databases, and microcontrollers, were developed in a “Computer Science Project” split into two subjects (one in each term). The following year specialized in computing through a “Computing Project”, where advanced database knowledge, artificial intelligence, and user interfaces were put into practice. In the fourth and final year, no PBL projects were carried out to allow students time to work on their mandatory final thesis, a requirement for all undergraduate programs in Spain.

1.2.3. Phase 3: PBL 2.0

In addition, in 2021/2022, two new initiatives were launched focused on evaluating the implementation of PBL in schools. On the one hand, a consulting company specialized in the implementation of PBL in educational institutions was hired to evaluate our School. Simultaneously, an external professor specializing in neuroscience and active teaching methods was invited to join our faculty for a month. Both the consultant and the teacher had the opportunity to review documentation, attend our classes, and interview students, faculty, and school board members.

This experience revealed that teachers had significantly different concepts of what an academic project was in the context of PBL, cooperative methodologies in the classroom could be improved, and implementation was uneven between undergraduate, graduate, and online degree programs. After a decade of implementation of PBL in the School, it became evident that improvements are needed mainly related to the systematization of certain processes. Among these, the process of defining the project, the evaluation of the students and the evaluation of the PBL implementation process. All of them were already considered areas for improvement by other authors, such as [14], who identified “transfer”, understood as “knowing how to apply what has been learned in one context to new contexts”, as an unresolved issue. Other experiences identified other critical success factors, such as the gradual introduction of PBL from the early years [18], as well as a comprehensive list of questions and answers, after implementation and implementation in three different Spanish universities [19].

In the 2022–2023 academic year, the need to understand what was really happening in the classrooms was addressed, pursuing three main objectives: (i) to establish a common vocabulary for all teachers on the essence and implementation of PBL in the School, (ii) to disseminate this shared vocabulary among all teachers, and (iii) to provide teachers with methodological tools that they could apply effectively in their classrooms.

1.3. Justification for the Analysis

Even though many references talk about a positive impact of PBL in students learning outcomes [11,13], there are no real studies that analyze how much PBL contributes to a better acquisition of specific competencies in Computer Engineering studies. Even more known effects of the application of project-based methodology, such as the increasing of motivation or a deep development of soft skills have not been study in the computer’s higher education programs. A review to identify some gaps and future research were developed in [20], but not from the perspective of the impact on technical and soft skills for computing engineers.

Some previous analysis [11,12] focused on existing literature regarding the factors that influence the effectiveness of PBL (both knowledge and thinking skills) but none of them are specifically designed to computer engineering discipline.

The primary goal of this study is to contribute to the assessment of the outcomes of implementing the PBL methodology in the Computer Engineering degree at the UEM over the past twelve years. The study aims to address multiple critical questions, including:

• Depth of technical competency acquisition: Does the application of PBL genuinely lead to a deeper acquisition of technical competencies?
• Development of soft skills: To what extent are soft skills developed through PBL?
• Enhanced student motivation: Does PBL increase student motivation for learning, contributing to the goal of comprehensive student education?

To answer these questions, we plan to gather student feedback and opinions. We want to ascertain whether the following aspects are achieved, according to the perceptions of the students:

• Enhanced student motivation: Does student motivation improve in PBL-based courses?
• Deeper learning of technical competencies: Do students perceive a deeper understanding of technical competencies?
• Awareness of soft skills development: Are students aware of the development of soft skills?
• Transition to the Professional Environment: Do our students adapt more easily to the professional environment, and has the time required for on-the-job training been reduced?

By addressing these questions and collecting data from student perceptions, the study seeks to provide valuable insights into the impact of PBL on technical competence acquisition, soft skills development, student motivation, and overall student education. The findings will help design future curricula and identify teaching methodologies to improve the learning experience and outcomes for Computer Engineering students.

2. Materials and Methods

This study is presented within the framework of the University’s current PBL 2.0 project, and its purpose is to evaluate the process of PBL implementation during the last decade. The research is specifically aimed at knowing the perception of the graduates of one of the degrees of the School about the effect that the teaching method used during their studies has had on their learning process and subsequent professional performance. To this end, feedback has been collected from graduates of the University’s degree in computer engineering through a questionnaire related to their perception of the professional skills and competencies achieved. The results of the questionnaire are used to validate the objectives pursued with the methodology.

In addition, the data collected during the last decade have been analyzed, as part of the University’s quality assurance processes. Fundamentally in terms of academic outcomes: pass rate, failure rate, success rate, and satisfaction with learning. The objective of this additional analysis is to refute or confirm some of the results obtained with the graduate perception questionnaire.

2.1. Research Design and Selected Sample

The population considered for the analysis was that of all graduates who have graduated from the degree in Computer Engineering at UEM during the last 20 years. The study carried out with these graduates consisted of a quasi-experimental research with a post-test design with which to compare the results obtained in an experimental group (PBL group), those graduates who used the PBL model in their studies, and a non-equivalent control group, which has used traditional learning methods. For the study, it is assumed that the PBL pedagogical method is the independent variable, and that the acquisition of technical and transversal skills and other aspects related to motivation, professional adaptation or workload are the dependent variables in this study.

The sampling was incidental and unintentional since it is a time range that allows evidence to be obtained from the experimental and control groups. This population also includes various teaching formats, in face-to-face (F2F), online and hybrid modalities.

To carry out the research, a questionnaire was sent by e-mail to more than 600 graduates in Computer Engineering from the UEM, of which 114 graduates responded (

Table 1. Proportion of the sample to the total number of graduates, and distribution by learning modalities.

Group	Sample	% of Total Sample	Modality
			F2F	Virtual
Control Group (PBL Group)	64	56%	39	25
Control Group (PBL2012 Group)	79	69%	47	32
Total	114	19.5% of total alumni

). The level of response obtained is generally considered adequate in terms of size and representativeness, representing 19.5% of the total number of postgraduate students distributed in the different strata of interest for research.

56% of the responses correspond to students who identified PBL as the methodology used by their teachers (PBL group), while 35% did not identify PBL as the main methodology, so they considered that they studied according to a traditional model. This means that even though PBL was already the main methodology used, some of the students had the perception that they had not used it, so it was considered interesting to treat them as a different group from the sample. In addition, another different group was considered to be those who began their studies after the implementation of the PBL methodology in the School in 2012. In this case, the percentage of participants in this group rises to 67% (regardless of whether they identify PBL as the methodology used or not). That is why some results are also presented using this group as an experimental group (PBL2012 group), since it is of interest to compare the three groups: those who started to study before 2012 (No PBL running at all), those who started to study after 2012 that identify PBL as the main methodology (PBL Group) and those who started to study after 2012 regardless if they identify or not the methodology (PBL2012 Group).

By age, 22% are under 25 years old, 38% between 26 and 35 years old, and 40% are over 35 years old. The gender distribution is well in line with the current situation of the degree, with an 86/14 male/female graduate split. Most students (68%) have completed their studies in a face-to-face modality; the remaining proportion have taken them online or in a hybrid blended format.

2.2. Questionnaire Design

The questionnaire was designed to assess the degree of acquisition during studies of some soft skills and technical competencies established in the degree. These competencies have been selected from among those indicated in the current legislation that regulates the profession of Computer Engineer and are representative examples of the competencies that a professional must acquire.

The questionnaire questions were pre-tested with a small group of the sample who recommended some fine adjustments to the way the topics were presented to make them more self-explanatory. The results obtained in this validation phase were implemented in the final tool.

In the case of technical competencies, three have been selected by been considered particularly relevant: (i) ability to lead the activities that are the object of the projects in the field of Informatics; (ii) ability to define, evaluate, and select hardware and software platforms for the development and execution of computer systems, services, and applications; and (iii) ability to conceive and develop centralized or distributed computer systems or architectures integrating hardware, software, and networks.

In the case of soft skills, seven have been selected related to skills required in the professional environment: (i) teamwork; (ii) strategic communication; (iii) autonomous learning; (iv) resilience; (v) creativity; (vi) ethical-social competence; (vii) digital competence.

For each of these competencies, an evaluation rubric has been developed where the respondent is asked to identify the level of achievement of the competency that best matches their situation, according to a scale of five levels based on Bloom’s taxonomy. Thus, the graduate is asked whether, because of the studies, he or she can (i) remember, (ii) remember and understand, (iii) apply if told how to do it, (iv) analyze and evaluate situations to decide how to apply autonomously, or (v) create novel solutions to the problem (Figure 1).

In each of the rubrics, the graduate is asked about some items related to the competence, the details of which can be consulted in Appendix A.

The questionnaire also includes other questions related to the role of the teaching method in motivation, the workload derived, the ease of adapting to the activity and dynamics of the professional environment, and the level of autonomy at the beginning of the professional activity. These questions are answered using a 5-item Likert scale (in the low-high range), and additional comments or ratings are also solicited through open-ended questions.

Finally, the respondent is asked for their opinion on the methodology used during the studies, for which they must list the three aspects that they consider best, the three worst aspects, and what are the changes that they consider should be made. In order to identify the personal profiles of the respondents, the questionnaire collects information on the year of the start of the studies, current age, gender and teaching modality used in the studies.

2.3. Analysis Method of the Results

The results obtained in the questionnaire have been analyzed using a combined quantitative-qualitative methodology. The quantitative study consisted of descriptive (mean) and correlational (Pearson’s chi-squared test) analysis of the sample. To obtain the mean values, the levels of Bloom’s taxonomy have been converted into an ordinal numerical scale ranging from 1 to 5. In this scale, 1 represents the level of remember, and 5 represents the creative level. For chi-square tests, it has been considered that there is a significant dependence between variables when p < 0.05.

As mentioned, in addition to the data collected through the questionnaire, other academic results have also been quantitatively analyzed, as well as various quality indicators associated with the degree in the last ten years (those of PBL implementation). This analysis is therefore aimed at validating or refuting some of the results obtained from the questionnaire, with the aim of demonstrating the real impact of the PBL methodology in relation to two of the main objectives of its implementation: to motivate students, develop soft-skills, obtain a deeper knowledge of technical skills, and bring students closer to the work environment.

On the other hand, the qualitative study of the data has consisted of a content analysis using coding techniques and interpretative analysis [21].

The fields of open answer associated with the questions considered key have been analyzed: motivation, workload, professional adaptation, professional autonomy. In addition, the questionnaire closes with two free fields to indicate the main positive and negative aspects. A qualitative analysis has been applied to all these fields following the indications in [22].

The process followed has 3 phases: (i) initial reading (we proceed to the reading of all the literals obtained underlining the highlighted parts); (ii) categorization (categories and subcategories are associated with each of these literals); and (iii) connection (the results are associated with the context, in our case with the teaching methodology that is previously identified in the questionnaire). These steps have been carried out for each of the free text fields used in the questionnaire.

3. Results

3.1. Quantitative Analysis of the Questionnaire Responses

Based on the data collected with the self-perception questionnaire. The analysis was carried out using two data sets.

• First, the perception of the level of acquisition of competencies based on the learning methodology that each participant identified as the most used during their studies. This first analysis aimed to answer the following research question: did students who identify PBL as their fundamental learning methodology perceive a higher level of competence acquisition?
• Second, the correlation between the year in which they began studying and their perception of the level of competence acquired was analyzed. This analysis is especially important because it is known that only those who started studying after 2012 did so with a PBL model, regardless of whether they perceived it that way or not. This second analysis, therefore, answers the following question: did students who studied with the PBL methodology already implanted perceive a higher level of competence acquisition?

Although both studies may seem very similar, both add value, since of the 79 students who started at the School from 2012 (i.e., “ PBL2012 group”), only 56 chose PBL as the fundamental learning methodology (“PBL group”), the rest opted for other methodologies such as Problem-Based Learning, lectures or the case method. These data establish certain differences between the perception that graduates have about the methodology they used and the one they actually used, which would deserve an interesting analysis in the future. In any case, it is especially relevant to carry out the analysis from both points of view to try to eliminate the effect that the erroneous perception of the methodology could have on the results.

In addition, the impact of the methodology on other variables such as overall satisfaction with their studies, motivation, workload, adaptation to their first job and professional autonomy once they have graduated has also been analyzed.

Finally, although other demographic variables such as modality, gender and age of graduates were also analyzed, no significant results were found in the comparison between men and women or between the different age ranges. While it is true that some results suggest that there may be some differences, the sample size for women is not large enough to be able to draw relevant conclusions.

The results obtained from the self-perception questionnaire in each of the two analyses are discussed below. These results identify only those aspects dependent on the methodology, both in terms of the perception of the methodology used (PBL group) and the year in which they studied (PBL2012 group). To this end, Pearson’s chi-square test has been used, accepting as dependent only those aspects with values of p < 0.05 as usually recommended.

On the other hand, data collected over the last 12 years have also been analyzed, referring to the academic results of the students and their satisfaction with those subjects directly linked to the application of the methodology. This satisfaction was measured while the subjects were being taken using Likert scale questionnaires from 1 to 5.

3.1.1. Did Students Who Identify PBL as Their Fundamental Learning Methodology Perceive a Higher Level of Competency Acquisition?

Of the total sample, 56% identified PBL as the main methodology used by their teachers (PBL group). When comparing the results of this group with the rest of the participants, some interesting results emerge that are detailed below:

In relation to technical skills, there is a significant correlation between the participants of the PBL group and their perception of the ability to lead projects in the field of computer science. Especially in the identification of the phases and activities of the projects (p = 0.006) and in the use of the resources needed during them (p = 0.000). In the same way, this group perceives itself to be more competent when it comes to analyzing and evaluating software languages and technologies than the rest of the groups (p = 0.000). It is noteworthy, in any case, that the average evaluation of self-perception at the level of acquisition of all technical skills is 3 tenths higher in the PBL group compared to the rest (3.8 vs. 3.5 out of 5).

Regarding soft skills, there was only significant dependence between the individuals in the PBL group and their ability to communicate strategically (p = 0.035), but not with any of the other competencies evaluated. However, once again, the average evaluation of their level of acquisition of all the soft skills evaluated is almost 2 tenths higher in those who belong to the PBL group (4.2 vs. 4.0 out of 5).

Finally, Pearson’s chi-square test reveals that there is a very large dependence between motivation and the perception of autonomy when joining the first job and the fact of identifying PBL as the learning methodology used (p = 0.011 and p = 0.003 respectively). It is also interesting that, in relation to the workload associated with the methodology, those belonging to the PBL group report, on average, a lower workload than the rest (3.56 vs. 3.59 out of 5). Although it is small, the difference contrasts with the belief that PBL implies a greater effort for the student. Both variables, motivation, and workload, could explain the higher overall satisfaction indicated by individuals in the PBL group compared to the rest (4.2 vs. 3.6 out of 5).

3.1.2. Did Students Who Had Already Studied with the PBL Methodology (from 2012) Perceive a Higher Level of Competence Acquisition?

Of the total sample, 67% indicate that they began their studies at the School in 2012 or later, that is, once the implementation of the methodology began. However, only 70% of them recognize PBL as the basic methodology used by their teachers, which is, in fact, an interesting result. In any case, comparing the results of all those who started in 2012 or later years (PBL2012 group) with the rest of the participants, the results differ slightly from those obtained with the first analysis, but point in the same direction:

In relation to technical skills, for example, the dependency between the level of acquisition of these skills and the fact of having started studies before or after 2012 disappears. However, the difference in the average evaluation of the self-perception of the level of acquisition of all technical competencies is greater, which is 4 tenths higher in relation to the management of the project phases (4.3 vs. 3.9 out of 5) or 6 tenths higher in terms of knowledge of project development methodologies (4.0 vs. 3.4 out of 5) in favor of the group that began their studies from 2012 onwards. The same situation is repeated when talking about the mastery of software development languages and methodologies (4.3 vs. 3.8 out of 5). And in general, the self-assessment of all technical aspects is 2 tenths higher in group PBL2012.

Regarding soft skills, the fundamental change when comparing the groups before and after 2012 is that there is a very high dependency that had not appeared in the first analysis among those in group PBL2012 and the perception of their own capacity for resilience (p = 0.003), in addition to maintaining dependence on the ability to communicate strategically in favor of those who belong to group PBL2012. In the same way as in the first analysis, again the average evaluation of their perception in the level of acquisition of all the soft skills evaluated is 2 tenths higher for students in group PBL2012 (4.2 vs. 4.0 out of 5).

Finally, Pearson’s chi-square test confirms the dependence that was already detected in the first analysis between the PBL2012 group and motivation during their studies (p = 0.001). This trend can also be observed if we analyze the average score obtained by the assessment item in both groups (4.1 vs. 3.3 out of 5). In relation to the workload associated with the methodology, the result is repeated, the PBL group perceived a lower workload and slightly increased the difference between the perception of both groups (3.57 vs. 3.63 out of 5). Likewise, those in the PBL2012 group continue to perceive greater overall satisfaction with their time at the university (4.0 vs. 3.4 out of 5).

Another interesting result is that, if we analyze the overall satisfaction with their time at the university, which they declare to have had after some time of graduating, we can see (Figure 2) that satisfaction has been increasing, with each cohort, as progress has been made in the implementation of PBL, going from a 3.4 in the early years to a 4.0 in the last years.

3.2. Quantitaive Analysis of Academic Results along the Last Decade

To carry out this last quantitative analysis, academic data relating to thirteen subjects of the bachelor’s degree in Computer Engineering have been analyzed. These thirteen have been selected because they are those that are directly related to the PBL methodology, according to the following criteria: a subject is related to PBL if it is a specific subject of work on a project or if it is one of the subjects, with a more traditional methodology, which provides the necessary knowledge to carry out the projects.

Table 2. Traditional and project-based subjects considered in the study.

	Traditional Subject	Project-Based Subject
First Year	Object Oriented Programming	Engineering Project
Second Year	Programming with Linear Structures	Computer Science Project I & II
	Data Bases
	Programming Advance Techniques
Third year	Artificial Intelligence	Computing Project I, II & III
	User Interfaces
	Intelligent Systems and Knowledge Representation

shows the subjects selected by year and how they are classified into two groups: traditional and project-based.

Regarding the information used, the analysis has focused on the evolution in the percentage of passes, fails, not presented and the success rate (students passed among those who take the exam). Student satisfaction has also been analyzed.

Based on the number of passes in the subjects under study, it can be seen (Figure 3) that the average percentage of passes in traditional subjects has been gradually increasing from 60–65% in the years prior to the implementation of PBL, to figures close to 85% in the 22–23 academic year.

It is important to note that the 19–20 and 20–21 academic years were affected by the global health crisis resulting from the COVID-19 pandemic and that this has undoubtedly had an impact that is difficult to measure, but that it seems to have particularly affected the more traditional subjects. As for the project subjects, it seems to have had less impact and the pass rate has been gradually increasing since the first year of implementation. However, there is some irregularity that could be due to the specific project chosen in each year.

As for the number of no-shows, a criterion that allows us to assess whether the objective of increasing motivation and reducing the dropout rate is achieved (Figure 4), the evolution has been very positive, both in traditional subjects and in project subjects. Dropout rates have dropped from 20–25% to rates between 5% and 10%. Once again, the impact of the pandemic and subsequent years is noteworthy, where the number of students who did not show up was slightly increased, but in no case reaching the levels of the first years.

Finally, when looking into the evolution of the success rate (Figure 5), defined as the number of credits passed compared to those presented, it can be seen that the rate increased in the first years and, with the arrival of project subjects, there was a slight decrease in both project and traditional subjects, which subsequently tended to converge at levels above 90% in the last years of the study.

In relation to student satisfaction with the methodologies used, no significant differences were found between project subjects and traditional subjects, maintaining a similar trend over the years (Figure 6).

Finally, when inquiring into the self-perception of what has been learned (Figure 7), it is observed that, although similar values have been maintained over the years in traditional subjects, project subjects were perceived in the early years as less useful for learning, but their assessment has been improving and their fluctuations are also equal to the more traditional ones. This change in trend could be due to a change in the focus of the subjects of the projects after the first years, carried out from the 20–21 academic year. This change was motivated by the need to focus more on the life cycle of the project, how to organize, how to plan or distribute the work, without neglecting the more technical part, but giving greater importance to the attitude of professionalism when facing the first-year project.

3.3. Qualitative Analysis Results

This section describes the process and the results derived from the qualitative analysis described in the methodology section of this article. The subsections align with the open-ended text questions included at the end of the survey, which were as follows:

• What was the best? Name the 3 most favorable aspects.
• What was the worst? Indicate the 3 most unfavorable aspects.

As mentioned in the previous section, the quantitative data point to a better acquisition of certain competencies among students who recognized PBL as the majority methodology used during their studies and among those who started from 2012, the first year of implementation of PBL in the School. The qualitative analysis presented here helps to identify the two most important aspects of PBL’s impact on students: motivation and professional adaptation, and also identifies which aspects were the best and worst valued in relation to the methodology used.

3.3.1. Motivation

The categories used were: “motivating”, “partially motivating”, “not motivating”, and “online”, referring to the teaching methodology used. The subcategories included “adaptation to the work environment”, “practical application”, “deep learning of technical skills”, “use of innovative technologies”, “self-learning”, “teamwork”, “dropout”, “dependence on teachers”, “utility of what was learned”, “research spirit”, “evaluation”, “content”, “project development”, “work-life balance”, “teacher guidance”, “transversal skills”, “critical thinking”, and “engineering methodology.” These subcategories aimed to capture the themes of all the survey items, and all of them were duplicated with a “no” prefix when expressing their absence.”

In Appendix B, some examples of the categorization performed with the obtained literals are presented. We preserved the original language to maintain the initial expression of the students.

In the connection phase, efforts were made to associate categories with the methodology most selected in their respective programs. The findings are presented in

Table 3. Categories linked with learning methodology for motivation.

	PBL	Other
Motivating	37	10
Partially Motivating	3	2
No motivating	1	15
Online	1	3
Blank	25	17
Total	67	47
% Motivating	55%	21%
% No motivating	1%	32%

, where the number in each cell denotes the frequency of occurrences for the category represented in the column with the identified methodology in the corresponding row. For instance, the methodology “Project-Based Learning” was categorized as “Motivating” 37 times. Blank entries occur when respondents either do not provide a literal or when the provided literal cannot be linked to an associated category.

55% of respondents identify the project-based learning methodology as motivating compared to 21% for other methodologies.

Table 4. Categories linked with learning methodology for motivation.

	PBL	Other
“Adaptation to the work environment”	8/0	1/1
“Practical application”	7/0	4/3
“Deep learning of technical skills”	7/0	0/1
“Use innovative technologies”	1/0	0/0
“Self-learning”	6/0	2/0
“Teamwork”	3/1	0/0
“Dropout “	0/0	1/0
“Dependence on teachers”	4/0	2/0
“Utility of what was learned “	2/0	1/0
“Research spirit”	2/0	0/2
“Evaluation”	0/0	0/1
“Content”	0/1	0/2
“Project Development”	2/0	0/0
“Conciliation”	1/0	2/0
“Teacher Guide”	0/1	0/3
“Transversal skills”	1/0	0/0
“Critical Thinking”	0/0	1/0
“Engineering Methodology”	0/0	0/3

displays the occurrences of each subcategory identified in the rows, corresponding to the methodology identified in the columns. Two values are presented for each cell, the first being positive and the second negative. For instance, the 7/0 value in the first cell indicates that on 7 occasions, the project-based learning methodology was identified as positive for adaptation to the work environment, with no instances of negative identification. The ‘conciliation’ row is shaded in grey because it can be inferred from the literals that respondents considered it an ‘online’ methodology not available for selection in the preceding questionnaire questions, and therefore, it does not contribute information to this analysis.

It can be observed that the most frequent subcategories for project-based learning are “adaptation to the work environment”, “practical application”, and “deep learning of technical skills.” The most common negative aspects of the methodology are “lack of teamwork”, “lack of content”, and “lack of teacher guidance.” In the first case, it is derived from the difficulty that the respondent considers teamwork to have. In the second case, it is mentioned in the verbatim “[…] they had more theoretical knowledge, but at a practical level, it made a difference”, referring to colleagues from other universities (understood to use different methodologies). In the last case, it refers to the excerpt “[…] It’s very easy to come to the classroom, sit down, and say ‘do this,’ without explanations or prior basic knowledge […]”, this feeling may be due to a poor implementation of the methodology by the teacher or because the students, accustomed to more guided and less active training activities, initially express rejection.

In the case of “others”, combining the responses from the methodologies of “lectures” and “problem-based learning”, the most frequent subcategories are “practical application”, “self-learning”, and “dependence on the teacher.” This indicates that the motivation felt during the teaching-learning process depended on the teacher’s ability. The most common negative subcategories are “lack of practical application” and “non-engineering methodology”, reflecting the idea that studying engineering degrees requires a more practical approach than what is achieved with other methodologies.

3.3.2. Adaptability to the Job

Table 5. Categories linked with learning methodology for professional adaptation.

	PBL	Other	Total
Easy	90%	47%	34
Medium	10%	13%	5
Difficult	0%	40%	6
Total	30	15	45

displays, in the first column, the categories considered for the open question about professional adaptation. Once again, it is evident that many respondents either did not provide a response or the obtained response did not allow classification into any of the categories (out of a total of 114 responses, only 48 could be categorized). Ninety percent of the responses that identified project-based learning believe that their adaptation to the professional environment was easy, compared to 47% for the rest of the methodologies.

Appendix C shows the subcategories and the number of occurrences for each of them for PBL and for other methodologies.

The highest values correspond to the subcategory ‘thanks to PBL’. All responses are from subjects who have studied with PBL, with 9 of them considering that the ease of their professional adaptation is due to the use of the PBL methodology. One of them explicitly states that their ease of adaptation was not due to PBL. Another interesting piece of information in the table is related to the subcategory ‘use of methodologies seen in university’. There are 6 students who used PBL methodologies, and they indicate that they applied methods and methodologies learned in university, which later facilitated their integration into the profession.

3.3.3. What Was the Best?

The categories mentioned as the “best thing” are displayed in Appendix D and are arranged in order based on the frequency of mentions by the surveyed group that identified PBL as a teaching methodology.

In the case of PBL, respondents mention that teamwork and professors were the best aspects, with 25 and 13 occurrences, respectively. They are followed by practical application, faculty proximity, simulation of reality, and professional adaptation, each with 8 occurrences. In third place, they indicate methodology, the use of current technologies, and content with 7, 6, and 5 occurrences, respectively.

It is worth noting that “methodology” is mentioned as the best aspect on 7 occasions and 0 in the rest of the options. Furthermore, in the case of lectures, it is mentioned once as a negative aspect.

“Practical application” and “self-learning” are the two aspects identified by the problem-based learning methodology group, while “faculty proximity” is identified by the lecture-based classes.

3.3.4. What Was the Worst?

Appendix E shows the categorization done for the responses where participants indicated what was the worst. It’s important to highlight that the initial categorization resulted in values that were too scattered, leading to the need for a second round:

• Workload. It is the most frequently mentioned category. Various aspects are referred to: (i) it is considered that at times the workload required for the project is not calculated accurately, (ii) self-imposed demands lead to a lack of time to deliver a project “up to standard”, (iii) project implementation requires more day-to-day effort, (iv) in some cases, teachers do not coordinate to establish delivery dates, (v) good initial planning is crucial, (vi) it is of particular importance during “peak moments”.
• Project definition. The identified aspects are: (i) it is crucial that the objectives are well-established, (ii) in some cases, the utilized project excludes certain subjects considered of interest, (iii) integrative projects within a single course should revolve around a unifying theme, (iv) they should be simulations of reality, (v) it is difficult to include theoretical foundations, (vi) when each group carries out a different project, it is perceived as disorganization, and finally, (vii) it would be a very good idea to include multidisciplinary by involving students from other fields.
• Teachers. In this case, the following points are highlighted: (i) lack of uniformity in the application of Problem-Based Learning (PBL), with cases where it is not applied at all, (ii) lack of support, direction, guidance, or explanation in project implementation, (iii) in some cases, a lack of subject matter expertise is identified, (iv) lack of coordination, and finally, (v) there is a need to enhance teacher training in PBL.
• Teamwork: In this case, there are multiple instances of team members not actively contributing, exhibiting variations in skill levels, motivation, maturity, or expectations. Additionally, there is an identified need for providing specialized training in teamwork for both students and instructors.
• Project Development: Involves (i) a lack of guidance from the instructor, (ii) additional challenges when projects are conducted remotely, (iii) difficulties arising when building upon products developed by other teams in previous subjects due to a lack of quality, (iii) the use of outdated technologies, and finally, (iv) a perceived lack of organization in some instances.

The group of students who did not use PBL precisely identified the methodology as the worst aspect. The arguments they put forth include: (i) classes are too theoretical, lacking practical application, (ii) they lack more hands-on labs and (iii) activities that are more closely related to professional reality. They use adjectives such as tedious, monotonous, or repetitive. They believe that, at times, many activities are used, which, in many cases, are not useful.

4. Discussion and Conclusions

This study shows that the Project-Based Learning methodology seems adequate to acquire certain technical competencies and soft skills, or at least this is how it is perceived by the graduates who have used it.

Although it is always uncertain when analyzing perceptions, it seems that there is a direct relationship between the use of the methodology and students’ perception of success and satisfaction in certain competencies. Some soft skills such as autonomous learning, strategic communication or resilience benefit greatly from the methodology. However, in other skills such as teamwork or decision-making, there does not seem to be a particularly positive impact of PBL.

In the case of technical competencies, the results indicate that those students who perceived PBL as their fundamental learning methodology do have a perception of having acquired a higher level of competence, especially in those competencies related to project management and planning. In the same way, students also admit to being more motivated when learning with the PBL methodology. Likewise, from the most objective point of view, dropout rates in subjects are lower when PBL is used compared to the use of other methodologies such as lectures.

Although a specific analysis would be necessary, it seems to be intuited that the better acquisition of technical and soft skills, and the increase in motivation or autonomy associated with PBL, may also be the most likely cause of the observed improvement in grades. Over the past twelve years, academic results, subject success rates, and achievement rates have improved. The decrease in dropout rates is very noticeable, as well as the improvement in grades, not only in the Projects subjects, but also in the more traditional subjects related to the Projects.

Regarding the students’ workload, it is observed that the relationship with the application of PBL is inverse and, contrary to what might be expected, the use of the method requires, according to the students’ perception, a lower workload. However, one of the main problems of the methodology, perceived by the students, is related to teamwork or the autonomy required to plan and seek solutions independently.

Finally, the use of PBL also improves other aspects related to incorporation into the labor market, such as the perception of autonomy in problem solving or the ability to easily adapt to new contexts.

Undoubtedly, all these results are still circumscribed to a specific implementation of the methodology, in this case that of the School, so this study opens the door to new research: in other degrees, with other student populations, with other implementations of PBL or, for example, focusing on other competencies and learning outcomes.