PBL Impact on Learning Outcomes in Computer Engineering: A 12-Year Analysis

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Please see the attached file for the comments.

Comments for author File: Comments.pdf

Author Response

Comments

1. The introduction effectively sets the stage for the study by contextualizing the implementation of Project-Based Learning (PBL) within the European Higher Education Area (EHEA) framework and the specific needs of the STEAM School at Universidad Europea. However, Authors could provide a more critical review of existing literature on PBL to highlight gaps this study aims to fill.

Response: A two new paragraphs have been included in section 1.3 to address the comment

2. The methodology section is detailed, describing the quasi-experimental design, sampling method, and the use of questionnaires for data collection. However, it could benefit more if Authors provide more detail on the development and validation of the questionnaire, including reliability and validity measures.

Response: No formal validation was carried out but an informal one is described to explain how the questionnaire was pre-tested.

3. Authors can provide a clearer connection between the quantitative and qualitative findings.

Response: A two new paragraphs have been included in section 3.2 to address the comment

4. I don’t quite understand the number “20” stated in the abstract: “The sample of students probes from 20 years of graduates.” Why 20 years?

Response: A better explanation is included in the abstract which has been rewritten in order to addressing other reviewer comment.

Reviewer 2 Report

Comments and Suggestions for Authors

The paper covers a very interesting and relevant topic, however, some improvements could contribute to the quality of the manuscript.

The introduction could benefit from a more extensive and current literature review in the field of PBL and learning outcomes.

Line 61 – “The first phase was described in [19].” Does this mean that the readers of this paper should refer to the methodology in another paper? Or is the further text referring to the reference 19? Clarification could contribute to the clarity of the text.

Lines 76-77 are very confusing, they say: “As a result of this initial analysis, in 2015, the STEAM School decided to update the 76 curricula to include specific project-related subjects.”, which analysis are the authors referring to? To reference 19 which is dated 2017 in the previous sub-chapter? Please, clarify in the text.

Subsection 1.3, lines 120-133 – “The study aims to address multiple critical questions, including: <..> Deeper learning of technical competencies: Do students perceive a deeper understanding of technical competencies?  Awareness of soft skills development: Are students aware of the development of soft skills?” it is confusing that the paper aims to measure the perception of the students, especially, since it does not mean that the learning outcomes were actually affected by the perceptions of students only (refer to the title of the paper).

From the presentation of the methodology, it is not clear that respondents are employed. Clarification is needed, especially to address this aim (lines 135-137): “Transition to the Professional Environment: Do our students adapt more easily to the professional environment, and has the time required for on-the-job training been reduced?”

Lines 176-187 could be revised as now it is not clear how the methodology was applied. For example, the authors state: "56% of the responses correspond to students who identify PBL as the methodology 176 used by their professors (PBL group), while 35% have previously studied according to a 177 traditional model." What does that mean? That the answers of the respondents do not provide the right information, i.e. 35% of the respondents who state that they were exposed to PBL methodology in fact were not? Or anything else? It would be great if the authors provided a more clear description.

Also, please, clarify that statement: “That is the reason why we present also some results using this group as experimental 181 group (PBL2012 group).” (lines 181-182)

Also, the initial questionnaire could be provided for clarity, which is especially related to the questions: “What was the best?” (line 513), and “What was the worst?” (line 528). It is not clear, what are the contexts for the questions.  Also, how did the authors analyse the answers when in both, the best and the worst options the same items were ranked highly (Appendix D and E): teamwork, teachers. More clarification could be provided.  

It is not clear how the adaptability to the job after students graduated was examined (refer to the abstract, lines 8-9.

The is no discussion in the discussion part, only conclusions.

Overall, the research is relevant and interesting and could contribute to the field.  

Comments on the Quality of English Language

The authors could contribute more to the level of formality of the language to make it more academic. Some are fragment sentences and therefore, underdeveloped, some features of the informal language are detected in the text. 

Author Response

The introduction could benefit from a more extensive and current literature review in the field of PBL and learning outcomes.

Response: The beginning of section 1.3 has been rewritten including some new references [13] and [18]

Line 61 – “The first phase was described in [19].” Does this mean that the readers of this paper should refer to the methodology in another paper? Or is the further text referring to the reference 19? Clarification could contribute to the clarity of the text.

Response: The section 1.2.1 has been rewritten explaining in a more detailed way the point.

Lines 76-77 are very confusing, they say: “As a result of this initial analysis, in 2015, the STEAM School decided to update the 76 curricula to include specific project-related subjects.”, which analysis are the authors referring to? To reference 19 which is dated 2017 in the previous sub-chapter? Please, clarify in the text.

Response: An brief explanation has been written but the dates are correct. By the end of 2015 an analysis were carried out but it was not published until 2017.

Subsection 1.3, lines 120-133 – “The study aims to address multiple critical questions, including: <..> Deeper learning of technical competencies: Do students perceive a deeper understanding of technical competencies?  Awareness of soft skills development: Are students aware of the development of soft skills?” it is confusing that the paper aims to measure the perception of the students, especially, since it does not mean that the learning outcomes were actually affected by the perceptions of students only (refer to the title of the paper).

Response: The results obtained from questionnaire are based always on alumni's perception about the acquisition of competencies. Learning outcomes are not affected by perception but what we are analyzing is whether the students perceive a better acquisition of competencies depending on the methodology they used. We have tried to explain better on the text.

From the presentation of the methodology, it is not clear that respondents are employed. Clarification is needed, especially to address this aim (lines 135-137): “Transition to the Professional Environment: Do our students adapt more easily to the professional environment, and has the time required for on-the-job training been reduced?”

Response: The way to refer to respondents has been changed to talk always about alumni or graduates to avoid misunderstandings.

Lines 176-187 could be revised as now it is not clear how the methodology was applied. For example, the authors state: "56% of the responses correspond to students who identify PBL as the methodology 176 used by their professors (PBL group), while 35% have previously studied according to a 177 traditional model." What does that mean? That the answers of the respondents do not provide the right information, i.e. 35% of the respondents who state that they were exposed to PBL methodology in fact were not? Or anything else? It would be great if the authors provided a more clear description.

Also, please, clarify that statement: “That is the reason why we present also some results using this group as experimental 181 group (PBL2012 group).” (lines 181-182)

Response: Part of the section 2.1 has been rewritten to explain better the point but let me explain here: There are three groups the we used: those who started to study before 2012 (No PBL running at all), those who started to study after 2012 that identify PBL as the main methodology (PBL Group) and those who started to study after 2012 regardless if they identify or not the methodology (PBL2012 Group).

Also, the initial questionnaire could be provided for clarity, which is especially related to the questions: “What was the best?” (line 513), and “What was the worst?” (line 528). It is not clear, what are the contexts for the questions.  Also, how did the authors analyse the answers when in both, the best and the worst options the same items were ranked highly (Appendix D and E): teamwork, teachers. More clarification could be provided.  

Response: Beginning of section 3.2. has been rewritten to explain better the context of these questions.

It is not clear how the adaptability to the job after students graduated was examined (refer to the abstract, lines 8-9.

Response: One of the rubric they answered were focus on adaptability to the job. It is explained after figure 1 example.

The is no discussion in the discussion part, only conclusions.

Response: This section has been rewritten to included some discussion.

The authors could contribute more to the level of formality of the language to make it more academic. Some are fragment sentences and therefore, underdeveloped, some features of the informal language are detected in the text. 

Response: The text has been reviewed to use a better and more formal English. My apologize for that

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I believe the Authors have addressed all of my concerns successfully.

One minor thing: the table on Page 6 should be revised properly.

Many thanks for this opportunity to review this interesting paper.

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript needs a revision of the language editor but could be accepted for publication.

Comments on the Quality of English Language

The manuscript needs a revision of the language editor but could be accepted for publication.