Analysis of Students’ Online Learning Engagement during the COVID-19 Pandemic: A Case Study of a SPOC-Based Geography Education Undergraduate Course
Round 1
Reviewer 1 Report
There is an abundance of related research on online learning for higher education students during COVID-19, including but not limited to online learning engagement. Given the lack of research significance of the current study and the many problems in the manuscript, this manuscript cannot be published in the Sustainability journal. Some suggestions are as follows:
1. The motivation and value of this research are not clearly stated in the introduction section. The content of the literature review did not elaborate on the current research status related to student online learning engagement during the COVID-19 pandemic.
2. Many statements in the article are not supported by previous studies.
3. The title of Section 7 is discussion, but the content of Section 7 is limitations and future work.
Author Response
Point 1: The motivation and value of this research are not clearly stated in the introduction section. The content of the literature review did not elaborate on the current research status related to student online learning engagement during the COVID-19 pandemic.
Response 1: Thank you for your advice and we are very sorry for our negligence of the research status related to student online engagement during the COVID-19 pandemic. In our revised version, we have added the motivation and value of our research. The motivation of our research is that some problems affecting the quality of online learning such as "low participation" and "low cognition level of learning" are still prevalent in online courses. Since the quality of learning largely depends on the learner's commitment to learning, the value of this research is that constructing an online learning engagement analysis model, and analyzing the relationship between online learning engagement and learning performance through a case-based study may find out which dimensions of learning engagement are the deficiencies of students and detail the reasons that affect online learning quality.
Point 2: Many statements in the article are not supported by previous studies.
Response 2: Thank you for pointing out the shortcomings, we have added some references to increase reliability of statements. Firstly, the phenomenon that the cognitive level of students do not reach the stage of application innovation also existed in studies of Biggs J and Dyer et al. Secondly, the study of Luan, L and Dong, Y et al. shows the positive influence of teacher support strategy in online education, same as this study. Thirdly, in this study, positive emotional engagement especially curiosity is helpful to improve students' learning. Moreover, Berlyne and Gruber et al. prove that curiosity can promote the memory effect of the material. Finally, to social engagement, the result confirms the conclusion of Blasco-Arcas et al., that interactive learning between teachers and students and between students can effectively improve students' learning performance.
Point 3: The title of Section 7 is discussion, but the content of Section 7 is limitations and future work.
Response 3: Thank you for pointing out the problem here. Section 6 and 7 are merged. The title of Section 6 is revised as “Discussion”.
Reviewer 2 Report
Thank you for the submission.
I think, in general, there is a good flow of the text and an appropriate use of references in the relevant sections.
However, I feel there are two major issues.
1: Link of the submission with the aim and scope of sustainability. I am not clear how the submission is linked to sustainability. The word itself appears once in the text. But where it appears it is not enough to clarify this link. Is this linked to sustainability from the point of view of use of resources? Is it because of face-to-face traveling could result in higher carbon print due to travelling. Is it because the course used as example covers topics of sustainability, hence raising awareness on the topic?
I would encourage you to clarify this in the text. Otherwise is not clear (to me as reviewer) why choosing this journal.
2. There are some areas that need further reviewing and rewording. This is particularly important in section three (research methods). Most of the subsections here seem to be linked to the literature review / introduction, rather than explaining what was done to collect data for this publication.]
Also, results section includes referenced sections that perhaps would better fit in discussion.
Additionally:
Some comments in format/editorial suggestions: Please bear in mind that the suggestions below are just some of the ones I found. Therefore, I strongly suggest you check the whole document.
Line 48: replace debated FOR debate
Line 92: replace SPOC FOR Small-scale private online course (SPOC)
Line 349: replace "3 questions in each dimension and 5 points" FOR "three questions in each dimension and five points"
Line 358: UNICET is just mentioned here in section results. However, this should be part of research methods.
Line 400: add space between "levels" and "[37,38"
References:
References 2, 3, 4, 5, 6, 7, 8, 9 and 26: please check format (use of italics for name of publication to keep consistency).
Author Response
Main comments
Point 1: Link of the submission with the aim and scope of sustainability. I am not clear how the submission is linked to sustainability. The word itself appears once in the text. But where it appears it is not enough to clarify this link. Is this linked to sustainability from the point of view of use of resources? Is it because of face-to-face traveling could result in higher carbon print due to travelling. Is it because the course used as example covers topics of sustainability, hence raising awareness on the topic?
I would encourage you to clarify this in the text. Otherwise is not clear (to me as reviewer) why choosing this journal.
Response 1: We appreciate it very much for this good suggestion. In our revised version, we have added Line 54-70. We sought to explore the positive effects of this study in promoting the sustainable advancement of online learning. In the meantime, as you mentioned in comments, the course used as example covers topics of sustainability, hence raising awareness on the topic.
“To address the gaps in the current literature and explore how online learning can be transited and sustained in various learning environment, the present study constructs a multidimensional online learning engagement analysis model to investigate student engagement in emergency online learning during the COVID-19 pandemic, and explores the relationship between students’ online learning engagement and their online learning performance by taking the Secondary School Geography Curriculum Standards and Textbooks Research, a small-scale private online course (SPOC) of the geography education undergraduate course as an example. The Secondary School Geography Curriculum Standards and Textbooks Research is an introductory geography education course for pre-service teachers. Geography’s inherently interdisciplinary nature is its strength in advancing sustainability education. This discipline can develop a more holistic understanding of global environmental challenges in that it encompasses all the sciences (including social sciences and humanities) and possesses realization of the sheer magnitude of human impact on the global environment. Therefore, geography education is a crucial tool for educating citizens of all ages about the complexity of human-nature interconnections, how to make more reasoned decisions for the planet, and what can (and should) be done to ensure a more sustainable future[10].”
Point 2: There are some areas that need further reviewing and rewording. This is particularly important in section three (research methods). Most of the subsections here seem to be linked to the literature review / introduction, rather than explaining what was done to collect data for this publication.
Also, results section includes referenced sections that perhaps would better fit in discussion.
Response 2: Thank you for pointing out the problem here. In the section four ‘Data Sources’, we explain in detail how the data used for the analysis in this study was collected.
In the third section we mainly want to discuss clearly how the analytical model as a research method is constructed. Therefore, in the section '3.1. Online Learning Engagement Model Dimensions', we illustrate what specific indicators are included in the four dimensions of online learning engagement in this study by linking them to the literature. In the section ' 3.2. Online Learning Engagement Model Construction', we explain the sources of data for each dimension and the methods to be used to analyze these data. Namely:
Cognitive Engagement Based on Content Analysis
Behavioral Engagement Based on Learning Platform Data Analysis
Emotional Engagement Based on Self-Report Analysis
Social Engagement Based on Social Networking Analysis
As for the results section includes referenced sections, in the revised version, we have already moved them to the discussion part (Line 388-393).
Minor Comments
Point 3: Please bear in mind that the suggestions below are just some of the ones I found. Therefore, I strongly suggest you check the whole document.
Response 3: Thank you much for your good suggestion. The language and structure of the manuscript have been thoroughly and carefully reviewed by all authors. We have revised the problem you mentioned. For other similar problems, we also modified and improved.
Point 4: Line 48: replace debated FOR debate
Response 4: Thanks for the reminder. We have now amended it based on your comments.
Point 5: Line 92: replace SPOC FOR Small-scale private online course (SPOC)
Response 5: Thank you for pointing out the problem. We have added the full name of SPOC in the revised version.
Point 6: Line 349: replace "3 questions in each dimension and 5 points" FOR "three questions in each dimension and five points"
Response 6: Thanks for the reminder. We have now amended it based on your comments.
Point 7: Line 358: UNICET is just mentioned here in section results. However, this should be part of research methods.
Response 7: We appreciate this advice. We have added “The four sub-dimensions of social engagement are analyzed using UCINET software.” in section Research Method.
Point 8: Line 400: add space between "levels" and "[37,38]"
Response 8: Thanks for the reminder. We have now amended it based on your comments.
Point 9: References:
References 2, 3, 4, 5, 6, 7, 8, 9 and 26: please check format (use of italics for name of publication to keep consistency).
Response 9: Thank you for pointing out the citation issues. We have made modifications to the citation section in the revised version.
Reviewer 3 Report
The reviewer appreciates the authors’ submission and found the research interesting and timely. The introduction, review of literature, and discussion of constructs were generally good and well-written. The conclusions relate back to existing literature. However, there are a series of concerns about the study that should be addressed.
Since this research is posited as a case study, more information needs to be provided about the course design, content, and format. If this paper is to be accepted into Sustainability due to environmental-based research, the authors should provide much more context about the course and its relationship to sustainability in some fashion. Furthermore, more specific detail about the posts are needed. For example, are the student postings (which data for three of the four variables are attained) required as part of the class? Were peer replies required? What were the prompts/topics for the discussion boards? Etc. This information is particularly important as it directly relates to your variables and the case study.
Several variables need more clarity.
Learning Performance. There are a few sentences describing learning performance, but they are underwhelming and unclear. It looks like learning performance was based on a total grade percentage scale of 0 to 100. More information is needed to clearly articulate how the grades were assigned. Were the “discussion boards” required and or/part of the overall grade? If so, does this impact the correlation analyses between some of the variables (e.g., number of posts and overall grade)? What other components accounted for the total grade?
Cognitive engagement:
Often learning objectives are designed to align with Bloom’s taxonomy. For example, there is typically an expectation for students to “create,” “apply,” “evaluate,” etc. I would like more explanation for how these were coded and examples of student responses would be highly appropriate for the reader to see.
Behavioral engagement:
The data here are unclear. Is this the average number of posts per discussion board or is this the total number of posts for the course? Was there an expectation of the number of posts required in the course or was this voluntary? The average number of posts is .2, and replies to others and peers’ replies are both at 0.05. Did students have any expectation to engage in the behavior of posting? If the instructor had no expectation for students to do so, then the blame can’t be put on students as indicated in the conclusions “The reason for this may be that online learning relies more on the autonomy of students, and students need to make more efforts to complete online learning than offline learning.” Even in the face-to-face classroom, if the instructor does not have the expectation for students to engage with each other on course content, and set them up to do so, then they typically will not.
Emotional engagement: This construct was good. However, please report scale reliability analysis for the sub-constructs.
Other comments:
Title: Undergraduate Program or Undergraduate Course? It appears this research analyzed a specific course and not a program.
May want to spell out SPOC when used in the abstract.
Line 56 – confusing typo? (face-to-face communication is lost?)
Line 63 – be more specific with (in the dozens). This could be 2 dozen or 10 dozen…
Line 66 – spell out MOOC when first used (massive open online course)
Line 157 is unclear to the reviewer. Please expand this to offer a more concrete explanation.
Figure 1. (fix “Pees” replies)
Mixed spellings of behavior.
Line 248 – what is meant by 12 posts (discussion boards with unique prompts) or 12 posts total (were they required?).
Lines 259 – 261 are unclear.
Lines 292-293. “The survey suggests that except for boredom, all students are assigned 100% in terms of curiosity pleasure, and sense of belonging.” The means reported do not indicate this from what I can see.
Line 338. “The data indicates that most students concentrate on “analyze” at the cognitive level, but failed to enter the “evaluate” and “create” stage” – were the students asked to evaluate or create in this discussion?
Some issues throughout with the format of within-text citations (e.g., Fredeicks, Li Shaung et al. …).
Author Response
Main comments
Point 1: If this paper is to be accepted into Sustainability due to environment-based research, the authors should provide much more context about the course and its relationship to sustainability in some fashion.
Response 1: Thank you for pointing out the shortcomings, we have added clarification on the relationship between the course and sustainability. Humans continue to push the earth closer to planetary boundaries so that we may face a series of environmental problems. Geography’s inherently interdisciplinary nature is its strength in contributing to sustainability education. This discipline can develop a more holistic understanding of global environmental challenges in that it reaches across all the sciences (including social sciences and humanities) and possesses realization of the sheer magnitude of human impact on the global environment. Geographical education therefore represents an important vehicle for citizens of all ages to help them understand the complexity of human-nature interconnections, how to make more reasoned decisions for the planet and what can (and should) be done to achieve a more sustainable future. In order to guarantee the quality of geography education, it is necessary to train initial geography teachers.
Point 2: Since this research is posited as a case study, more information needs to be provided about the course design, content, and format.
Response 2: We appreciate it very much for this good suggestion. In our revised version, we have added a range of course-related information such as the relationship between the course and sustainability, the composition of the course grade, and the topics in discussion area in the course.
Point 3: Furthermore, more specific detail about the posts are needed. For example, are the student postings (which data for three of the four variables are attained) required as part of the class? Were peer replies required? What were the prompts/topics for the discussion boards? Etc. This information is particularly important as it directly relates to your variables and the case study.
Response 3：Thank you for your advice and we are very sorry for our negligence of the specific detail about the posts. More details about posting have been added in the revised version (page9,line286).This course involves two types of discussion posts: posts in the course assignment discussion area and posts in the voluntary question and answer (Q&A) discussion area. Course assignment posts are part of the class and the teacher requires students to participate in most of these discussion questions. Q&A posts are voluntary. Peer replies are always encouraged in the course but not compulsory. Topics for course assignment posts have been added to the article(page9,table 3).
Point 4: Several variables need more clarity.
Response 4：Thank you for your comments. We have made detailed additions to several variables in the revised version. The corresponding modifications are reflected in the responses to your specific comments that follow.
Point 5: Learning Performance. There are a few sentences describing learning performance, but they are underwhelming and unclear. It looks like learning performance was based on a total grade percentage scale of 0 to 100. More information is needed to clearly articulate how the grades were assigned. Were the “discussion boards” required and or/part of the overall grade? If so, does this impact the correlation analyses between some of the variables (e.g., number of posts and overall grade)? What other components accounted for the total grade?
Response 5: Thank you for pointing out the shortcomings, we have added clarification on what is relevant for the allocation of grades(page12,line 314). In the course, course assignment posts are part of the class and the teacher requires students to participate in most of these discussion questions. Whether or not a student participates in discussions affects his or her grade. However, we do not consider that this affects the correlation analysis. We have clarified in the revised version that the usual grade is determined by accompanying tests in class and students’ course assignment posts, and the final grade is determined by the final examination. It is worth noting that we have added examples of situations where students may receive bonus marks, such as actively answering questions in class and posting in the Q&A discussion area.
Point 6: Behavioral engagement: The data here are unclear. Is this the average number of posts per discussion board or is this the total number of posts for the course? Was there an expectation of the number of posts required in the course or was this voluntary? The average number of posts is 0.2, and replies to others and peers’ replies are both at 0.05. Did students have any expectation to engage in the behavior of posting? If the instructor had no expectation for students to do so, then the blame can’t be put on students as indicated in the conclusions “The reason for this may be that online learning relies more on the autonomy of students, and students need to make more efforts to complete online learning than offline learning.” Even in the face-to-face classroom, if the instructor does not have the expectation for students to engage with each other on course content, and set them up to do so, then they typically will not.
Response 6：Thank you for pointing out the problem here. The number of posts in the original text refers to the number of topic posts in the Q&A discussion area, and the average number of posts refers to the average number of topic posts per student in the Q&A discussion area. We have clarified in the revised version(line 293; table 4; table 5). The Q&A discussion area is voluntary, but the teacher have indicated in the course that posting in the Q&A discussion area earns bonus marks. In addition, students were encouraged to be active posters during the course. While some students participated in multiple posting activities, the average numbers of student posts and replies are low. Therefore, although teachers encouraged posting, we feel that students’ expectations of posting behavior vary for each individual.
Additional Minor Comments
Point 7: Title: Undergraduate Program or Undergraduate Course? It appears this research analyzed a specific course and not a program.
Response 7：Thank you for your helpful reminders. We have made modifications to the title.
Point 8: May want to spell out SPOC when used in the abstract.
Response 8：Thank you for pointing out the problem. We have added the full name of SPOC to the abstract of the revised version.
Point 9: Line 56— confusing typo? (face-to-face communication is lost?)
Response 9：Thanks for the reminder. Our original writing did give rise to ambiguity and we have now amended it based on your comments.
Point 10: Line 63- be more specific with (in the dozens). This could be 2 dozen or 10 dozen...
Response 10：Thanks for the reminder. Information on the number of SPOC courses has been described more specifically in the revised version.
Point 11: Line 66- spell out MOOC when first used (massive open online course)
Response 11：Thank you for pointing out the problem. We have added the full name of MOOC in the revised version when first used.
Point 12: Line 157 is unclear to the reviewer. Please expand this to offer a more concrete explanation.
Response 12：We appreciate this advice. We have added more concrete explanation of this content in response to your suggestion.
Point 13: Figure 1. (fix “Pees” replies) Mixed spellings of behavior.
Response 13：Thank you for pointing out the problem with the figure. We have fixed the spelling of "Peers’ replies" and corrected the mixed spelling of "behavior".
Point 14: Line 248- what is meant by 12 posts (discussion boards with unique prompts) or 12 posts total (were they required?).
Response 14：Thank you for your enquiry. 12 refers to the total number of topic posts made by students in the Q&A discussion area. There are two types of discussion areas in our course. The Q&A discussion area is where students voluntarily participate in discussions. Details about the discussion areas and postings have been modified in the revised version.
Point 15: Lines 259- 261 are unclear.
Response 15：Thank you for pointing out the ambiguity in this statement. The relevant content regarding the assessment of course grades has been added in the revised version.
Point 16: Lines 292-293. “The survey suggests that except for boredom all students are assigned 100% in terms of curiosity pleasure, and sense of belonging.” The means reported do not indicate this from what I can see.
Response 16：Thank you for pointing out the problem. This is where our presentation was confusing. The analysis statement has been corrected in the revised version.
Point 17: Line 338. The data indicates that most students concentrate on “analyze” at the cognitive level, but failed to enter the “evaluate” and “create” “stage” — were the students asked to evaluate or create in this discussion?
Response 17：Thank you for your enquiry. We have added the topics discussed and their introduction in the revised version (table 3). As the course progressed, the discussion questions demanded a higher level of abstract cognition from students. The early discussion topics tended to require students to remember and understand, while the later discussion topics focused more on students' ability to evaluate and create.
Point 18: Some issues throughout with the format of within-text citations (e.g., Fredeicks, Li Shaung et al. ...).
Response 18：Thank you for pointing out the citation issues. We have made modifications to the citation section in the revised version.
Reviewer 4 Report
generalise , e.g. lines 55-56 - a singular noun evokes the concrete indivuduals;
line 81 - spelling - "ides"; as for the source "Schaufeli introduces" - should be Schaufelli et al. introduce;
line 80 - missing source - Tyler in the List;
line 130 -Bloom´s taxonomy revised? or original
- Was the course and the raised questions in the Englisch or mother tongue? - should be included as a language choice can be one of the factors influencing online learning and investigated factors.
line 337 subchapter 6.1 - rephrasing needed - now, it is much more than a commnet on the data
Author Response
Point 1: lines 55-56 - a singular noun evokes the concrete individuals.
Response 1: Thanks for the reminder. Our original writing did give rise to ambiguity and we have now amended it based on your comments.
Point 2: line 81 - spelling - "ides"; as for the source "Schaufeli introduces" - should be Schaufelli et al. introduce.
Response 2: Thank you for pointing out the problem with the spelling and the misuse of the source. We have fixed the spelling of "ides" and corrected the source of " Schaufeli introduces ".
Point 3: line 80 - missing source - Tyler in the List.
Response 3: Thank you for pointing out the citation issues. We have made modifications to the citation section in the revised version.
Point 4: line 130 -Bloom´s taxonomy revised? or original.
Response 4: Thank you for pointing out the ambiguity in this statement. “Revised” has been added in the revised version.
Point 5: Was the course and the raised questions in the English or mother tongue? - should be included as a language choice can be one of the factors influencing online learning and investigated factors.
Response 5: Thank you for pointing out the ambiguity in this statement. “Taught in Chinese” has been added in the revised version.
Point 6: line 337 subchapter 6.1 - rephrasing needed - now, it is much more than a comment on the data.
Response 6: We appreciate this advice. We have rephrased subchapter 6.1 in response to your suggestion.
Round 2
Reviewer 1 Report
I applaud the author for making a great effort to put forward my suggestions. However, I regret that the current revised article is still not convincing enough.
First, this article was written in a style that is not clear and unintelligible to a reasonably well-informed international professional readership. For example, most of the supplementary literature in this article comes from CNKI, which is inconvenient for international readers.
Second, it is unclear what research gap the authors want to address. No matter whether since the outbreak of the epidemic or before, there are many studies on learning engagement and online learning effectiveness. From the content of the introduction and literature review, there is still no research gap that this study has solved. Therefore, it is also unclear to me what new findings and research implications this study can bring to the field of learning engagement in online education.
Author Response
Point 1: First, this article was written in a style that is not clear and unintelligible to a reasonably well-informed international professional readership. For example, most of the supplementary literature in this article comes from CNKI, which is inconvenient for international readers.
Response 1: Thank you for pointing out the citation issues. If we understand correctly, “most of the supplementary literature in this article comes from CNKI” , does it mean that our manuscript cites too many Chinese references. If this is the case, here are our considerations. As a case study based on the practice of a Chinese university, we felt that citing Chinese literature was inevitable. For example, in the literature review section on online learning engagement, four out of the ten papers we cited were in Chinese. This is because we wanted to introduce research findings that were appropriate to the Chinese context into this study as well, in order to make the constructed model of online learning engagement more relevant. In the latest revised version, there are 47 references, 12 of which are in Chinese. We wonder if this can be considered as an appropriate, though high, proportion of Chinese literature?
Point 2: Second, it is unclear what research gap the authors want to address. No matter whether since the outbreak of the epidemic or before, there are many studies on learning engagement and online learning effectiveness. From the content of the introduction and literature review, there is still no research gap that this study has solved. Therefore, it is also unclear to me what new findings and research implications this study can bring to the field of learning engagement in online education.
Response 2: Thank you for your comments. Based on the online learning environment, our study constructs an online learning engagement analysis model, and explores the relationship between students’ online learning engagement and their online learning performance by taking the Secondary School Geography Curriculum Standards and Textbooks Research, a small-scale private online course (SPOC) of the geography education undergraduate course at Nanjing Normal University, as an example. Our study has the potential to shed light on the following research gaps:
While learning engagement and learning effectiveness have been well researched in the field of online education, little research has been done on online learning engagement in the field of geography in higher education. Our study introduces the more well-researched online learning engagement model to the field of geography online education, which will be beneficial to the scientific analysis of geography online learning data.
As an empirical case study, our findings can provide research support for other studies. For example, compared with other engagement dimensions, emotional engagement has the greatest correlation with learning performance, showing that positive emotional engagement is helpful to improve students' learning.
And as, students’ online learning performance can be enhanced both by behavioral engagement in knowledge reprocessing and positive emotional engagement.
Reviewer 2 Report
Thank you very much for resubmitting and addressing the comments provided.
I have comments in two areas.
1. formatting
2. writing content
FORMATING.
Research methods:
Lines 282-283 and 298:
please change the numbers from 0-9 to letters instead of numbers when relevant. Example: it says 6 modules, it should say, six modules. it says 2 weeks, it should say two weeks.
Please consider similar comment was made in previous version. I acknowledge changes were made in sections where this was highlighted, but please check all the text, as there may be more than the ones I could see.
References:
Please rectify where needed. For example, the title for some references is abbreviated (some examples: references 18, 19, 29, while for most is in full. You need consistency.
Also, most references capitalise each word, while others only capitalise the first one (some examples: references 37 and 41)
WRITTING CONTENT
Research methods:
I acknowledge that changes were made in this section. However, still feel further changes are made.
From the positive side, there is a lot of detail in this section. Nevertheless, I strongly feel that some of the text for each of the subsections would be better suited to the literature review section rather than research methods.
For example, I think the following text should go to literature review:
Mazzolini analyzes posts in 400 course forums and has found that teacher participation rate, posting time and the contents of posts affected learners' participation in online discussions [28]. Nandi aruges that students' willingness to discuss in the online learning environment is easily influenced by teachers' behavior [29]. According to Khe’s study, online learning behaviors include learners' discussion, recognition and response [30]. Feng Xiaoying points out that online learning behaviors included individual behaviors that learners interact with learning platforms and interactive behaviors with groups such as teachers and students [31].
While the following paragraph could remain in research methods, by including the numbers of references that the author considered.
Based on prior research on the division of behavioral engagement dimension, this study includes four distinct learning behaviors that are frequently utilized in online learning platforms into the behavioral engagement dimension.
Also, please check spelling. I can see in line 170 that it says "aruges" and it should say "argues".
I did not made an extensive review of the spelling. But please check, as there may be more misspellings.
Also the following sentences need to be referenced:
The focus of research has shifted from learning states, to learning behaviors and cognitive states, to learning emotional experiences. Researchers have recently concentrated on the social interaction of online learning and have proposed the dimensions of social engagement.
The above are just examples to clarify what I mean with my comment on research methods section. However, this applies to all the subsections of research methods, not only to the examples listed above.
Author Response
Point 1:
Lines 282-283 and 298:
please change the numbers from 0-9 to letters instead of numbers when relevant. Example: it says 6 modules, it should say, six modules. it says 2 weeks, it should say two weeks.
Please consider similar comment was made in previous version. I acknowledge changes were made in sections where this was highlighted, but please check all the text, as there may be more than the ones I could see.
Response 1: Thank you very much for your comments. We have already made changes in the content you mentioned (Lines 282-283 and 299). In addition, we have re-read the numbers from 0-9 during the entire article and made some changes. However, we still expect to keep the numbers from 0-9 in some sentences of the article depending on the context: for example, sentences containing multiple numerical values or those based on numerical representation of quantitative analysis results.
Point 2:
Please rectify where needed. For example, the title for some references is abbreviated (some examples: references 18, 19, 29, while for most is in full. You need consistency.
Also, most references capitalise each word, while others only capitalise the first one (some examples: references 37 and 41)
Response 2: Thanks for the suggestion! We have re-corrected the references section. We checked the information of the references (e.g., titles and journal names) and try to make sure they have better consistency.
Point 3:
For example, I think the following text should go to literature review:
Mazzolini analyzes posts in 400 course forums and has found that teacher participation rate, posting time and the contents of posts affected learners' participation in online discussions [28]. Nandi aruges that students' willingness to discuss in the online learning environment is easily influenced by teachers' behavior [29]. According to Khe’s study, online learning behaviors include learners' discussion, recognition and response [30]. Feng Xiaoying points out that online learning behaviors included individual behaviors that learners interact with learning platforms and interactive behaviors with groups such as teachers and students [31].
Response 3: Thank you for pointing out the problem. We have moved this part to literature review.
Point 4:
While the following paragraph could remain in research methods, by including the numbers of references that the author considered.
Based on prior research on the division of behavioral engagement dimension, this study includes four distinct learning behaviors that are frequently utilized in online learning platforms into the behavioral engagement dimension.
Response 4: Thanks for the reminder. We have now amended it based on your comments.
Point 5:
Also, please check spelling. I can see in line 170 that it says "aruges" and it should say "argues".
I did not made an extensive review of the spelling. But please check, as there may be more misspellings.
Response 5: We appreciate your pointing out the problem! We have checked the spelling in the article again to minimize misspellings.
Point 6:
Also the following sentences need to be referenced:
The focus of research has shifted from learning states, to learning behaviors and cognitive states, to learning emotional experiences. Researchers have recently concentrated on the social interaction of online learning and have proposed the dimensions of social engagement.
Response 6: Thanks for the reminder. We have now amended it based on your comments.
Point 7:
The above are just examples to clarify what I mean with my comment on research methods section. However, this applies to all the subsections of research methods, not only to the examples listed above.
Response 7: We appreciate it very much for your reminder. We have revised the problem you mentioned. For other similar problems, we also modified and improved. We hope this revised version can reach the acceptable level.
Reviewer 3 Report
Thank you for considering and providing clear revisions to the manuscript that align with the points presented in review #1. The manuscript has been improved significantly and now aligns more with a typical case study format. Although I question some of the analyses (e.g., coding discussion posts to Bloom's -i.e. are students creating in the discussion board?), the authors describe their rationale and procedures for the chosen methodologies and analyses. Although improvements to the paper were made, I would still recommend another thorough review by all authors for improvements in the language and structure of the manuscript (minor revisions). For example, instead of saying "generally not more than 10 dozens," I would say "generally less than 120 students;" also, be precise in the length of the study (# months/weeks) instead of "this study took weeks".
Author Response
Point 1: Although improvements to the paper were made, I would still recommend another thorough review by all authors for improvements in the language and structure of the manuscript (minor revisions). For example, instead of saying "generally not more than 10 dozens," I would say "generally less than 120 students;" also, be precise in the length of the study (# months/weeks) instead of "this study took weeks".
Response 1: We appreciate it very much for your good suggestion. The language and structure of the manuscript have been thoroughly and carefully reviewed by all authors. We have revised the problem you mentioned. For other similar problems, we also modified and improved. We hope this revised version can reach the acceptable level.
Round 3
Reviewer 1 Report
I applaud the author for making a great effort to put forward my suggestions. Unfortunately, I think this manuscript still does not illustrate the research gap well and not response the second question that I have proposed in the second round. It is unclear what new findings and research implications this study can bring to the field of online education. I don't think the current manuscript meet the publication requirement of Sustainability.
Author Response
Point 1: Unfortunately, I think this manuscript still does not illustrate the research gap well and not response the second question that I have proposed in the second round. (No matter whether since the outbreak of the epidemic or before, there are many studies on learning engagement and online learning effectiveness. From the content of the introduction and literature review, there is still no research gap that this study has solved.)
Response 1: We really appreciate it very much for your enquiry into the gaps in our research. After reflecting on this comment of yours, we find that our manuscript does not explicitly address this issue. Therefore, in the introduction and literature review sections (lines 38-53, 70-75, 137-146), we have added a discussion of the gaps that our study might fill. We hope that this revision of our work will lead to a more systematic and improved manuscript.
Point 2: It is unclear what new findings and research implications this study can bring to the field of online education.
Response 2: Thank you for your helpful reminders. After careful consideration, we feel that the following findings from our study may be of some potential significance.
(1) Based on the descriptive analysis of the data of emotional engagement, we came to the result: Students have a maximum score of 5 in terms of curiosity, pleasure and sense of belonging, but only a maximum score of 4 for boredom. In comparing the amount of engagement of each dimension, curiosity is the most invested by students, followed by pleasure, the least is boredom.
Here we respond to the lack of research on emotional engagement in online learning mentioned in the literature review. Meanwhile, compared with other engagement dimensions, emotional engagement has the greatest correlation with learning performance, showing that positive emotional engagement is helpful to improve students' learning.
(2) As for the cognitive engagement, the data indicates that most students concentrate on "analyze" at the cognitive level, but failed to enter the "evaluate" and "create" stage. This suggests that online learning allow students to make connections between old and new knowledge and develop problem-solving skills, but does not provide substantial improvement at higher cognitive levels.
These results imply that we cannot be too optimistic about the quality of student engagement under the auspices of emergency online learning.
(3) The data for the behavior " Q&A topic posting", which appears most frequently in comparison, shows that the average number of topic posts in voluntary Q&A forum is 0.2 per person, with a standard deviation of 0.58, indicating that most students do not have behavioral engagement when learning online.
Our study shows that if the discussion mode of the learning platform is not attractive enough, students' enthusiasm will be inadequate.
(4) Our study applies degree centrality, closeness centrality, betweenness centrality and cohesive subgroup to mutually reflect the influence of social engagement on learning performance. Among them, degree centrality and betweenness centrality are significantly correlated with learning performance, indicating that when students interact with more peers and teachers in the process of learning, and play a mediating role in the communication of other peers, the closer they are to the center of interaction, the better their learning performance will be.
