Power to the Learner: Towards Human-Intuitive and Integrative Recommendations with Open Educational Resources
Round 1
Reviewer 1 Report
The study was well executed with a strong research design. I had not considered the SDG for Quality Education for this paper, but that is precisely what it does: an integrative educational recommender that accounts for the learners' interests, knowledge and content novelty, content popularity, and improving predictions of learning engagement.
There are a few typos: line 54, p. 2 should be "build"; a space is needed after the period on line 315, p. 10, leaner or learner? etc. The authors should be applauded for investigating humanly-intuitive representation to promote self-reflection and meta-cognition, along with self-regulated learning. This is exciting work.
Author Response
Many thanks for the positive and constructive comments about our work, especially for appreciating how humanly intuitive learner models and our work towards that will connect to self-regulated, self-reflecting learning which will impact SDG 4.
We have addressed the points you have brought up below:
Point 1: There are a few typos: line 54, p. 2 should be "build"; a space is needed after the period on line 315, p. 10, leaner or learner?
Response 1: We have gone through the manuscript several times to make spelling and grammar corrections.
Reviewer 2 Report
The paper has a sound and solid technical content. I have some editorial comments.
1. The title could be selected such that it reflects the methodology used in this paper.
2. There is a need for comprehensive proofreading as there are several grammar, spelling, punctuation, and formatting errors.
3. The number of self citations seems excessive (even though they are part of the same project). Please limit that to maximum the main 3-4 prior works.
4. Figures and tables are expected to appear soon after they are mentioned in the text.
5. Please use subheadings and subtitles more efficiently rather so that all the subtitles are numbered.
6. The conclusion section could be more concise and focused on the contributions of the projects.
7. Can Figure 8 be removed as it seems to be repeated in Figure 9?
8. The use of bold fonts in some parts of the text seems not aligned with the template.
9. The main process followed in this paper can be explained more effectively using a flowchart or block diagram.
Author Response
Many thanks for the positive and constructive comments about our work, especially for appreciating the solidity and soundness of the work. We have addressed the points you have brought up below:
Point 1. The title could be selected such that it reflects the methodology used in this paper.
Response 1: We thank you for your suggestion. After a detailed discussion with the co-authors, we concluded that the current title is suitable for this work. Our aim is to make this manuscript easily discoverable to researchers who are working on educational recommenders and open educational resources while depicting the strong features of this work by including the humanly-intuitiveness and integrative nature in the title. As we describe the key experiments and methodology in the abstract, we feel the current title serves its purpose.
Point 2. There is a need for comprehensive proofreading as there are several grammar, spelling, punctuation, and formatting errors.
Response 2: We have gone through the manuscript several times to make spelling and grammar corrections.
Point 3. The number of self-citations seems excessive (even though they are part of the same project). Please limit that to maximum the main 3-4 prior works.
Response 3: We agree that there are many self-citations in the manuscript. This is due to the fact that this work builds heavily on various different directions of research that we conducted previously within the same project over 3+ years (e.g. well researched opinions, published datasets, developing humanly-intuitive user interface components and learning platforms etc.). We strongly believe that many of the cited papers are essential for us to communicate a clear picture of this work and its context. With the recommendation of the assistant editor, we have reduced the number of citations to eight.
Point 4. Figures and tables are expected to appear soon after they are mentioned in the text.
Response 4: We have edited the manuscript to make sure that the figures and tables appear as close to their mentions as possible.
Point 5. Please use subheadings and subtitles more efficiently rather so that all the subtitles are numbered.
Response 5: We have edited the manuscript and done some restructuring to make sure that all topics are numbered and referenceable.
Point 6. The conclusion section could be more concise and focused on the contributions of the projects.
Response 6: We have gone through the manuscript and made the conclusion section significantly shorter. Given there are several key conclusions at the end of this work, the section is slightly longer.
Point 7. Can Figure 8 be removed as it seems to be repeated in Figure 9?
Response 7: We have removed the above-referenced figure and only included figure 9. We have redirected relevant references to this figure.
Point 8. The use of bold fonts in some parts of the text seems not aligned with the template.
Response 8: We have removed the bold-faced parts from the manuscript. We only use italic face in places we believe that the phrases should be emphasised.
Point 9. The main process followed in this paper can be explained more effectively using a flowchart or block diagram.
Response 9: We have included figure 6 (a Sankey diagram) that graphically illustrates the phased methodology used in building the integrative learner models.
Reviewer 3 Report
The authors proposed the TrueLearn INK and the TrueLearn PINK models to build a humanly-intuitive, sub-symbolic representation improving predictive performance. The paper is well-structured, and the results are compared with the literature. However, the language must be improved. Also, the numbering of sections must be fixed. Congratulations to the authors for the well-researched subject.
Author Response
Many thanks for the positive and constructive comments about our work, especially for appreciating the solidity and soundness of the work. We have addressed the points you have brought up below:
Point 1. The language must be improved. Also, the numbering of sections must be fixed.
Response 1: We have gone through the manuscript several times to make spelling and grammar corrections.
Reviewer 4 Report
The paper under review introduces a new online learner system with experiments assessing its performance. The analysis is comprehensive and the material presented is quite rich. And I believe it makes some contributions to the literature. However, I think there is much room for the paper to be improved, especially in terms of "readability".
1. The paper reads like a technical report or an NGO report rather than a journal article. The information is not covered through cohesive "arguments" (i.e. logically- linked paragraphs); rather, it is conveyed in a "bullet point" style with a lot of abbreviations, which makes the paper a bit "choppy," lacking a smooth flow.
2. The Introduction section is too short to be informative. For example, what has been done in your analysis? Without describing the analysis performed, how can the reader understand your contributions? Likewise, more information on the design of your study, the method adopted (especially how it resolves an existing problem), and the data used would also help the reader understand your contributions.
3. The Introduction section should start with 1, rather than 0. Also, with only two paragraphs, what's the point of having sub-section titles in the Introduction. Similarly, many of the sub-section titles could be dropped or converted into sentences.
4. The discussion on previous work is largely inadequate. More detailed discussions on some examples of existing online learner models would be helpful.
5. The style of abbreviations is not consistent. For example, wouldn't ERS be better than EduRecSys in terms of consistency in style? Also, why are some in normal font and some others in italic type?
6. Typos can be found at times. For example, page 2, line 76: "should he" should be "should be".
Author Response
Many thanks for the positive and constructive comments about our work, especially for appreciating the comprehensiveness and richness of the work. We have addressed the points you have brought up below:
Point 1. The paper reads like a technical report or an NGO report rather than a journal article. The information is not covered through cohesive "arguments" (i.e. logically- linked paragraphs); rather, it is conveyed in a "bullet point" style with a lot of abbreviations, which makes the paper a bit "choppy," lacking a smooth flow.
Response 1: We thank you for your suggestion. We have rephrased and restructured the paper to have a more logical flow. We have also explicitly mentioned references to other sections in the paper that are relevant to improving the connectedness and added linking sentences to improve the flow.
Point 2. The Introduction section is too short to be informative. For example, what has been done in your analysis? Without describing the analysis performed, how can the reader understand your contributions? Likewise, more information on the design of your study, the method adopted (especially how it resolves an existing problem), and the data used would also help the reader understand your contributions.
Response 2: We have added more content to the introduction to make sure that the context and the contributions are covered more thoroughly.
Point 3. The Introduction section should start with 1, rather than 0. Also, with only two paragraphs, what's the point of having sub-section titles in the Introduction. Similarly, many of the sub-section titles could be dropped or converted into sentences.
Response 3: We have corrected the numbering and restructured the manuscript in a way that all sections have numbering.
Point 4. The discussion on previous work is largely inadequate. More detailed discussions on some examples of existing online learner models would be helpful.
Response 4: We have increased the size of the related work section where we have described in detail, how the related work attempts to address the problems and how they are different from the proposal.
Point 5. The style of abbreviations is not consistent. For example, wouldn't ERS be better than EduRecSys in terms of consistency in style? Also, why are some in normal font and some others in italic type?
Response 5: We have changed the abbreviation to EdRecSys as this is a more suitable abbreviation. While the abbreviation style differs from others in the manuscript, we feel EdRecSys is a more suitable acronym than ERS as EdRecSys abbreviation is more established and popular in the educational recommender systems/educational data mining communities. (e.g.: http://events.kmi.open.ac.uk/edrecsys2020/, https://edrecsys.wordpress.com/, https://edrecsys2017.wordpress.com/ etc.)
Point 6. Typos can be found at times. For example, page 2, line 76: "should he" should be "should be".
Response 6: We have gone through the manuscript and made spelling and grammar corrections.
Round 2
Reviewer 2 Report
Thanks for your answers and modifications. I have some minor feedback:
A basic system introducing the recommendation system or recommenders and in particular educational recommender is missing in abstract and introduction.
Line 3: What is "our" sustainability agenda
Lines 4-7: run-on sentence. Can you break it into two sentences or edit?
Line 11: What is the cold-start problem?
Line 26: two instances of "at scale"
Line 30: "brings now" : grammar error
Lines 30-40: two instance of using "at scale". one is redundant.
In equation 3 and some other equations, parameters are not fully explained.
Figure 7 (8, and 9 as well) needs grids for more clarity. Also, it needs more discussion.
Another round of proofreading and editing is a must.
Author Response
We thanks you for your feedback. The response to the pointed feedback is as follows:
Point 1: A basic system introducing the recommendation system or recommenders and in particular educational recommender is missing in abstract and introduction.
Response 1: We have added a few sentences in lines 55 onwards to describe how a recommender system and how an educational recommender works specifically.
Point 2: Line 3: What is "our" sustainability agenda
Response 2: Changed it to the "world's" sustainability agenda.
Point 3: Lines 4-7: run-on sentence. Can you break it into two sentences or edit?
Response 3: Modified it to be more readable. Broken the sentence into multiple sentences.
Point 4: Line 11: What is the cold-start problem?
Response 4: Described cold-start problem in line 12
Point 5: Line 26: two instances of "at scale"
Response 5: Corrected
Point 6: Line 30: "brings now": grammar error
Response 6: Corrected
Point 7: Lines 30-40: two instance of using "at scale". one is redundant.
Response 7: Corrected
Point 8: In equation 3 and some other equations, parameters are not fully explained.
Response 8: Changed the text in all the equations to describe the terms used in the equations
Point 9: Figure 7 (8, and 9 as well) needs grids for more clarity. Also, it needs more discussion.
Response 9: Added grid lines to the plot figures and added more discussion including an additional paragraph starting from line 593
Point 10: Another round of proofreading and editing is a must.
Response 10: We got a native speaker to parse through the manuscript and make grammatical corrections.
Reviewer 4 Report
The paper has been improved.
Author Response
We thank you for your valuable feedback.
