A Novel Deep Learning Technique for Detecting Emotional Impact in Online Education

Round 1

Reviewer 1 Report (New Reviewer)

The manuscript discusses a novel Deep Learning Technique regarding  the detection of student emotions. For this aim, facial recognition algorithms have been used to extract useful information from online platforms and image classification techniques have been applied to detect the
emotion of student or teacher faces comparing two deep learning models. Such studies are very useful and could contribute to increase productivity and improve the education process.

The manuscript is interesting and deserves publication in the Journal.
However, before that I suggest the following points, which need a furter discussion and aclaration:
1. The authors must include a larger discussion regarding the correlation between the emotional status of the teachers and their students and how this is contributing to the education process.

2. A deeper explanation of the better results for positive emotions must be included and if necessary some additional quantitative measures
may be included in order to  better understand the phenomenon.

3. The authors need to discuss what is the meaning of a negative F1, as by definition, F1 is a positive quantity (Eq. 3).

In conclusion, I recommend the publication of the manuscript after improving the above points.

Author Response

The manuscript is interesting and deserves publication in the Journal.
However, before that I suggest the following points, which need a furter discussion and aclaration:
1. The authors must include a larger discussion regarding the correlation between the emotional status of the teachers and their students and how this is contributing to the education process.

Response: Thank you for your comment. We add a detailed discussion as a new section (Discussion) to show the relation between the teachers and their students.

2. A deeper explanation of the better results for positive emotions must be included and if necessary some additional quantitative measures may be included in order to better understand the phenomenon.

Response: Thank you for your comment. We tried the best to show the effect of the used emotion in the results section.

3. The authors need to discuss what is the meaning of a negative F1, as by definition, F1 is a positive quantity (Eq. 3).
   Response: Thank you for your comment. We added the definition of the used measures and terms as given in section 4.1.

In conclusion, I recommend the publication of the manuscript after improving the above points.

Response: Thank you for your effort in reviewing and enhancing this paper.

 

Reviewer 2 Report (New Reviewer)

The quality of writing in this paper is extremely low. The submission is full of grammatical errors, unprofessional expressions, poorly structured sentences, vague and unclear statements, etc. As such this submission simply cannot be passed onto the next reviewing stage.

In addition to the above, briefly, the novelty of the work is low at best, and the methodology employed entirely inappropriate in the context of the aims stated.

Author Response

The quality of writing in this paper is extremely low. The submission is full of grammatical errors, unprofessional expressions, poorly structured sentences, vague and unclear statements, etc. As such this submission simply cannot be passed onto the next reviewing stage.

Response: Thank you for your comment. We revised the whole paper and checked the language by a native speaker researcher.

In addition to the above, briefly, the novelty of the work is low at best, and the methodology employed entirely inappropriate in the context of the aims stated.

Response: Thank you for your comment. We highlight the main contribution of this research particularity in the introduction section.

Reviewer 3 Report (New Reviewer)

I suggest simplifying it or better explaining with realistic examples. This paper has a potential to be accepted, but some important points have to be clarified or fixed before we can proceed and a positive action can be taken.

Comments for author File: Comments.pdf

Author Response

I suggest simplifying it or better explaining with realistic examples. This paper has a potential to be accepted, but some important points have to be clarified or fixed before we can proceed and a positive action can be taken.

Response: We checked the whole paper and tried our best to make it clearer for the readers.  Also, the experiments section, we provided several examples (cases) and its results according to the proposed method.

Round 2

Reviewer 2 Report (New Reviewer)

While I recognize the authors' attempt at addressing my criticisms, and some improvement is indeed apparent, the key methodological flaw remains: the data used is entirely inappropriate to the work's aims or claims, and must not be published as it lacks credibility.

Author Response

Comment: While I recognize the authors' attempt at addressing my criticisms, and some improvement is indeed apparent, the key methodological flaw remains: the data used is entirely inappropriate to the work's aims or claims, and must not be published as it lacks credibility.

Response: Thank you for your efforts in reviewing and enhancing this paper. We removed the datasets used in this research as was given in Table 1.

Reviewer 3 Report (New Reviewer)

In several instances I also suggested to cite more relevant and recent literature. Furthermore I made additional suggestions for more in-depth analyses of the data.

Comments for author File: Comments.pdf

Author Response

Comment: In several instances, I also suggested to cite more relevant and recent literature. Furthermore I made additional suggestions for more in-depth analyses of the data.

Response: Thank you for your efforts in reviewing and enhancing this paper. We used some new citations as per request, and we added more analysis for the data.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Although many previous research have been conducted on emotional intelligence, only few adopted its use in education. The detection of student emotions could contribute significantly in increasing the productivity and improving the education process. This paper use deep learning to detect student emotions and aims to map the relationship between teaching practices and student learning based on the emotional impacts for both students and their teachers. Facial recognition are used to extract useful information from online platforms. According to the emotion analysis part, several Deep learning techniques have been applied to train and test the emotion classification process. The obtained results showed that the performance of the proposed method is promising using both techniques as presented in the experimental results section.

 

This paper is straightforward and easy to follow, while the major issue is I can not get the connection between emotion classification and online education. Actually, there are very few explanation and experiments about online education. Other issues includes:

1.     For 9 classes classification, the data are un-balanced, like ‘contempt’ have much fewer samples than ‘sad’. What to do with this un-balanced data.

2.     Most of the works are focus on facial recognition based emotion classification. How to connect the emotion classification to “The detection of student emotions could contribute significantly in increasing the productivity and improving the education process.” What is the result for help online education?

Author Response

Although many previous research have been conducted on emotional intelligence, only few adopted its use in education. The detection of student emotions could contribute significantly in increasing the productivity and improving the education process. This paper use deep learning to detect student emotions and aims to map the relationship between teaching practices and student learning based on the emotional impacts for both students and their teachers. Facial recognition are used to extract useful information from online platforms. According to the emotion analysis part, several Deep learning techniques have been applied to train and test the emotion classification process. The obtained results showed that the performance of the proposed method is promising using both techniques as presented in the experimental results section.

This paper is straightforward and easy to follow, while the major issue is I can not get the connection between emotion classification and online education. Actually, there are very few explanation and experiments about online education. Other issues includes:

Comment: For 9 classes classification, the data are un-balanced, like ‘contempt’ have much fewer samples than ‘sad’. What to do with this un-balanced data.

Author reply to the reviewer: Many thanks to the reviewer for this valuable comment, the main contribution of this paper is to link the emotional impacts of both students and teachers on online education. The unbalanced data will be resolved in future work as both oversampling and under-sampling techniques will be implemented and applied on the gathered data set to make the accuracy much better according to the emotion classification process. As for clarification, a paragraph that explain this issue has been added to the modified manuscript at section 3 in page No. 9 .

2. Most of the works are focus on facial recognition based emotion classification. How to connect the emotion classification to “The detection of student emotions could contribute significantly in increasing the productivity and improving the education process.” What is the result for help online education?

Author reply to the reviewer: Many thanks to the reviewer for this valuable comment.

A discussion section, as given in Section 5, has been added to the modified manuscript defining this issue.

Also, as a results, the purpose of this research was to analyze to what extent there are differences in the experience of teacher students regarding their achievement with respect to individual emotions, this has been evaluated by comparing teacher students' who attended the course physically with those who attended it virtually. As a result of what we conducted here, students with positive emotions have more benefits than negative emotions in university courses in teacher education.

Author Response File: Author Response.pdf

Reviewer 2 Report

1) Ambiguity caused by picking single emotion labels. 

Emotion classification may rely on requiring participants to choose a single emotion category label? 

If so then when people report they are feeling disgusted or disappointed could they also be feeling angry? 

This would also challenge the main conclusions.

2) I can't say anything about the different ways that machine learning can be used to look at the data. But I was wondering if 83.87 percent accuracy is really good enough in this line of work. I'm not an expert on these methods or how to judge a "good" model, but it seems like these analyses make a lot of mistakes when they try to classify emotions.

3) Please make comments on prospective of present technique for development of emotion recognition hardware. Authors should discuss some previously reported studies on image/patter recognition such  Sci Rep 11, 5577 (2021); Small 2021, 17, 2103543.

Author Response

Comment 1) Ambiguity caused by picking single emotion labels. 

Emotion classification may rely on requiring participants to choose a single emotion category label? 

If so then when people report they are feeling disgusted or disappointed could they also be feeling angry? 

This would also challenge the main conclusions.

Author reply to the reviewer: Many thanks to the reviewer for this valuable comment, the prepared data set has considered this issue, as many emotions figures have been appeared in more than one data set. Also, the conclusion section has been totally modified as requested. Moreover, adiscussion section, as given in Section 5, has been added to the modified manuscript defining this issue.

 

Comment 2) I can't say anything about the different ways that machine learning can be used to look at the data. But I was wondering if 83.87 percent accuracy is really good enough in this line of work. I'm not an expert on these methods or how to judge a "good" model, but it seems like these analyses make a lot of mistakes when they try to classify emotions.

Author reply to the reviewer: Many thanks to the reviewer for this valuable comment, the main contribution of this paper is to link emotional impacts of both students and teachers on online education. The unbalanced data will be resolved in future work as both oversampling and under sampling techniques will be implemented and applied on the gathered data set to make the accuracy much better according to the emotion classification process. As for clarification, a paragraph that explain this issue has been added to the modified manuscript at section 3 in page No. 9 .

3) Please make comments on prospective of present technique for development of emotion recognition hardware. Authors should discuss some previously reported studies on image/patter recognition such  Sci Rep 11, 5577 (2021); Small 2021, 17, 2103543.

Author reply to the reviewer: Many thanks to the reviewer for this valuable comment, the given studies ahve been added to the modified manuscript.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

 I appreciate authors’ efforts of responding and modification in discussion section.

If only consider the emotion classification part, the contribution is very limited.

As author mentioned “the main contribution of this paper is to link the emotional impacts of both students and teachers on online education.”

But there are no experiments results to show how emotion effect online education for students and teachers? And what is the measuring methods to evaluate the education results? What is the numerical

Without these results, this paper can not state the contributions as “link the emotional impacts of both students and teachers on online education.”

Author Response

Thank you for your efforts in reviewing and enhancing this paper. We really appreciate your efforts.

Comments:

I appreciate authors’ efforts of responding and modification in discussion section.

If only consider the emotion classification part, the contribution is very limited.

Response: Thank you for the comments. In this paper, we proposed a classification technique for the images based on the emotional part to categorize the student and teachers behavior on the online education. The benefit of applying the proposed system in real life relies on grouping the students based on their class emotions into two groups, those who will be engaging in the face to face education and the other who can get better results by attending online courses. Moreover, according to the tested sample, 67% of the students who are not interested during the face to face education - based on their emotions will get better results if they attend the same course virtually.

As author mentioned “the main contribution of this paper is to link the emotional impacts of both students and teachers on online education.”

But there are no experiments results to show how emotion effect online education for students and teachers? And what is the measuring methods to evaluate the education results? What is the numerical

 

Response: Thank you for the comments. The application of the proposed system to a sample of online courses and in-class students resulted in an unexpected conclusion. In-class students mainly were positive and interactive emotions such as happy, surprised, sad, and angry. Furthermore, the online students' emotions were the negative faces such as contempt, disgust, fear, and natural. It is worth mentioning that the ahegao emotion just appeared a few times with only online course students and never appeared with in-class students. Furthermore, according to the grades achievements, it was expected that online course students get lower total grades according to their emotions during the class. However, the system proved the opposite, where online course students achieved higher than in-class students.

 

Without these results, this paper can not state the contributions as “link the emotional impacts of both students and teachers on online education.”

Response: We said they because the results of the classification of the emotional behaviors help in classify the behavior of the student and teachers during the activity of the online education.

 

Round 3

Reviewer 1 Report

Not enough experiment results about Online Education.