Closing the Gap: Automated Distractor Generation in Japanese Language Testing

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The article addresses the challenge of generating high-quality multiple-choice questions, particularly focusing on creating effective distractors, for Japanese language proficiency tests. Specifically, the authors propose an automated solution for generating cloze tests with distractors, and they evaluate the quality, difficulty, and preferred distractor types of these questions, comparing them to human-made questions through both automated and manual evaluations. The topic is very interesting and addresses a crucial aspect of language assessment (distractor generation in multiple-choice questions), discussing how NLP-based methods may contribute to improving this process. However, the paper occasionally lacks clarity, especially when delving into Japanese-specific elements and explanations, which should be made clearer for readers who are not familiar with Japanese. Speaking from the perspective of a non-Japanese speaker, I found some parts rather challenging and had to resort to additional external resources to thoroughly grasp certain sections of the paper (and I am not sure I have). Considering the paper’s potential relevance to researchers working across various linguistic domains, I believe enhancing clarity would strengthen the article and ensure that the paper is accessible and valuable to a broader audience, including those who may not have prior knowledge of Japanese.

I have added some comments in the attached file, but I would generally recommend the author to provide more explicit explanations in the Japanese-specific sections. 

Comments for author File: Comments.pdf

Comments on the Quality of English Language

Sometimes the text is not entirely clear, proofreading from an English native speaker might be helpful. 

 

Author Response

Thank you for your valuable feedback and the provided PDF document. We sincerely appreciate your input as it has greatly contributed to enhancing the quality of our paper.

Enclosed, please find a PDF outlining the specific actions we have taken to address the issues you highlighted.

Once again, thank you for your valuable contributions.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear author(s)

Thank you for submitting this manuscript. It was a very interesting and innovative study but I think it should be discussed more in terms of theory... as you simply look at the quality of the model itself rather than seeing what others have done with similar models.

As well as more links to the theory, I would like to see examples of questions the model has produced with an explanation of why they are good or bad distractors. Also, what made the distractors easy or difficult as JLPT questions

Comments on the Quality of English Language

The English level just requires minor editing

Author Response

Thank you for your valuable feedback. We sincerely appreciate your input as it has greatly contributed to enhancing the quality of our paper.

Enclosed, please find a PDF outlining the specific actions we have taken to address the issues you highlighted.

Once again, thank you for your valuable contributions.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

all fine now, thank you for the changes