Multi-Perspective Adaptive Paperless Examination Cheating Detection System Based on Image Recognition
Abstract
:1. Introduction
2. Related Work
2.1. Single-Perspective Cheating Behavior Detection Method
2.2. Dual-Perspective Cheating Behavior Detection Method
3. Proposed Three-Perspective Adaptive Paperless Online Cheating Detection System
3.1. Three-Perspective Cheating Detection System Framework
3.2. Gaze Direction Recognition Model Based on Swin Transformer
3.3. Cheating Tool Detection Model Based on Lightweight-YOLOv5-CA Network
3.4. Multimodal Cheating Behavior Determination Model Based on BP Neural Network
3.5. Adaptive Three-Perspective Cheating Behavior Determination Model
4. Experimental Results and Analysis
4.1. Experimental Environment and Data
4.2. Results of Gaze Direction Recognition
4.3. Result of Cheating Tool Detection
4.4. Results of Three-Perspective Adaptive Cheating Behavior Detection
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Office of Educational Technology of the United States. Artificial Intelligence and the Future of Teaching and Learning. 2023. Available online: https://digital.library.unt.edu/ark:/67531/metadc2114121 (accessed on 1 May 2023).
- Ministry of Education of the People’s Republic of China. Huai Jinpeng, Minister of Education in China, Delivered a Keynote Speech at the 2024 World Digital Education Conference: Jointly Promote the Application, Sharing, and Innovation of Digital Education. 2024. Available online: http://www.moe.gov.cn/jyb_xwfb/moe_176/202402/t20240201_1113761.html (accessed on 1 February 2024).
- Luong, H.H.; Khanh, T.T.; Ngoc, M.D. Detecting Exams Fraud Using Transfer Learning and Fine-Tuning for ResNet50. In Future Data and Security Engineering. Big Data, Security and Privacy, Smart City and Industry 4.0 Applications, Proceedings of the 9th International Conference (FDSE), Ho Chi Minh City, Vietnam, 23–25 November 2022; Springer: Singapore, 2022. [Google Scholar]
- Dilini, N.; Senaratne, A.; Yasarathna, T. Cheating detection in browser-based online exams through eye gaze tracking. In Proceedings of the 2021 6th International Conference on Information Technology Research(ICITR), Moratuwa, Sri-Lanka, 1–3 December 2021. [Google Scholar]
- Singh, J.; Aggarwal, R.; Tiwari, S. Exam Proctoring Classification Using Eye Gaze Detection. In Proceedings of the 2022 3rd International Conference on Smart Electronics and Communication (ICOSEC), Trichy, India, 20–22 October 2022. [Google Scholar]
- Susithra, V.; Reshma, A.; Gope, B. Detection of anomalous behaviour in online exam towards automated proctoring. In Proceedings of the 2021 International Conference on System, Computation, Automation and Networking(ICSCAN), Puducherry, India, 30–31 July 2021. [Google Scholar]
- Garg, K.; Verma, K.; Patidar, K. Convolutional Neural Network based Virtual Exam Controller. In Proceedings of the 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 13–15 May 2020. [Google Scholar]
- Jadi, A. New detection cheating method of online-exams during COVID-19 pandemic. Int. J. Comput. Sci. Netw. Secur. 2021, 21, 123–130. [Google Scholar]
- Kaddoura, S.; Gumaei, A. Towards effective and efficient online exam systems using deep learning-based cheating detection approach. Intell. Sys. Appl. 2022, 16, 200153. [Google Scholar] [CrossRef]
- Atoum, Y.; Che, L.; Liu, A.X. Automated online exam proctoring. IEEE Trans. Multimed. 2017, 19, 1609–1624. [Google Scholar] [CrossRef]
- Alrubaish, F.A.; Humaid, G.A.; Alamri, R.M. Automated detection for student cheating during written exams: An updated algorithm supported by biometric of intent. In Proceedings of the Advances in Data Science, Cyber Security and IT Applications: First International Conference on Computing (ADSCSITA), Riyadh, Saudi Arabia, 10–12 December 2019. [Google Scholar]
- Hossain, Z.T.; Roy, P.; Nasir, R. Automated Online Exam Proctoring System Using Computer Vision and Hybrid ML Classifier. In Proceedings of the 2021 IEEE International Conference on Robotics, Automation, Artificial-Intelligence and Internet-of-Things (RAAICON), Dhaka, Bangladesh, 3–4 December 2021. [Google Scholar]
- Indi, C.S.; Pritham, V.; Acharya, V. Detection of malpractice in e-exams by head pose and gaze estimation. Int. J. Emerg. Technol. Learn. 2021, 16, 47. [Google Scholar] [CrossRef]
- Bawarith, R.; Basuhail, A.; Fattouh, A. E-exam cheating detection system. Int. J. Adv. Comput. Sci. Appl. 2017, 8, 1–6. [Google Scholar] [CrossRef]
- Masud, M.M.; Hayawi, K.; Mathew, S.S. Smart online exam proctoring assist for cheating detection. In Proceedings of the Advanced Data Mining and Applications: 17th International Conference(ADMA), Sydney NSW, Australia, 2–4 February 2022. [Google Scholar]
- Hu, S.; Jia, X.; Fu, Y. Research on Abnormal Behavior Detection of Online Examination Based on Image Information. In Proceedings of the 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics(HMSC), Hangzhou, China, 25–26 August 2018. [Google Scholar]
- Tejaswi, P.; Venkatramaphanikumar, S.; Kishore, K.V.K. Proctor net: An AI framework for suspicious activity detection in online proctored examinations. Measurement 2023, 206, 112266. [Google Scholar] [CrossRef]
- Motwani, S.; Nagpal, C.; Motwani, M. AI-based proctoring system for online tests. In Proceedings of the 4th International Conference on Advances in Science & Technology (ICAST), Sion, Mumbai, 7 May 2021. [Google Scholar]
- Prathish, S.; Bijlani, A.N.S.K. An intelligent system for online exam monitoring. In Proceedings of the 2016 International Conference on Information Science (ICIS), Kochi, India, 12–13 August 2016. [Google Scholar]
- Elshafey, A.E.; Anany, M.R.; Mohamed, A.S. Dr. Proctor: A Multi-modal AI-Based Platform for Remote Proctoring in Education. In Proceedings of the Artificial Intelligence in Education: 22nd International Conference(AIED), Utrecht, The Netherlands, 14–18 June 2021. [Google Scholar]
- Jia, J.; He, Y. The design, implementation and pilot application of an intelligent online proctoring system for online exams. Interact. Technol. Smart Educ. 2022, 19, 112–120. [Google Scholar] [CrossRef]
- Tweissi, A.; Etaiwi, W.; Eisawi, D. The Accuracy of AI-Based Automatic Proctoring in Online Exams. Electron. J. E-Learn. 2022; 20, 419–435. [Google Scholar]
- Maniar, S.; Sukhani, K.; Shah, K. Automated Proctoring System using Computer Vision Techniques. In Proceedings of the 2021 International Conference on System, Computation, Automation and Networking (ICSCAN), Puducherry, India, 30–31 July 2021. [Google Scholar]
- Soltane, M.; Laouar, M.R. A Smart System to Detect Cheating in the Online Exam. In Proceedings of the 2021 International Conference on Information Systems and Advanced Technologies (ICISAT), Tebessa, Algeria, 27–28 December 2021. [Google Scholar]
- Gopane, S.; Kotecha, R. Enhancing Monitoring in Online Exams Using Artificial Intelligence. In Proceedings of the International Conference on Data Science and Applications (ICDSA), Kolkata, India, 10–11 April 2021. [Google Scholar]
- Ozdamli, F.; Aljarrah, A.; Karagozlu, D. Facial Recognition System to Detect Student Emotions and Cheating in Distance Learning. Sustainability 2022, 14, 13230. [Google Scholar] [CrossRef]
- Malhotra, N.; Suri, R.; Verma, P. Smart Artificial Intelligence Based Online Proctoring System. In Proceedings of the 2022 IEEE Delhi section conference (DELCON), New Delhi, India, 11–13 February 2022. [Google Scholar]
- Abozaid, A.; Atia, A. Multi-Modal Online Exam Cheating Detection. In Proceedings of the 2022 International Conference on Electrical, Computer and Energy Technologies(ICECET), Prague, Czech Republic, 20–22 July 2022. [Google Scholar]
- Ahmad, I.; AlQurashi, F.; Abozinadah, E. A novel deep learning-based online proctoring system using face recognition, eye blinking, and object detection techniques. Int. J. Adv. Comput. Sci. Appl. 2021, 12. [Google Scholar] [CrossRef]
- Ashwinkumar, J.S.; Kumaran, H.S.; Sivakarthikeyan, U.; Rajesh, K.P.; Lavanya, R. Deep Learning based Approach for Facilitating Online Proctoring using Transfer Learning. In Proceedings of the 2021 5th International Conference on Computer, Communication and Signal Processing(ICCCSP), Chennai, India, 24–25 May 2021. [Google Scholar]
- Ozgen, A.C.; Öztürk, M.U.; Torun, O. Cheating Detection Pipeline for Online Interviews. In Proceedings of the 2021 29th Signal Processing and Communications Applications Conference(SIU), Istanbul, Turkey, 9–11 June 2021. [Google Scholar]
- Sapre, S.; Shinde, K.; Shetta, K. AI-ML based smart online examination framework. In Progresses in Artificial Intelligence & Robotics: Algorithms & Applications, Proceedings of 3rd International Conference on Deep Learning, Artificial Intelligence and Robotics (ICDLAIR), Virtual, 10 April 2022; Springer: Cham, Switzerland, 2022. [Google Scholar]
- Fan, Z.; Xu, J.; Liu, W. Gesture based misbehavior detection in online examination. In Proceedings of the 2016 11th International Conference on Computer Science & Education(ICCSE), Nagoya, Japan, 23–25 August 2016. [Google Scholar]
- Li, H.; Xu, M.; Wang, Y. A visual analytics approach to facilitate the proctoring of online exams. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (HFCS), Online, 8–13 May 2021. [Google Scholar]
- Pandey, A.K.; Kumar, S.; Rajendran, B. e-Parakh: Unsupervised Online Examination System. In Proceedings of the IEEE Region 10 International Conference (TENCON), Osaka, Japan, 16–19 November 2020. [Google Scholar]
- Li, X.; Chang, K.M.; Yuan, Y. Massive open online proctor: Protecting the credibility of MOOCs certificates. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, Vancouver BC, Canada, 14–18 March 2015. [Google Scholar]
- Liu, Z.; Lin, Y.; Cao, Y. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10–17 October 2021. [Google Scholar]
- Hou, Q.; Zhou, D.; Feng, J. Coordinate Attention for Efficient Mobile Network Design. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021. [Google Scholar]
- Wang, C.Y.; Mark Liao, H.Y.; Wu, Y.H. CSPNet: A New Backbone that can Enhance Learning Capability of CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 14–19 June 2020. [Google Scholar]
- Xie, S.; Girshick, R.; Dollár, P. Aggregated Residual Transformations for Deep Neural Networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Zhang, X.; Sugano, Y.; Fritz, M. MPIIGaze: Real-world dataset and deep appearance-based gaze estimation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 1, 162–175. [Google Scholar] [CrossRef] [PubMed]
Network Output | Definition of Candidate’s Behavior | |
---|---|---|
Gaze Direction | Cheating Tools | |
Y1 (cheating) | Desktop (left/right/downleft/down/downright) | Paper |
Y2 (cheating) | Desktop (left/right/downleft/down/downright) | Book |
Y3 (cheating) | Desktop (left/right/downleft/down/downright) | Phone |
Y4 (cheating) | Behind computer (left/right/downleft/downright) | Paper |
Y5 (cheating) | Behind computer (left/right/downleft/downright) | Book |
Y6 (cheating) | Behind computer (left/right/downleft/downright) | Phone |
Y7 (cheating) | Desktop (left/right/downleft/down/downright) | Paper, Book, Phone |
Y8 (cheating) | Behind computer (left/right/downleft/downright) | Paper, Book, Phone |
Y9 (normal) | Left/right/downleft/down/downright/normal/up/upleft/upright | No |
Network Output | Definition of Candidate’s Behavior | |
---|---|---|
Gaze Direction | Cheating Tools | |
Y1 (cheating) | Horizontal perspective (upleft/up/upright) | People and paper |
Y2 (cheating) | Horizontal perspective (upleft/up/upright) | People and book |
Y3 (cheating) | Horizontal perspective (upleft/up/upright) | People and phone |
Y4 (normal) | Left/right/downleft/down/downright/center/up/upleft/upright | No |
Gaze Direction | Cheating Behavior Determination Model |
---|---|
Upleft/up/upright | Glasses |
Left/right/downleft/down/downright | Desktop |
Type of Gaze Direction | Gaze Direction | Accuracy | Type of Gaze Direction | Gaze Direction | Accuracy |
---|---|---|---|---|---|
Abnormal (overhead Perspective) | Left | 90% | Abnormal (horizontal perspective) | Upleft | 100% |
Right | 88% | Up | 100% | ||
Down | 100% | Upright | 92% | ||
Downleft | 83% | Normal | Normal | 100% | |
Downright | 79% | Average | 92.4% |
Perspective | Indication | Model | Book | Paper | People | Phone | Average |
---|---|---|---|---|---|---|---|
Overhead perspective | mAP50 | YOLOv5 | 99.3% | 99.3% | 99.5% | 91% | 97.3% |
Lightweight-YOLOv5-CA | 99.5% | 99.3% | 99.5% | 91.5% | 97.4% | ||
mAP50:90 | YOLOv5 | 89% | 88.6% | 88.9% | 71.2% | 84.6% | |
Lightweight-YOLOv5-CA | 89.8% | 89.37% | 89.1% | 71.6% | 85.1% | ||
Horizontal perspective | mAP50 | YOLOv5 | 99.2% | 99.3% | 99.4% | 97% | 98.7% |
Lightweight-YOLOv5-CA | 99.3% | 99.3% | 99.4% | 97.1% | 98.8% | ||
mAP50:90 | YOLOv5 | 88.7% | 88.4% | 88.8% | 75.4% | 85.3% | |
Lightweight-YOLOv5-CA | 89.7% | 89.6% | 89% | 77.4% | 86.4% |
Method | Monitor Perspective | Overhead Object Detection | Gaze Recognition | Horizontal Object Detection | Cheating Behavior Determination |
---|---|---|---|---|---|
Ref. [38] | Overhead and Horizontal | 89.1% | × | -- | 81.1% |
Ref. [12] | Face and Horizontal | × | 82.9% | 89% | 65.3% |
Ref. [11] | Face and Overhead | 91.87% | 68.9% | × | 81.83% |
Ref. [28] | Face and Overhead | 60% | 89.95% | × | -- |
Ref. [10] | Face and Overhead | -- | -- | × | 83.56% |
Ours | Overhead and Face and Horizontal | 92.4% | 98.8% | 95.7% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hu, Z.; Jing, Y.; Wu, G.; Wang, H. Multi-Perspective Adaptive Paperless Examination Cheating Detection System Based on Image Recognition. Appl. Sci. 2024, 14, 4048. https://doi.org/10.3390/app14104048
Hu Z, Jing Y, Wu G, Wang H. Multi-Perspective Adaptive Paperless Examination Cheating Detection System Based on Image Recognition. Applied Sciences. 2024; 14(10):4048. https://doi.org/10.3390/app14104048
Chicago/Turabian StyleHu, Zuhui, Yaguang Jing, Guoqing Wu, and Han Wang. 2024. "Multi-Perspective Adaptive Paperless Examination Cheating Detection System Based on Image Recognition" Applied Sciences 14, no. 10: 4048. https://doi.org/10.3390/app14104048
APA StyleHu, Z., Jing, Y., Wu, G., & Wang, H. (2024). Multi-Perspective Adaptive Paperless Examination Cheating Detection System Based on Image Recognition. Applied Sciences, 14(10), 4048. https://doi.org/10.3390/app14104048