Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning
Abstract
:1. Introduction
- The study collects a comprehensive video dataset to classify different cricket strokes. In contrast to previous studies that only use image datasets and cover a maximum of five strokes, this study covers eight strokes, including ‘flick’, ‘back foot punch’, ‘pull’, ‘cut’, ‘cover drive’, ‘straight drive’, ‘on drive’, and ‘sweep’.
- A novel technique is employed to extract features from the video dataset. The MediaPipe library extracts seventeen critical points of the human body. Based on these key points, the batsman’s stroke is accurately classified.
- The study uses fine-tuned machine learning and deep learning models to classify the strokes based on the extracted feature dataset. Cross-validation is employed to validate the model’s performance, ensuring accurate results.
- This research provides a more comprehensive and accurate approach to classifying cricket strokes. The novel technique that extracts features from video datasets and utilizes state-of-the-art machine learning and deep learning models helps improve classification accuracy.
2. Related Work
3. Proposed Methodology
3.1. Video Cricket Strokes Dataset
3.2. Feature Extraction from Videos
Algorithm 1 Batsmen stroke prediction. |
Input: Video strokes dataset (VSD) |
Output: Stroke prediction {cover driver, pull, sweep, state drive, on drive, cut and back foot punch}
|
3.3. Cricket Stroke Exploratory Data Analysis
3.4. Target Label Encoding
3.5. Dataset Splitting
3.6. Model Training
3.7. Performance Metrics
4. Results and Discussion
4.1. Results for Machine and Deep Learning Models
4.2. K-Fold Cross-Validation Analysis
4.3. Time Complexity
4.4. Performance Comparison
4.5. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Desmarais, Y.; Mottet, D.; Slangen, P.; Montesinos, P. A review of 3D human pose estimation algorithms for markerless motion capture. Comput. Vis. Image Underst. 2021, 212, 103275. [Google Scholar] [CrossRef]
- Pavllo, D.; Feichtenhofer, C.; Grangier, D.; Auli, M. 3d human pose estimation in video with temporal convolutions and semi-supervised training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 7753–7762. [Google Scholar]
- Nadeem, A.; Jalal, A.; Kim, K. Automatic human posture estimation for sport activity recognition with robust body parts detection and entropy markov model. Multimed. Tools Appl. 2021, 80, 21465–21498. [Google Scholar] [CrossRef]
- Song, L.; Yu, G.; Yuan, J.; Liu, Z. Human pose estimation and its application to action recognition: A survey. J. Vis. Commun. Image Represent. 2021, 76, 103055. [Google Scholar] [CrossRef]
- Mili, R.; Das, N.R.; Tandon, A.; Mokhtar, S.; Mukherjee, I.; Paul, G. Pose Recognition in Cricket using Keypoints. In Proceedings of the 2022 IEEE 9th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), Islamabad, Pakistan, 12–16 January 2021; pp. 1–5. [Google Scholar]
- Vestly, D.J.; Hariharan, S.; Kukreja, V.; Prasad, A.B.; Swaraj, K.; Gopichand, D. Parametric Analysis of a Cricketer’s Performance using Machine Learning Approach. In Proceedings of the 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 17–19 May 2023; pp. 344–348. [Google Scholar]
- Rupapara, V.; Rustam, F.; Aljedaani, W.; Shahzad, H.F.; Lee, E.; Ashraf, I. Blood cancer prediction using leukemia microarray gene data and hybrid logistic vector trees model. Sci. Rep. 2022, 12, 1000. [Google Scholar] [CrossRef] [PubMed]
- Umer, M.; Ashraf, I.; Ullah, S.; Mehmood, A.; Choi, G.S. COVINet: A convolutional neural network approach for predicting COVID-19 from chest X-ray images. J. Ambient. Intell. Humaniz. Comput. 2022, 1–13. [Google Scholar] [CrossRef]
- Rustam, F.; Siddique, M.A.; Siddiqui, H.U.R.; Ullah, S.; Mehmood, A.; Ashraf, I.; Choi, G.S. Wireless capsule endoscopy bleeding images classification using CNN based model. IEEE Access 2021, 9, 33675–33688. [Google Scholar] [CrossRef]
- Rupapara, V.; Rustam, F.; Amaar, A.; Washington, P.B.; Lee, E.; Ashraf, I. Deepfake tweets classification using stacked Bi-LSTM and words embedding. PeerJ Comput. Sci. 2021, 7, e745. [Google Scholar] [CrossRef]
- Mehmood, A.; On, B.W.; Lee, I.; Ashraf, I.; Sang Choi, G. Spam comments prediction using stacking with ensemble learning. In Journal of Physics: Conference Series; IOP Publishing: Bristol, UK, 2018; Volume 933, p. 012012. [Google Scholar]
- Aslam, S.; Ashraf, I. Data mining algorithms and their applications in education data mining. Int. J. 2014, 2, 50–56. [Google Scholar]
- Rashid, A.; Asif, S.; Butt, N.A.; Ashraf, I. Feature level opinion mining of educational student feedback data using sequential pattern mining and association rule mining. Int. J. Comput. Appl. 2013, 81, 31–38. [Google Scholar] [CrossRef]
- Saad, E.; Din, S.; Jamil, R.; Rustam, F.; Mehmood, A.; Ashraf, I.; Choi, G.S. Determining the efficiency of drugs under special conditions from users’ reviews on healthcare web forums. IEEE Access 2021, 9, 85721–85737. [Google Scholar] [CrossRef]
- Richter, C.; O’Reilly, M.; Delahunt, E. Machine learning in sports science: Challenges and opportunities. Sport. Biomech. 2021, 1–7. [Google Scholar] [CrossRef] [PubMed]
- Keshtkar Langaroudi, M.; Yamaghani, M. Sports result prediction based on machine learning and computational intelligence approaches: A survey. J. Adv. Comput. Eng. Technol. 2019, 5, 27–36. [Google Scholar]
- Yang, W. Analysis of sports image detection technology based on machine learning. Eurasip J. Image Video Process. 2019, 2019, 17. [Google Scholar] [CrossRef] [Green Version]
- Moodley, T.; van der Haar, D. Cricket stroke recognition using computer vision methods. In Information Science and Applications: ICISA 2019; Springer: Berlin/Heidelberg, Germany, 2019; pp. 171–181. [Google Scholar]
- Devanandan, M.; Rasaratnam, V.; Anbalagan, M.K.; Asokan, N.; Panchendrarajan, R.; Tharmaseelan, J. Cricket Shot Image Classification Using Random Forest. In Proceedings of the 2021 3rd International Conference on Advancements in Computing (ICAC), Colombo, Sri Lanka, 9–11 December 2021; pp. 425–430. [Google Scholar]
- Bandara, I.; Bačić, B. Strokes classification in cricket batting videos. In Proceedings of the 2020 5th International Conference on Innovative Technologies in Intelligent Systems and Industrial Applications (CITISIA), Sydney, Australia, 25–27 November 2020; pp. 1–6. [Google Scholar]
- Karmaker, D.; Chowdhury, A.; Miah, M.; Imran, M.; Rahman, M. Cricket shot classification using motion vector. In Proceedings of the 2015 Second International Conference on Computing Technology and Information Management (ICCTIM), Johor, Malaysia, 21–23 April 2015; pp. 125–129. [Google Scholar]
- Yip, Z.Y.; Khairuddin, I.M.; Isa, W.H.M.; Majeed, A.P.A.; Abdullah, M.A.; Razman, M.A.M. Badminton Smashing Recognition through Video Performance by using Deep Learning. MEKATRONIKA 2022, 4, 70–79. [Google Scholar] [CrossRef]
- Luo, J.; Hu, Y.; Davids, K.; Zhang, D.; Gouin, C.; Li, X.; Xu, X. Vision-based movement recognition reveals badminton player footwork using deep learning and binocular positioning. Heliyon 2022, 8, e10089. [Google Scholar] [CrossRef]
- Kulkarni, K.M.; Shenoy, S. Table tennis stroke recognition using two-dimensional human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 25 June 2021; pp. 4576–4584. [Google Scholar]
- Ahmad, Z.; Illanko, K.; Khan, N.; Androutsos, D. Human action recognition using convolutional neural network and depth sensor data. In Proceedings of the 2019 International Conference on Information Technology and Computer Communications, Paris, France, 29 April–2 May 2019; pp. 1–5. [Google Scholar]
- Lazarescu, M.; Venkatesh, S.; West, G. Classifying and learning cricket shots using camera motion. In Proceedings of the Advanced Topics in Artificial Intelligence: 12th Australian Joint Conference on Artificial Intelligence, AI’99, Sydney, Australia, 6–10 December 1999; Proceedings 12; Springer: Berlin/Heidelberg, Germany, 1999; pp. 13–23. [Google Scholar]
Refs. | Dataset | Strokes | Technique | Accuracy |
---|---|---|---|---|
[18] | Images | Glance, drive, block, and cut | AlexNet | 74.33% |
[19] | Images | Cut, cover drive, straight drive, pull, leg glance, scoop | Random forest | 87% |
[20] | Videos | Backward and forward | LSTM | 100% |
[18] | Videos | Strokes and gameplay | AlexNet | 96.66 |
Attribute | Dtype | Attribute | Dtype | Attribute | Dtype |
---|---|---|---|---|---|
nosex | float | lshoulderx | float64 | nosey | float64 |
lshouldery | float64 | nosez | float64 | lshoulderz | float64 |
rshoulderx | float64 | lelbowx | float64 | rshouldery | float64 |
lelbowy | float64 | rshoulderz | float64 | lelbowz | float64 |
relbowx | float64 | rWristx | float64 | relbowy | float64 |
rWristy | float64 | relbowz | float64 | rWristz | float64 |
lWristx | float64 | rhipx | float64 | lWristy | float64 |
rhipy | float64 | lWristz | float64 | rhipz | float64 |
lhipx | float64 | rkneex | float64 | lhipy | float64 |
rkneey | float64 | lhipz | float64 | rkneez | float64 |
lkneex | float64 | rankelx | float64 | lkneey | float64 |
rankely | float64 | lkneez | float64 | rankelz | float64 |
lankelx | float64 | rheelx | float64 | lankely | float64 |
rheely | float64 | lankelz | float64 | rheelz | float64 |
lheelx | float64 | lfindexx | float64 | lheely | float64 |
lfindexy | float64 | lheelz | float64 | lfindexz | float64 |
rfindexx | float64 | rfindexy | float64 | rfindexz | float64 |
Strokes | Records | Percentage |
---|---|---|
State Drive | 1060 | 11.78% |
On Drive | 2276 | 25.29% |
Cover Drive | 1236 | 13.74% |
Cut | 1011 | 11.24% |
Pull | 779 | 8.66% |
Sweep | 511 | 5.68% |
Flick | 908 | 10.09% |
Backfoot Punch | 1217 | 13.53% |
Model | Hyperparameters |
---|---|
LSTM | loss = ‘categorical_crossentropy’, optimizer = ‘adam’, metrics = ‘accuracy’, activation=‘softmax’, batch_size = 64, validation_split = 0.1, epoch = 10 |
KNN | n_neighbors = 2 |
LR | C = 0.1, intercept_scaling = 10, random_state = 100 |
DT | random_state = 0, max_depth = 300 |
SVM | decision_function_shape = ‘ovo’, probability = True |
RF | random_state = 0, max_depth = 300 |
Model | Precision | Recall | F1 Score | ||||||
---|---|---|---|---|---|---|---|---|---|
70:30 | 80:20 | 90:10 | 70:30 | 80:20 | 90:10 | 70:30 | 80:20 | 90:10 | |
LSTM | 0.409 | 0.452 | 0.470 | 0.476 | 0.502 | 0.527 | 0.476 | 0.502 | 0.527 |
LR | 0.658 | 0.655 | 0.649 | 0.637 | 0.643 | 0.651 | 0.637 | 0.643 | 0.651 |
DT | 0.948 | 0.951 | 0.970 | 0.947 | 0.956 | 0.968 | 0.947 | 0.956 | 0.968 |
SVM | 0.863 | 0.867 | 0.873 | 0.842 | 0.846 | 0.857 | 0.842 | 0.846 | 0.857 |
KNN | 0.982 | 0.989 | 0.988 | 0.984 | 0.989 | 0.989 | 0.984 | 0.989 | 0.989 |
RF | 0.996 | 0.998 | 0.996 | 0.995 | 0.998 | 0.997 | 0.995 | 0.998 | 0.997 |
Method | Cohen Kappa Score | Geometric Mean Score | Log Loss | Accuracy | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
70:30 | 80:20 | 90:10 | 70:30 | 80:20 | 90:10 | 70:30 | 80:20 | 90:10 | 70:30 | 80:20 | 90:10 | |
LSTM | 0.372 | 0.413 | 0.431 | 0.526 | 0.50 | 0.475 | 0.0 | 0.0 | 0.0 | 14.09 | 13.26 | 12.82 |
LR | 0.562 | 0.570 | 0.584 | 0.636 | 0.643 | 0.651 | 0.452 | 0.457 | 0.465 | 1.09 | 1.089 | 1.067 |
DT | 0.938 | 0.947 | 0.962 | 0.947 | 0.955 | 0.967 | 0.941 | 0.955 | 0.969 | 1.895 | 1.601 | 1.161 |
SVM | 0.812 | 0.817 | 0.831 | 0.841 | 0.845 | 0.856 | 0.823 | 0.829 | 0.848 | 0.449 | 0.438 | 0.420 |
KNN | 0.981 | 0.986 | 0.987 | 0.984 | 0.988 | 0.988 | 0.983 | 0.988 | 0.990 | 0.358 | 0.231 | 1.709 |
RF | 0.994 | 0.997 | 0.996 | 0.995 | 0.997 | 0.996 | 0.994 | 0.997 | 0.997 | 0.086 | 0.076 | 0.06 |
Model | K-Fold Accuracy | Standard Deviation (±) |
---|---|---|
LSTM | 0.115 | 0.172 |
LR | 0.63 | 0.12 |
DT | 0.87 | 0.10 |
SVM | 0.83 | 0.13 |
KNN | 0.94 | 0.06 |
RF | 0.95 | 0.07 |
Model | Time Computation (s) | ||
---|---|---|---|
70:30 | 80:20 | 90:10 | |
LSTM | 43.807 | 84.171 | 90.34 |
LR | 0.914 | 0.518 | 0.649 |
DT | 0.542 | 0.453 | 0.776 |
SVM | 8.571 | 11.14 | 13.35 |
KNN | 0.008 | 0.012 | 0.014 |
RF | 7.542 | 5.174 | 5.695 |
Refs. | Strokes | Strokes | Model | Accuracy |
---|---|---|---|---|
[18] | Glance, drive, block, and cut | 4 | AlexNet | 74.33% |
[19] | Cut, cover drive, straight drive, pull, leg glance, scoop | 6 | RF | 87% |
[20] | Backward and forward | 2 | LSTM | 100% |
[18] | Strokes and gameplay | 2 | AlexNet | 96.66% |
This study | Straight drive, on drive, cover driver, cut, pull, sweep, flick, back foot punch | 8 | RF | 99.7% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Siddiqui, H.U.R.; Younas, F.; Rustam, F.; Flores, E.S.; Ballester, J.B.; Diez, I.d.l.T.; Dudley, S.; Ashraf, I. Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning. Sensors 2023, 23, 6839. https://doi.org/10.3390/s23156839
Siddiqui HUR, Younas F, Rustam F, Flores ES, Ballester JB, Diez IdlT, Dudley S, Ashraf I. Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning. Sensors. 2023; 23(15):6839. https://doi.org/10.3390/s23156839
Chicago/Turabian StyleSiddiqui, Hafeez Ur Rehman, Faizan Younas, Furqan Rustam, Emmanuel Soriano Flores, Julién Brito Ballester, Isabel de la Torre Diez, Sandra Dudley, and Imran Ashraf. 2023. "Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning" Sensors 23, no. 15: 6839. https://doi.org/10.3390/s23156839
APA StyleSiddiqui, H. U. R., Younas, F., Rustam, F., Flores, E. S., Ballester, J. B., Diez, I. d. l. T., Dudley, S., & Ashraf, I. (2023). Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning. Sensors, 23(15), 6839. https://doi.org/10.3390/s23156839