Convolutional Neural Network Defect Detection Algorithm for Wire Bonding X-ray Images
Abstract
:1. Introduction
- Based on the C2F module in YOLOv8, we propose a module (C2SF) that can fully extract semantic information at different gradients and construct a feature extraction backbone network with this module.
- We propose a self-adaptive weighted multi-scale feature fusion module (SMA) with different dilation rates to expand the receptive field to predict multi-scale defects. In the SMA module, different weights are given based on the contribution of feature information at different scales to the detection results, thereby making full use of the feature information.
- To meet the requirements of wire bonding defect detection, we designed a new defect detection network framework YOLO-CSS based on the C2SF module and the SMA module. This network significantly improves detection accuracy and speed while maintaining detection speed.
2. Related Works
2.1. Image Data Acquisition
2.2. Description of the Dataset
3. Method
3.1. YOLO-CSS Framework
3.2. Backbone Structure of YOLO-CSS
3.3. Neck Network Based on Weighted Multi-Scale Feature Fusion
4. Experiments
4.1. Model Training Details
4.2. The Validation of the Effectiveness of the SMA Module
4.3. Model Evaluation Metrics
4.4. Ablation Experiment
4.5. Comparison of Results from Different Attention Mechanisms
5. Conclusions
- We will continue to explore deep-learning-based object detection algorithms for few-shot learning, aiming to overcome the challenges in acquiring chip wire bonding defect images in practical production scenarios.
- To address the issue of image blurring and indistinct defects caused by low-dose image reconstruction algorithms, we will combine image preprocessing techniques with the yolo-css detection model to improve the accuracy of defect localization and recognition.
- Enhance the algorithm’s generalization capability to detect various types of chip wire bonding defects, thereby reducing production costs in real-world scenarios.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Alam, L.; Kehtarnavaz, N. A survey of detection methods for die attachment and wire bonding defects in integrated circuit manufacturing. IEEE Access 2022, 10, 83826–83840. [Google Scholar] [CrossRef]
- Park, Y.-J.; Ahn, G.; Hur, S. Optimization of pick-and-place in die attach process using a genetic algorithm. Appl. Soft Comput. 2018, 68, 856–865. [Google Scholar] [CrossRef]
- Chen, J.; Zhang, Z.; Wu, F. A data-driven method for enhancing the image-based automatic inspection of IC wire bonding defects. Int. J. Prod. Res. 2021, 59, 4779–4793. [Google Scholar] [CrossRef]
- Perng, D.-B.; Chou, C.-C.; Lee, S.-M. Design and development of a new machine vision wire bonding inspection system. Int. J. Adv. Manuf. Technol. 2007, 34, 323–334. [Google Scholar] [CrossRef]
- Kovac, O.; Girasek, T.; Pietrikova, A. Image processing of die attach’s X-ray images for automatic voids detection and evaluation. In Proceedings of the 2016 39th International Spring Seminar on Electronics Technology (ISSE), Pilsen, Czech Republic, 18–22 May 2016; pp. 199–203. [Google Scholar]
- Liao, G.; Chen, P.; Du, L.; Su, L.; Liu, Z.; Tang, Z.; Shi, T. Using SOM neural network for X-ray inspection of missing-bump defects in three-dimensional integration. Microelectron. Reliab. 2015, 55, 2826–2832. [Google Scholar] [CrossRef]
- Fan, M.; Wei, L.; He, Z.; Wei, W.; Lu, X. Defect inspection of solder bumps using the scanning acoustic microscopy and fuzzy SVM algorithm. Microelectron. Reliab. 2016, 65, 192–197. [Google Scholar] [CrossRef]
- Ngan, K.N. Automated Inspection of IC Bonding Wires Using Hough Transforf4. In Proceedings of the Proceedings.14 Annual Conference of Industrial Electronics Society, Singapore, 24–28 October 1988; pp. 938–942. [Google Scholar]
- Wang, M.-J.J.; Wu, W.-Y.; Hsu, C.-C. Automated post bonding inspection by using machine vision techniques. Int. J. Prod. Res. 2002, 40, 2835–2848. [Google Scholar] [CrossRef]
- Perng, D.-B.; Lee, S.-M.; Chou, C.-C. Automated bonding position inspection on multi-layered wire IC using machine vision. Int. J. Prod. Res. 2010, 48, 6977–7001. [Google Scholar] [CrossRef]
- Chen, P.; Chen, M.; Wang, S.; Song, Y.; Cui, Y.; Chen, Z.; Zhang, Y.; Chen, S.; Mo, X. Real-time defect detection of TFT-LCD displays using a lightweight network architecture. J. Intell. Manuf. 2023. [Google Scholar] [CrossRef]
- Dimitriou, N.; Leontaris, L.; Vafeiadis, T.; Ioannidis, D.; Wotherspoon, T.; Tinker, G.; Tzovaras, D. A deep learning framework for simulation and defect prediction applied in microelectronics. Simul. Model. Pract. Theory 2020, 100, 102063. [Google Scholar] [CrossRef]
- Wu, H.; Gao, W.; Xu, X. Solder joint recognition using mask R-CNN method. IEEE Trans. Compon. Packag. Manuf. Technol. 2019, 10, 525–530. [Google Scholar] [CrossRef]
- Shu, Y.; Li, B.; Lin, H. Quality safety monitoring of LED chips using deep learning-based vision inspection methods. Measurement 2021, 168, 108123. [Google Scholar] [CrossRef]
- Wang, S.; Wang, H.; Yang, F.; Liu, F.; Zeng, L. Attention-based deep learning for chip-surface-defect detection. Int. J. Adv. Manuf. Technol. 2022, 121, 1957–1971. [Google Scholar] [CrossRef]
- Stern, M.L.; Schellenberger, M. Fully convolutional networks for chip-wise defect detection employing photoluminescence images: Efficient quality control in LED manufacturing. J. Intell. Manuf. 2021, 32, 113–126. [Google Scholar] [CrossRef]
- Huang, H.; Tang, X.; Wen, F.; Jin, X. Small object detection method with shallow feature fusion network for chip surface defect detection. Sci. Rep. 2022, 12, 3914. [Google Scholar] [CrossRef]
- Dong, H.; Song, K.; He, Y.; Xu, J.; Yan, Y.; Meng, Q. PGA-Net: Pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Trans. Ind. Inform. 2019, 16, 7448–7458. [Google Scholar] [CrossRef]
- Wang, W.; Lu, X.; He, Z.; Shi, T. Using convolutional neural network for intelligent SAM inspection of flip chips. Meas. Sci. Technol. 2021, 32, 115022. [Google Scholar] [CrossRef]
- Hermanek, P.; Rathore, J.S.; Aloisi, V.; Carmignato, S. Principles of X-ray computed tomography. In Industrial X-ray Computed Tomography; Springer: Cham, Switzerland, 2018; pp. 25–67. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.-Y.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 618–626. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 11534–11542. [Google Scholar]
- Hou, Q.; Zhou, D.; Feng, J. Coordinate attention for efficient mobile network design. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 13713–13722. [Google Scholar]
- Yu, G.; Chang, Q.; Lv, W.; Xu, C.; Cui, C.; Ji, W.; Dang, Q.; Deng, K.; Wang, G.; Du, Y. PP-PicoDet: A better real-time object detector on mobile devices. arXiv 2021, arXiv:2111.00902. [Google Scholar]
- Liu, M.; Chen, Y.; Xie, J.; He, L.; Zhang, Y. LF-YOLO: A lighter and faster yolo for weld defect detection of X-ray image. IEEE Sens. J. 2023, 23, 7430–7439. [Google Scholar] [CrossRef]
- Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. Scaled-yolov4: Scaling cross stage partial network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 13029–13038. [Google Scholar]
- Zhu, X.; Lyu, S.; Wang, X.; Zhao, Q. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Nashville, TN, USA, 20–25 June 2021; pp. 2778–2788. [Google Scholar]
- Wang, C.-Y.; Yeh, I.-H.; Liao, H.-Y.M. You only learn one representation: Unified network for multiple tasks. arXiv 2021, arXiv:2105.04206. [Google Scholar]
- Redmon, J.; Farhadi, A. Yolov3: An incremental improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023; pp. 7464–7475. [Google Scholar]
- Yu, Z.; Huang, H.; Chen, W.; Su, Y.; Liu, Y.; Wang, X. Yolo-facev2: A scale and occlusion aware face detector. arXiv 2022, arXiv:2208.02019. [Google Scholar]
Attention | mAP 0.5 (%) | mAP 0.5–0.95 (%) | Precision (%) | Recall (%) | FPS |
---|---|---|---|---|---|
None | 95.0 | 68.0 | 90.6 | 91.3 | 139 |
SENet | 96.2 | 70.6 | 95.1 | 94.1 | 128 |
ECANet | 97.3 | 71.4 | 96.1 | 93.8 | 130 |
CA | 97.1 | 70.4 | 95.7 | 93.3 | 131 |
CBAM | 97.3 | 72.1 | 95.7 | 95.2 | 128 |
Algorithm | mAP 0.5 (%) | mAP 0.5–0.95 (%) | Precision (%) | Recall (%) | FPS | Size (MB) |
---|---|---|---|---|---|---|
LFYOLO [28] | 85.7 | 53.1 | 85.7 | 80.6 | 65.6 | 14.9 |
Yolov5-mobilenetv3 | 81.5 | 49.5 | 83.1 | 76.4 | 129.8 | 14.0 |
PicoDet | 52.6 | 26.5 | 54.4 | 52.6 | 110.2 | 2.98 |
Scaled_yolov4 [29] | 75.9 | 43.9 | 64.3 | 77.9 | 71.4 | 100.0 |
TPH-yolov5 [30] | 77.7 | 50.3 | 65.1 | 79.7 | 5.7 | 83.7 |
Yolor [31] | 79.8 | 58.9 | 40.9 | 88.7 | 87.6 | 285.1 |
Yolov3 [32] | 83.2 | 52.8 | 70.6 | 80.7 | 62.5 | 117.2 |
Yolov5 | 90.0 | 66.3 | 90.6 | 87.1 | 134.1 | 13.8 |
Yolov7 [33] | 59.0 | 31.0 | 67.5 | 52.9 | 20.3 | 71.3 |
Yolo_facev2 [34] | 62.8 | 30.6 | 56.7 | 63.0 | 131.6 | 16.6 |
Ours | 97.3 | 72.1 | 95.7 | 95.2 | 125.9 | 10.3 |
Algorithm | High Loop | Low Loop | Interconnect | Broken Wire | Wire Missing |
---|---|---|---|---|---|
Yolov5 | 88.1% | 91.7% | 89.0% | 89.1% | 93.1% |
Ours | 93.3% | 96.2% | 93.6% | 96.1% | 97.0% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhan, D.; Huang, R.; Yi, K.; Yang, X.; Shi, Z.; Lin, R.; Lin, J.; Wang, H. Convolutional Neural Network Defect Detection Algorithm for Wire Bonding X-ray Images. Micromachines 2023, 14, 1737. https://doi.org/10.3390/mi14091737
Zhan D, Huang R, Yi K, Yang X, Shi Z, Lin R, Lin J, Wang H. Convolutional Neural Network Defect Detection Algorithm for Wire Bonding X-ray Images. Micromachines. 2023; 14(9):1737. https://doi.org/10.3390/mi14091737
Chicago/Turabian StyleZhan, Daohua, Renbin Huang, Kunran Yi, Xiuding Yang, Zhuohao Shi, Ruinan Lin, Jian Lin, and Han Wang. 2023. "Convolutional Neural Network Defect Detection Algorithm for Wire Bonding X-ray Images" Micromachines 14, no. 9: 1737. https://doi.org/10.3390/mi14091737
APA StyleZhan, D., Huang, R., Yi, K., Yang, X., Shi, Z., Lin, R., Lin, J., & Wang, H. (2023). Convolutional Neural Network Defect Detection Algorithm for Wire Bonding X-ray Images. Micromachines, 14(9), 1737. https://doi.org/10.3390/mi14091737