Fully Automated Dispensing System Based on Machine Vision
Abstract
:1. Introduction
2. Dispensing Positioning Information Extraction
2.1. Dispensing Target Image Acquisition
2.2. Image Processing of PCB Boards
2.2.1. Grayscale Processing of PCB Boards
2.2.2. Filtering of PCB Boards
2.2.3. Binarization of the Dispensing Target
2.3. Dispensing Region Edge Extraction and Refinement
2.3.1. Sobel Operator-Based Edge Extraction of the Dispensing Region
- The 3 × 3 convolutional template is walked through the whole from top to bottom and from left to right on the collected dotted PCB template image, and the center point in the volume template corresponds to the pixel point in the image;
- The pixel value relative to each template composed of pixel points on the whole template image is derived by applying discrete convolution operation to each template;
- The maximum gradient value obtained from the horizontal and vertical convolution of the 3 × 3 convolutional template is substituted with the pixel value of the center pixel point;
- Finally, the appropriate threshold T is selected by many tests to carry out the recognition of the dotted area.
2.3.2. Hilditch Operator-Based Edge Refinement of the Dispensing Region
2.4. Extraction of Positioning Information Based on the Minimum Outer Rectangle Algorithm
2.4.1. Extracting the Aspect Ratio of the Dispensing Area
- Dispensing area boundary value determination.
- 2.
- Dispensing area boundary rotation.
- 3.
- Calculate the minimum external rectangle.
2.4.2. Extracting the Area of the Dispensing Area
2.4.3. Dispensing Area Center Coordinate Point Extraction
2.5. Dispensing Location Information Extraction Experiment
3. Vision Dispensing Camera Calibration and Path Planning
3.1. Dispensing Vision System Hand-Eye Calibration
3.2. Dispensing Motion Path Planning
3.2.1. Dispensing Motion Path Analysis
- Single dispensing object path analysis.
- 2.
- Analysis of single dispensing object dispensing direction and starting point.
- 3.
- Multi-dispensing position path analysis.
3.2.2. Modeling of Path Planning Based on Simulated Annealing Algorithm
4. Automatic Dispensing Platform Experiment and Verification Analysis
4.1. Dispensing Platform System
4.1.1. Analysis of the Overall Mechanical Structure
4.1.2. Analysis of the Vision Dispensing System
4.1.3. Overall Control Analysis
4.2. Hand-Eye Calibration Coordinate Data Extraction
4.3. Experimental Verification of Multi-Point Glue Object Path Planning and Analysis
4.4. Image Acquisition Recognition and Positioning Accuracy Experiment
- 1.
- Localization information extraction experiment
- 2.
- Image recognition coordinate position accuracy error analysis.
- The experiment randomly selects two PCBs into groups A and B.
- On the PCB board in group A, the collected image is pre-processed by grayscaling, filtering, binarization, etc., and then the edge extraction and refinement of the PCB board dispensing area are carried out. Finally, the coordinates of the center point of the dispensing area are repeatedly collected and extracted 10 times based on the minimum outer rectangle algorithm. The extracted coordinates are read out as (X1, Y1).
- Repeat the experimental steps in group B to obtain (X2, Y2).
- Calculate u(p) · X1, u(p) · Y1, u(p) · X12, and u(p) · Y2 by repeating the experimental standard deviation.
4.5. Path Planning Experiment and Verification
- Set up five groups; each group chooses 20 PCB boards randomly placed, and the position of the PCB boards placed between different groups cannot be the same.
- Start timing from dispensing; when the dispensing is finished for the last PCB board, end the timing.
- Record the time for dispensing before and after the path optimization is carried out. The final experiment can be derived from the dispensing time of different groups, as shown in Table 8. According to the above table, it can be concluded that the traditional dispensing path has a long dispensing time. After the path optimization, the dispensing efficiency improved by 20~30%.
4.6. Dispensing Quality Testing Experiment
5. Discussion and Conclusions
- Firstly, the collected PCB images are preprocessed through grayscale processing, median filtering, and binarization. The dispensing area is extracted, the edges of the dispensing area are extracted by using the Sobel edge detection operator, the extracted edges are operated by using the Hilditch algorithm, and, finally, the center coordinates point, angle, and area of the dispensing area are extracted by using the minimum outer rectangle algorithm.
- The pixel coordinates after image processing are converted to the actual dispensing coordinates of the dispensing needle using hand-eye calibration. Analysis of the single PCB board dispensing path and dispensing direction, the overall path planning for 20 PCB boards, and the use of MATLAB software through the simulated annealing algorithm based on the optimization of the dispensing path were compared with the traditional dispensing path; the optimized total length of the dispensing path and the running time have been significantly reduced.
- For the overall program design, the dispensing experimental platform was constructed according to the design of the actual dispensing experiments. For the complete dispensing process, 20 PCB boards were used. For the host computer, image acquisition processing and coordinate conversion total time was 217.6 ms; from the first board to the last board, dispensing was completed in a time of 109 s. The entire dispensing platform total time was 126.8 s. Finally, dispensing was completed, and the dispensing area measurement was performed. Dispensing package accuracy was within ±0.5 mm; the optimized dispensing efficiency was improved. After optimization, the dispensing efficiency was improved by 20~30%.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Liu, H.; Chen, X.; Sun, X.; Gao, Q.; Qiao, K. A pL-fL grade micro-dispensing by pipetting needle glue liquid transfer. AIP Adv. 2021, 11, 065119. [Google Scholar] [CrossRef]
- Zha, G.F. Research on Dispensing Defect Based on Machine Vision. Master’s Thesis, HITSZ, Shenzhen, China, 2020. [Google Scholar] [CrossRef]
- Nelson, A. Machine Vision Trends for Today’s Industrial Age. Quality 2020, 59, 26VS–29VS. [Google Scholar]
- Scimeca, D. Machine vision’s hottest technologies and most popular applications. Vis. Syst. Des. 2021, 26, 10–12. [Google Scholar]
- Wu, X.; Song, J. Research on visual detection and control system of three axis servo dispensing machine. Mach. Des. Manuf. 2020, 5, 237–240. [Google Scholar] [CrossRef]
- Li, Y.; Zhang, D.; Xu, D. High precision assembly and efficient dispensing approaches for millimeter objects. Int. J. Adv. Robot. Syst. 2019, 16, 1729881419857393. [Google Scholar] [CrossRef]
- Xing-Wei, Z.; Ke, Z.; Xie, L.-W.; Yong-Jie, Z.; Xin-Jian, L. An Enhancement and Detection Method for a Glue Dispensing Image Based on the CycleGAN Model. IEEE Access 2022, 10, 92036–92047. [Google Scholar] [CrossRef]
- Wang, N.; Liu, J.; Wei, S.; Zhang, X. A Vision Location System Design of Glue Dispensing Robot. In Intelligent Robotics and Applications; Springer: Cham, Switzerland, 2015. [Google Scholar] [CrossRef]
- Cheng, F.; Zhang, X.; Zhang, J.S. A positioning system for dispensing machine based on machine-vision. Mach. Des. Manuf. 2013, 3, 101–104. [Google Scholar] [CrossRef]
- Chu, C.H.; Liu, Y.W.; Li, P.C.; Huang, L.C.; Luh, Y.P. Programming by Demonstration in Augmented Reality for the Motion Planning of a Three-Axis CNC Dispenser. Int. J. Precis. Eng. Manuf.-Green Technol. 2020, 7, 987–995. [Google Scholar] [CrossRef]
- Mao, L.; Lian, W.H.; Fan, Z.Q.; Zhu, L.; Hu, J.W. Overview of digital image processing algorithms. Sci. Technol. Innov. 2020, 19, 29–30. [Google Scholar] [CrossRef]
- Movafeghi, A.; Yahaghi, E.; Mirzapour, M.; ShayganFar, P. Defect Measurement in Welded Objects by Radiography Testing and Chambolle’s Image Processing Method. J. Nondestruct. Eval. 2021, 40, 49. [Google Scholar] [CrossRef]
- You, S.; Zhang, H.; Zhou, Y.L.; Zhang, X.L. Repair of center lines’s break point of weld structure light based on hilditch algorithm. Hot Work. Technol. 2020, 49, 134–138+145. [Google Scholar] [CrossRef]
- Cao, Y.W.; Yang, G.L.; Zhang, Y.Y.; Wang, R.J.; Cheng, Z.H. Rapid evaluation method of shape characteristics of aggregate particle based on the minimum outer rectangle. J. Chongqing Jiaotong Univ. (Nat. Sci.) 2019, 38, 61–65. [Google Scholar]
- Wang, D.; Zhang, J.X.; Fan, J.; Zhai, X.L.; Dong, L.; Pei, W.X.; Wang, J.J. Evaluation method of commodity specification grade of lycii fructus based on image processing technology. Mod. Tradit. Chin. Med. Mater. Medica-World Sci. Technol. 2020, 22, 2817–2823. [Google Scholar]
- Maini, S.; Aggarwal, A.K. Camera Position Estimation using 2D Image Dataset. Int. J. Innov. Eng. Technol. 2018, 10, 199–203. [Google Scholar]
- Kumar, A.; Ikeuchi, K.; Sato, Y.; Oishi, T.; Ono, S. Improving gps position accuracy by identification of reflected gps signals using range data for modeling of urban structures. Prod. Res. 2014, 66, 101–107. [Google Scholar] [CrossRef]
- Kumar, A.; Banno, A.; Ono, S.; Oishi, T.; Ikeuchi, K. Global Coordinate Adjustment of the 3D Survey Models under Unstable GPS Condition. Seisan Kenkyu 2013, 65, 91–95. [Google Scholar] [CrossRef]
- Li, C.C. Study and Realization of Camera Calibration Technology in Machine Vision. Master’s Thesis, NUAA, Nanjing, China, 2009. [Google Scholar]
- Zhang, Y.; Qi, J. Research on vision calibration method of large field view based on halcon. Mech. Electr. Eng. Technol. 2019, 48, 1–3. [Google Scholar]
- Bao, Q. A hybrid genetic simulated annealing algorithm for solving the travel quotient problem. China Storage Transp. 2021, 11, 204–205. [Google Scholar] [CrossRef]
- Graczyk, J.; Mihalache, N. Optimal bounds for the analytical traveling salesman problem. J. Math. Anal. Appl. 2022, 507, 125811. [Google Scholar] [CrossRef]
- Gao, J.; Ren, Y.M. Simulated annealing based algorithm for solving combinatorial optimization problems. Sci-Tech Dev. Enterp. 2021, 5, 66–68. [Google Scholar]
- Fan, W.P. Particle Swarm Optimization Algorithm and Its Application in Fuzzy Clustering ai Gorithm. Master’s Thesis, Lanzhou Jiaotong University, Lanzhou, China, 2021. [Google Scholar] [CrossRef]
- Hu, C.Z.; Zhou, J.L. Design of Multi Axis Motion Controller Based on STM32 + FPGA. Mach. Tool Hydraul. 2021, 49, 82–86. [Google Scholar]
- Sun, X.S.; Wang, S. Comparison of σ estimators in mean—Standard deviation control charts. Stat. Decis. 2021, 37, 32–35. [Google Scholar] [CrossRef]
Number | Dispensing Area Center Coordinate Point/Pixel | Dispensing Area/Pixel |
---|---|---|
1 | (294, 235) | 2215 |
2 | (352, 708) | 2235 |
3 | (436, 483) | 2218 |
4 | (707, 282) | 2230 |
5 | (610, 413) | 2228 |
Number | Pixel Coordinates (X/Y) | Actual Dispensing Position Coordinates (X/Y) |
---|---|---|
1 | (706.8, 433.9) | (31, 19) |
2 | (1874.5, 548.2) | (82, 24) |
3 | (1095.4, 1120.1) | (48, 49) |
4 | (2098.5, 2056.5) | (92, 90) |
5 | (526.3, 1599.5) | (23, 70) |
Parameters | Numerical Value |
---|---|
Initial temperature (t0) | 1000 |
Cooling factor () | 0.98 |
Number of iterations (N) | 500 |
Temperature value after annealing (eps) | −11.28 |
Group | ||
---|---|---|
1 | 1654.9 | 940.1 |
2 | 1713.4 | 959.8 |
3 | 1707.4 | 949.2 |
Experiment Serial Number | Total Number of PCB Boards/Block | Number of Correct Extractions/Block | Withdrawal Time/ms | Correct Extraction Rate |
---|---|---|---|---|
1 | 120 | 120 | 815 | 100% |
2 | 120 | 120 | 820 | 100% |
3 | 120 | 120 | 765 | 100% |
4 | 120 | 119 | 910 | 99.2% |
5 | 120 | 120 | 620 | 100% |
Location | X1 | Y1 | X2 | Y2 |
---|---|---|---|---|
1 | 236.32 | 658.49 | 845.69 | 759.15 |
2 | 236.35 | 658.30 | 845.33 | 759.58 |
3 | 236.58 | 658.98 | 845.85 | 759.37 |
4 | 236.40 | 658.11 | 845.08 | 759.88 |
5 | 236.38 | 658.67 | 845.92 | 759.02 |
6 | 236.60 | 658.32 | 845.51 | 759.88 |
7 | 236.51 | 658.88 | 845.67 | 759.63 |
8 | 236.60 | 658.04 | 845.96 | 759.56 |
9 | 236.47 | 658.25 | 845.10 | 759.37 |
10 | 236.52 | 658.59 | 845.94 | 759.69 |
Category | Dispensing Operation Speed (mm/s) | Gum Spitting Volume (mL/s) | Number of Dispensing at a Time | Number of Groups |
---|---|---|---|---|
Parameters | 1 | 1 | 20 | 5 |
Group | Number of Dispensing | Traditional Dispensing Is Time-Consuming/s | Optimized Dispensing Time/s | Efficiency Improvement Percentage |
---|---|---|---|---|
1 | 20 | 429.4 | 297.6 | 30.7% |
2 | 20 | 420.3 | 298 | 29.1% |
3 | 20 | 397.6 | 295.1 | 25.8% |
4 | 20 | 374 | 293.6 | 21.5% |
5 | 20 | 396.4 | 298.1 | 24.8% |
Number | Image Acquisition Dispensing Coordinates | Angle | Actual Dispensing Coordinates | Dispensing Area/Pixel |
---|---|---|---|---|
1 | (629.28, 713.64) | 135.6 | (27.6, 31.3) | 2236 |
2 | (1999.52, 1128.5) | 165.2 | (87.7, 49.5) | 2245 |
3 | (1108.04, 1598.28) | 36.2 | (48.6, 70.1) | 2231 |
4 | (264.48, 1550.5) | 64.8 | (11.6, 68) | 2229 |
5 | (2186.52, 579.12) | 95.6 | (95.9, 25.4) | 2236 |
6 | (1374.84, 1903.8) | 308.9 | (60.3, 83.5) | 2227 |
7 | (2031.04, 2197.92) | 225.8 | (88.8, 96.4) | 2239 |
8 | (642.96, 2008.68) | 294.3 | (28.2, 88.1) | 2236 |
9 | (948.48, 2058.84) | 10.8 | (41.6, 90.3) | 2234 |
10 | (2243.52, 2496.6) | 302.2 | (98.4, 109.5) | 2225 |
11 | (1051.08, 2421.36) | 260.5 | (46.1, 106.2) | 2226 |
12 | (836.76, 446.88) | 159.7 | (36.7, 19.6) | 2227 |
13 | (1942.56, 2118.13) | 122.3 | (85.2, 92.9) | 2236 |
14 | (978.12, 1459.2) | 231.9 | (42.9, 64.1) | 2234 |
15 | (585.96, 1146.84) | 56.2 | (25.7, 50.3) | 2238 |
16 | (565.44, 937.08) | 78.3 | (24.8, 41.1) | 2229 |
17 | (2268.6, 884.64) | 170.8 | (99.5, 38.8) | 2237 |
18 | (563.16, 1422.72) | 264.9 | (24.7, 62.4) | 2231 |
19 | (2154.6, 1561.9) | 320.5 | (94.5, 68.5) | 2228 |
20 | (886.92, 1694.06) | 83.7 | (38.9, 74.3) | 2233 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Huang, B.; Liu, X.; Yan, J.; Xie, J.; Liu, K.; Xu, Y.; Liu, J.; Zhao, X. Fully Automated Dispensing System Based on Machine Vision. Appl. Sci. 2023, 13, 9206. https://doi.org/10.3390/app13169206
Huang B, Liu X, Yan J, Xie J, Liu K, Xu Y, Liu J, Zhao X. Fully Automated Dispensing System Based on Machine Vision. Applied Sciences. 2023; 13(16):9206. https://doi.org/10.3390/app13169206
Chicago/Turabian StyleHuang, Bo, Xiang Liu, Jiawei Yan, Jiacheng Xie, Kang Liu, Yun Xu, Jianhong Liu, and Xintong Zhao. 2023. "Fully Automated Dispensing System Based on Machine Vision" Applied Sciences 13, no. 16: 9206. https://doi.org/10.3390/app13169206
APA StyleHuang, B., Liu, X., Yan, J., Xie, J., Liu, K., Xu, Y., Liu, J., & Zhao, X. (2023). Fully Automated Dispensing System Based on Machine Vision. Applied Sciences, 13(16), 9206. https://doi.org/10.3390/app13169206