Warning: fopen(/home/virtual/kwjs/journal/upload/ip_log/ip_log_2024-03.txt): failed to open stream: Permission denied in /home/virtual/lib/view_data.php on line 88 Warning: fwrite() expects parameter 1 to be resource, boolean given in /home/virtual/lib/view_data.php on line 89 A Study on the Development of Deep Learning Algorithm for Determining External Quality of Welded Parts Using Transfer Learning

J Weld Join > Volume 41(2); 2023 > Article
Seo, Han, Heo, Kim, and Cho: A Study on the Development of Deep Learning Algorithm for Determining External Quality of Welded Parts Using Transfer Learning

Abstract

Recently, deep learning has been applied to various welding techniques, such as laser welding, gas metal arc welding (GMAW), and resistance spot welding, and research on automation and quality prediction is being conducted. Even for GMAW, many researchers have attempted to predict quality through X-ray, current, and voltage measurements. If judgment in real time is not necessary, it is most effective to judge the quality of a welded part using an exterior image. Therefore, in this study, a welded appearance quality judgment model was analyzed using image deep learning. Welding defects were classified into pores, overlaps, craters, melting of the base material, cracks, and undercuts, and were divided into 7 categories including normal ones. In constructing the deep-learning model, transfer learning was performed using existing networks, such as ResNet and AlexNet. To improve the accuracy of deep learning, tests were performed while the optimization technique, maximum number of epochs, and minibatch size were changed. It was confirmed that the accuracy of weld defect prediction improved as the minibatch size increased, and the stochastic gradient descent model had the highest accuracy. Increasing the number of data for learning should make the technique of using images to judge the quality of GMAW welds using the proposed model more widely applicable.

1. Introduction

Machine learning (deep learning) is being utilized in a variety of industries. It is being applied to autonomous cars as a way to distinguish between objects, as well as to medical devices, games, color restoration, and the creation of new objects. In production and manufacturing, machine learning is being used to automate processes, inspect quality, and ensure user safety. Among the many different methods of applying machine learning, deep learning is widely used in machine learning using time series data measured by sensors, machine learning through pre-processing of measured data, and processes that use images to inspect quality or automation processes.
Likewise, there are many applications of machine learning and deep learning in the field of welding1-3): shape extraction from laser vision sensing images4), weld line tracking using laser sensors5), quality and weld line tracking using image sensors6), weld line tracking through time series data analysis7), and quality prediction through analysis of acoustic signals8). J. H. Kim et al. developed an algorithm to judge the quality of a welded part by learning the resistance in resistance spot welding9). T. W. Kim et al. verified the possibility of utilizing deep learning for non-destructive welding quality inspection by applying it to quality judgment in the joining technology of Al-Cu dissimilar materials using green laser10). H. Deng et al. developed a deep learning model to detect defects through image preprocessing in asymmetric laser welding images11).
As shown in the above examples, there are many application cases of deep learning in welding for quality judgment and automation. Moreover, deep learning has been applied to the gas metal arc welding (GMAW) process. M. S. Kim et al. proposed a deep learning model for judging the formation of backbead in GMAW12). R. T. Martinez et al. developed a deep learning technique for predicting the shape of GMAW welded parts13). S. Q. Moinuddin et al. suggested an algorithm to predict weld defects based on current and voltage data in GMAW process14). S. Shin et al. presented a deep learning method to predict pores after measuring current and voltage data in real time15). H. Zhu et al. performed image classification for pores, spatter, and overlap by monochromatizing images16). Furthermore, there are various methods to apply deep learning for quality judgment, such as analyzing defects by capturing X-ray images17,18).
There are many studies on quality prediction in GMAW, but there is a lack of papers related to appearance quality. Moreover, there are papers that analyze weld defects, but they do not cover various defects. In addition, in the case of actual welded part images, there are many changes in the image depending on the lighting, curved welded part, slag, and shooting angle. This makes it difficult to predict the quality. As a result, most industrial sites are performing visual quality inspection. Therefore, to automate the inspection of external quality of welded parts in this study, image data of defective and normal parts of weld appearance were obtained and labeled, and deep learning algorithms were applied to predict the quality. Here, the hidden layer was constructed through transfer learning, and the accuracy of deep learning was analyzed by applying multiple models. Finally, the best model was selected and a proposal was made to improve the accuracy of weld defect judgment.

2. Experimental Method

First of all, the welded part was photographed to secure a database for judging the quality of the welded part. Fillet welding was performed using the IGM welding robot, and a wide bead width was secured by weaving. Based on the weld appearance photos, the algorithm for judging the welding quality was configured as shown in Fig. 1. Fig. 1(a) shows the images of six types of weld defects: pores, base metal greening, undercut, overlap, crack, and crater. These were categorized into seven categories, including the normal weld condition. As shown in Fig. 1(b), the size of the images was varied to apply transfer learning. As shown in Fig. 1(c), a convolutional neural network (CNN) was used to predict the welded part quality, and the data was trained using transfer learning. The existing models for transfer learning are Alexnet and ResNet101. In Fig. 1(c), only ResNet is representative. AlexNet is a CNN architecture with a total of 8 layers, including 5 convolutional layers and 3 fully connected layers. ResNet is also a CNN architecture and consists of blocks with multiple convolutional layers and connections that bypass the convolutional layers. ResNet’s bypassing connections allow it to train a much deeper network without suffering from the gradient vanishing problem that is common in deep neural networks. Finally, the performance of the classification model was evaluated after testing with the trained model as shown in Fig. 1(d). All data trainings were performed using Matlab. As shown in Table 1, the optimization algorithms used were Adam, Rmsprop, and SGDM. Among the optimization algorithms, Gradient Descent (GD) is a timeconsuming method that calculates the gradient of the loss function in all directions and then updates the data in the direction with the highest gradient.
Fig. 1
Weld defect prediction deep learning algorithm workflow (a) Weld defect image and categories, (b) Image resizing for application in transfer learning, (c) Part of CNN based ResNet 101, (d) Predicted test data confusion chart based on the trained model
jwj-41-2-132gf1.jpg
Table 1
Deep learning parameter settings for welding defect detection
Parameter Input value
Initial learning rate Optimization algorithm: Adam 0.001
Optimization algorithm: RMSprop 0.001
Optimization algorithm: SGDM 0.01
Mini-batch size 64, 128
Max epoch 10, 30
Data Train: 70%, Test 30%
To improve the GD, the Gradient Descent (SGD) method was created, which uses probabilistic calculations to learn, and is faster. However, it moves in a random direction, which causes oscillations when learning. To improve this, the Stochastic Gradient Descent with Momentum (SGDM) method was proposed, which adds momentum. This method avoids oscillations by learning with inertia in the direction of the gradient of the previously moved loss function. While SGDM can be faster to learn, it can sometimes slow down training because it can lead to many trainings in the wrong direction. The Root Mean Square Propagation (RMSprop) is similarly trained by adjusting the learning rate based on the average of the gradient magnitudes of the loss functions executed to reduce oscillations, an issue with SGD. Adaptive Moment Estimation (Adam) combines the ideas of RMSprop and SGDM and adjusts the learning rate based on moment estimation. It is computationally efficient and is most commonly used for high-dimensional parameter spaces.
The performances of the classification models were compared by varying some of the parameters required for training. The tests were conducted by varying the minibatch size and epoch. The minibatch size was set to 64 and 128 because it is recommended to use powers of 2, which is the unit of bits. The epoch was set to a maximum of 30, as there is a problem with overfitting if the epoch is too large. The minimum was set to 10 to avoid stopping when the training was not complete. We shuffled all the training data randomly after each epoch to prevent overfitting. In total, 380 images were taken, with 70% of them used for training and 30% for testing.

3. Experiment Results

The results of training with the weld defect photos were expressed in a confusion chart as shown in Fig. 2. The confusion chart is shown in Fig. 2. The results of correctly predicting the defective or normal parts are colored in blue, and the incorrect results are colored in red. Parts with a large amount of data in the corresponding cells are expressed in darker colors. For cracks, craters, and overlaps, not much data was obtained because such defects rarely occur. The transfer learning model was ResNet101, and the optimization technique was Adam. Fig. 2 shows the variation of accuracy with the maximum number of epoxies and minibatch size. It can be seen that the accuracy improves more when the minibatch size is larger. Similarly, it can be seen that increasing the maximum number of epoxies to 30 improves the accuracy for the test data. Since increasing the epoch further would likely result in overfitting, we did not further train the model by increasing the maximum epoch. When using Adam to predict weld defects, the minibatch size of 128 and the maximum epoch of 30 resulted in the most accurate prediction at 90.2%.
Fig. 2
Confusion chart of predicted results according to changes in max epoch and batch size
jwj-41-2-132gf2.jpg
Similar to the above methods, the prediction accuracy of the optimization method was checked. The optimization techniques used were SGDM, which is supported by Matlab, RMSprop, and Adam. Fig. 3 shows the changes of the loss as the epoch increased. RMSprop showed the largest loss, while the SGDM method had the smallest loss. For SGDM and Adam, the loss decreased sharply when the epoch was above 10.
Fig. 3
Graph of loss change according to optimization technique
jwj-41-2-132gf3.jpg
As the epoch increased, the loss decreased and the accuracy improved. However, RMSprop showed an increase in loss when the epoch is higher than 10, indicating that the possibility of overfitting. Therefore, the prediction accuracy was compared for the case of 10 epoch. The prediction accuracy for the test data is shown in Fig. 4. The prediction accuracy of Adam in comparison to SGDM and RMSprop is shown in Fig. 2. SGDM, the optimization technique with the smallest loss shown in Fig. 3, had the highest accuracy of about 93%. Similar to the above methods, the transfer learning model was tested using Alexnet instead of Res- Net101.
Fig. 4
Confusion chart of prediction results by optimization method
jwj-41-2-132gf4.jpg
The results are shown in Fig. 5, which shows a slight increase in accuracy over ResNet. Finally, a comparison graph for all results is shown in Fig. 6. Fig. 6(a) shows that the optimization technique using SGDM has the highest prediction accuracy, and RMSprop has the lowest accuracy. Fig. 6(b) shows that increasing the minibatch size and the number of epoxies contributes to the accuracy improvement. However, randomly increasing the number of epoxies can reduce the accuracy of the validation data due to overfitting. Using all the previous processes, it was determined that the current SGDM model is the most accurate at 92%. Based on this, we checked the test model and obtained the results shown in Fig. 7. Fig. 7(a) shows 100% probability of base material melting. Similarly, Fig. 7(b) shows a 36.5% probability of a crater, which is not an accurate prediction. However, the probability of a crater is greater than that of any other defect, so it was predicted to be a crater. Similarly, the prediction accuracy is over 90% in Fig. 7(c) and (d). Currently, the amount of data is not large enough to obtain perfect prediction results, but the prediction accuracy will improve when the amount of data is sufficient.
Fig. 5
Confusion chart of prediction results by transfer network
jwj-41-2-132gf5.jpg
Fig. 6
Loss change graph according to deep learning conditions (a) Loss change according to optimization technique, (b) Loss change according to maximum epoch and mini-batch size
jwj-41-2-132gf6.jpg
Fig. 7
Weld defect prediction results and probability for test data (a) 100% probability of base metal melting, (b) The probability of being a crater is 36.8%, which is the most probable state compared to other defects, (c) 94.5% chance of being normal, (d) 99.9% chance of undercut
jwj-41-2-132gf7.jpg

4. Conclusions

A deep learning algorithm was built to determine the appearance quality of fillet welded parts used in industrial sites. Images of normal and defective parts of fillet welded parts were obtained and labeled. The algorithm was trained to recognize a total of six types of defects in the welded parts: pores, overlaps, cracks, undercuts, base metal melting, and craters. The hidden layer of deep learning was constructed through transfer learning, and the existing models ResNet 101 and Alexnet were used. Tests were conducted by varying the optimization technique, maximum epoch number, and minibatch size to improve the accuracy of deep learning. The results showed that the larger the minibatch size, the better the weld defect prediction accuracy. The accuracy of the validation model was higher at 30 than at 10, and the SGDM model showed the highest accuracy among the optimization techniques when the maximum epoch was 10.
In this study, we proposed a deep learning model for determining the appearance quality of welded parts. The SGDM model showed the best results, and the accuracy of weld defect judgment could be improved through various optimization techniques and changes in learning variables. However, the analysis of the results was insufficient due to a lack of publicly available images or data for weld defects. In the future, we plan to improve the accuracy of defect judgment by securing a large amount of data.

Acknowledgement

This results was supported by “Regional Innovation Strategy (RIS)” through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(MOE)(2023RIS-003) and supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019- R1A5A8083201).

References

1. Q. Wang, W. Jiao, P. Wang, and Y. Zhang, A tutorial on deep learning-based data analytics in manufacturing through a welding case study, J. Manuf. Process. 63 (2021) 2–13. https://doi.org/10.1016/j.jmapro.2020.04.044
[CROSSREF] 
2. Y. Zhang, D. You, X. Gao, N. Zhang, and P. P. Gao, Welding defects detection based on deep learning with multiple optical sensors during disk laser welding of thick plates, J. Manuf. Syst. 51 (2019) 87–94. https://doi.org/10.1016/j.jmsy.2019.02.004
[CROSSREF] 
3. B. Zhang, K. M. Hong, and Y. C. Shin, Deep-learning-based porosity monitoring of laser welding process, Manuf. Lett. 23 (2020) 62–66. https://doi.org/10.1016/j.mfglet.2020.01.001
[CROSSREF] 
4. K. Bae and S. J. Na, A Study of Vision-Based Measurement of Weld Joint Shape Incorporating the Neural Network, J. Eng. Manuf. 208(1) (1994) 61–69. https://doi.org/10.1243/PIME_PROC_1994_208_060_02
[CROSSREF] 
5. L. Yang, J. Fan, B. Huo, E. Li, and Y. Liu, Image denoising of seam images with deep learning for laser vision seam tracking, IEEE Sens. J. 22(6) (2022) 6098–6107. https://doi.org/10.1109/JSEN.2022.3147489
[CROSSREF] 
6. Z. Zhang, G. Wen, and S. Chen, Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding, J. Manuf. Process. 45 (2019) 208–216. https://doi.org/10.1016/j.jmapro.2019.06.023
[CROSSREF] 
7. B. W. Seo, Y. C. Jeong, and Y. T. Cho, Machine learning for prediction of arc length for seam tracking in tandem welding, J. Weld. Join. 38(3) (2020) 241–247. https://doi.org/10.5781/JWJ.2020.38.3.2
[CROSSREF] 
8. K. Asif, L. Zhang, S. Derrible, J. E. Indacochea, D. Ozevin, and B. Ziebart, Machine learning model to predict welding quality using air-coupled acoustic emission and weld inputs, J. Intell. Manuf. (2022) 1–15. https://doi.org/10.1007/s10845-020-01667-x
[CROSSREF] 
9. J. H. Kim, M. H. Ko, and N. K. Ku, A Study on Resistance Spot Welding Failure Detection Using Deep Learning Technology, J. Comput. Des. Eng. 24(2) (2019) 161–168. https://doi.org/10.7315/CDE.2019.161
[CROSSREF] 
10. T. W. Kim and H. W. Choi, Study on laser welding of Al- Cu dissimilar material by green laser and weld quality evaluation by deep learning, J. Weld. Join. 39(1) (2021) 67–73. https://doi.org/10.5781/JWJ.2021.39.1.8
[CROSSREF] 
11. H. Deng, Y. Cheng, Y. Feng, and J. Xiang, Industrial laser welding defect detection and image defect recognition based on deep learning model developed, Symmetry. 13(9) (2021) 1731https://doi.org/10.3390/sym13091731
[CROSSREF] 
12. M. S. Kim, S. M. Shin, D. H. Kim, and S. Rhee, A study on the algorithm for determining back bead generation in GMA welding using deep learning, J. Weld. Join. 36(2) (2018) 74–81. https://doi.org/10.5781/JWJ.2018.36.2.11
[CROSSREF] 
13. R. T. Martínez, G. A. Bestard, A. M. A. Silva, and S. C. A. Alfaro, Analysis of GMAW process with deep learning and machine learning techniques, J. Manuf. Process. 62 (2021) 695–703. https://doi.org/10.1016/j.jmapro.2020.12.052
[CROSSREF] 
14. S. Q. Moinuddin, S. S. Hameed, A. K. Dewangan, K. R. Kumar, and A. S. Kumari, A study on weld defects classification in gas metal arc welding process using machine learning techniques, MaterialsToday:Proceedings. 43 (2021) 623–628. https://doi.org/10.1016/j.matpr.2020.12.159
[CROSSREF] 
15. S. Shin, C. Jin, J. Yu, and S. Rhee, Real-time detection of weld defects for automated welding process base on deep neural network, Met. 10(3) (2020) 389https://doi.org/10.3390/met10030389
[CROSSREF] 
16. H. Zhu, W. Ge, and Z. Liu, Deep learning-based classification of weld surface defects, Appl. Sci. 9(16) (2019) 3312https://doi.org/10.3390/app9163312
[CROSSREF] 
17. N. Yang, H. Niu, L. Chen, and G. Mi, X-ray weld image classification using improved convolutional neural network, AIP Conference Proceedings, 1995. 020035 (2018) 020035https://doi.org/10.1063/1.5048766
[CROSSREF] 
18. W. Du, H. Shen, J. Fu, G. Zhang, and Q. He, Approaches for improvement of the X-ray image defect detection of automobile casting aluminum parts based on deep learning, NDT E Int. 107 (2019) 102144https://doi.org/10.1016/j.ndteint.2019.102144
[CROSSREF] 


ABOUT
BROWSE ARTICLES
ARTICLE CATEGORY 
FOR CONTRIBUTORS
Editorial Office
#304, San-Jeong Building, 23, Gukhoe-daero 66-gil, Yeongdeungpo-gu, Seoul 07237, Korea
Tel: +82-2-538-6511    Fax: +82-2-538-6510    E-mail: koweld@kwjs.or.kr                

Copyright © 2024 by The Korean Welding and Joining Society.

Developed in M2PI