1. Introduction
Gas metal arc welding (GMAW) is a method of welding by creating an arc between a wire and the base metal which is protected by a shielding gas consisting of inert gas. It is possible to automatically feed the wire to perform robot welding or automated welding, and GMAW is mainly used together with galvanized steel sheets to manufacture automobile chassis. However, when GMAW is used on galvanized steel sheets, the zinc plating on the surface is vaporized at a temperature that is lower than the base metal’s melting point. As a result, the zinc vapor becomes trapped in the weld and cannot escape, and porosity defects such as blow holes and pits occur1-3). These defects are a major factor in reducing weld quality and productivity4). To detect these defects, studies have been conducted on detecting porosity defects by analyzing the properties of arc voltage signals that occur during arc welding as an alternative to non-destructive inspections such as radiographic inspections and ultrasound inspections that can only be performed after the welding process is finished5-11). Such methods are more cost-effective than existing non-destructive inspection methods, and they can improve productivity in terms of time spent.
Deep learning techniques are used in a variety of fields in addition to quality prediction algorithms12-14). However, most of the studies that have employed such neural networks have not described how the neural network model’s hyper parameters are selected, and there are many cases where these values are simply presented. Because the researchers go through a process of finding the best structure by personally performing tests, there is clearly a need for improvement in this area.
Complex problems can be solved by increasing the number of hidden layers in a deep learning structure. However, when training is performed on a neural network that has many hidden layers to solve relatively simple problems, such as when the training data set is small or the training data have a simple format, feature values with low importance are extracted, which causes overfitting and lowers the model’s generalization performance14). Because the problems that can be solved by a neural network vary according to the neural network’s structure in this way, the task of properly selecting hyper parameters is very important. Because the hyper parameters of artificial neural networks are so various, it is counterproductive to find the optimal structure by testing hyper parameters for all cases. Therefore, many studies have been conducted on using genetic algorithms to optimize these variables16,17). Genetic algorithms find optimal solutions based on randomness, and the more samples there are, the easier it is to find the global optimal solution16). However, this has a drawback in that the amount of computation increases as the number of samples increases, and the computation time becomes longer.
Therefore, this study proposes a new algorithm that reduces computation time by adjusting the population of chromosomes in each generation. In order to use the genetic algorithm, galvanized steel sheet GMAW experiments were conducted to extract feature variables according to whether or not porosity occurred in the arc voltage signals that were generated during the welding process. Then, the deep neural network (DNN) model was trained, and the DNN model’s hyper parameters were optimized using the genetic algorithm proposed by this study. Lastly, the validity of the proposed optimization algorithm was verified by comparing its performance to a previous study’s DNN structure11).
2. Experiments
2.1 Materials
In this study, a hot-rolled galvanized steel sheet with an ultimate strength of 590 Mpa and a thickness of 2.3 mm was used as the test material. The hot-rolled steel sheet that was used was manufactured by passing a slab between rolls at a temperature of 800°C to shape it into the proper length and width. The material’s zinc plating thickness was around 10 ㎛, and its mechanical and chemical properties are shown in Table 1. The sheet was manufactured to have a width of 150 mm and a length of 180 mm.
2.2 Welding Equipment and Process
The welding experiments were conducted in short circuit transfer mode using a constant voltage direct current inverter-type electric welder. The overall configuration of the welding system is shown in Fig. 2. The welding workpiece was fixed above the jig by a lap joint, and the welding current between the welder and the jig was measured by a hall sensor and transmitted to a DAQ device. The welding voltage was measured in real-time from the workpiece and the wire feeder by the DAQ device. As the welding style, this study used the lap joint style with a torch angle of 45°, which is often used in automobile body chassis welding sites.
As shown in Table 2, the welding speed was 600 mm/min, and the distance between the contact tip and the base metal (CTWD) was fixed at 15 mm. The shielding gas was mixed at a ratio of 90% Ar and 10% CO2, and the gaps between the workpieces were fixed at 0.0 mm and 0.5 mm. Also, an ER70S-3 level wire with a diameter of 1.2 mm was used, and the experiments were conducted at a wire feed rate of 4 m/min. For reliability, the experiments were repeated two times under each set of conditions.
3. Genetic Algorithm
3.1 Genetic Algorithm Optimization Technique
Genetic algorithms do not test each of the variables for all cases, but find optimal solutions using randomnesses, such as parent selection, crossover operations, and mutation operations. The major advantage of genetic algorithms is that an appropriate solution can be found by the algorithm if the algorithm designer can represent the format of the solution as a genetic configuration and set up an appropriate fitness function, even in regards to problems for which no concrete method of solving them has been presented.
Fig. 3 shows a flowchart for the genetic algorithm. The first generation’s set of chromosomes is shown in Eq. (1).
In the equation above, Pi is the set of chromosomes for the ith generation. n is the total number of chromosomes (population). Pi,j is the jth chromosome of the ith generation. The genetic information of the first chromosome of the first generation (p1,1) is shown in Eq. (2).
q is the values of the genetic information of each chromosome. m is the number of types of genetic information of each chromosome. qi,j,k is the kth genetic information value of Pi,j, which is the jth chromosome in the ith generation. Generally, the genetic information value q is given minimum and maximum value limits and assigned a random value in the initial generation.
An appropriate function evaluates all of the n number of chromosomes in the first generation chromosome set P1, which has chromosomes that were given genetic information values.
f is the fitness function, and the genetic algorithm designer must configure it personally. If one or more of the chromosomes that have been evaluated by the appropriate function f has a score that is greater than or equal to the expected appropriate value, it becomes the optimal solution, and if not, the next generation is generated using the first generation chromosomes which were given scores. Roulette wheel selection was used, which is a method that probabilistically selects two chromosomes with a probability that increases as their fitness scores increase. When the first generation’s ath (p1,a) and bth (p1,b) chromosomes are selected as shown in Eqs. (4) and (5) below, the crossover operation is performed.
The arithmetic crossover was chosen from among a number of methods18) that have already been proposed as crossover methods for chromosomes that have gene values in real number format. The first descendent chromosome in the second generation that was created after crossover is called p2,1, and it is shown in Eq. (6).
Eq. (7) shows p2,1, which is the kth gene value of q2,1,k that was produced by arithmetic crossover.
αk has a random value between 0 and 1. That is, q2,1,k, which is the kth gene value of the descendent p2,1, has a random value between the kth gene value of the ath chromosome in the first generation and the kth gene value in the bth chromosome in the first generation. After the crossover operation is performed in this way, a mutation operation is performed to generate the second generation of chromosomes by changing gene values randomly with certain probabilities. Then, the fitness function f is used to perform evaluation again and determine whether a subsequent generation will be generated.
In this study, such a genetic algorithm was used as shown in Fig. 4 to optimize the hyper parameters of the artificial neural network for predicting GMAW defects. After the GMAW experiments, the arc voltage waveform feature variables were extracted, and the porosity according to the weld location was used as training data. The genetic algorithm was used to find the optimal hyper parameters for the neural network, and the optimized neural network was trained and verified.
The ReLU function was used as the activation function for the hidden layers of the neural network that will perform optimization. The output was 1 if there was a defect and 0 if there was no defect, and the softmax function, which is effective at processing the classification of multiple classes, was used as the activation function of the output layer.
3.2 Selecting Deep Learning Hyperparameters
The deep neural network shown in Fig. 5 has a structure in which perceptrons have been arranged in several layers. Learning is performed using a method that finds the optimal weight values for each node by adjusting the weights using an error back-propagation algorithm. The hyper parameters that were optimized in this study were primarily the number of hidden layers and the number of nodes that each hidden layer has. When these numbers are too large, the overfitting phenomenon occurs, and the neural network’s generality declines15). In addition, there are several parameters in the artificial neural network learning process, including the learning rate, batch size, and dropout rate19), and these were also optimized. Table 3 shows a comparison of the optimization results and a previous study’s11) DNN structure.
3.3 Genetic Algorithm-based Optimization
The hyper parameters that were selected in Section 3.2 were assigned to chromosomes as gene values in real number format, and the genetic information in the chromosomes was the number of hidden layers, number of nodes in each layer, learning rate, batch size, and dropout rate. The ranges for each of the gene values were set as 1-10 for the number of hidden layers, 10-50 for the number of nodes in each hidden layer, 0-0.2 for the learning rate, 100-10000 for the batch size, and 0-0.8 for the dropout rate.
The chromosomes of the first generation were randomly configured with gene values within the ranges above. In this study, the population of the first generation was set at 256. The artificial neural networks that correspond to each of the 256 randomly generated chromosomes were all trained to find out whether they had structures that were appropriate for the training data. After training was completed, the reciprocal value of the loss (MSE) that occurred for the test data, which consisted of data other than the training data, was used as the fitness score. Higher scores indicated better chromosomes.
After the fitness scores of all chromosomes in the first generation were calculated in this way, the parent chromosomes were selected using the roulette wheel selection style, which assigns probabilities to chromosomes by using the fitness scores as weight values. Next, the second generation of chromosomes was generated using the arithmetic crossover and mutation operations. In existing generation algorithms, the process above can be continuously repeated, but in this study, several rules were also applied to the number of chromosomes (population) in each generation.
Other previous papers that have used genetic algorithms20,21) usually had populations in which all generations were the same. Fig. 6(a) is an example of an existing genetic algorithm being used in this optimization study. Fig. 6(b) is the new, improved algorithm being presented in this paper. The number of chromosomes decreased as the algorithm went through the generations, and the epoch was increased slightly compared to the previous generation as the algorithm progressed. An analysis was performed, focusing on the descendent chromosomes that were selected in the previous generation. The steps between the first and second generations are shown in Fig. 6, and the same method was used from thereon. Also, a minimum number of chromosomes was set to prevent the number of chromosomes in a single generation from becoming too small. The result of these improvements was that the time spent producing similar optimal hyper parameters was reduced by around 30% compared to before. Using the shortened time, the number of chromosomes (popu- lation) in the first generation was set to be around twice as large as the existing algorithm, and the diversity of the initial chromosomes was also improved.
4. Results and Observations
4.1 Signal Analysis and Feature Extraction
Fig. 7 shows the bead shapes, x-ray images, and welding voltage waveforms that were captured during 20 seconds of welding experiments at a sampling rate of 10 kHz. For the waveform voltage data, sampling data were taken at 0.1-second intervals (1000 data items) from the raw measurement data, and this was assigned to a single window. A 90% overlap was performed, and feature variables were found for each window. Fig. 8 shows the short circuit transfer mode arc voltage waveforms that were generated during a 0.1-second interval window, and each of the variables was found based on the 1000 data items during the 0.1-second interval. Vp and Vs are the maximum and minimum voltage data values during the 0.1-second, respectively. s[V] is the standard deviation for the 0.1-second of data. s[Vs]is the standard deviation of the voltage data during 5 instantaneous short circuits (orange circles). s[Vp]is the standard deviation of the voltage data during 5 peaks (blue circles). s [Ta] is the standard deviation of the arc periods within 0.1-second. s [V (Ts)]and s [V (Ta)]are the standard deviation values of voltages measured during the short circuit periods and the arc periods, respectively. Table 4 shows each of these variables.
Fig. 7
Voltage waveform, bead shape, x-ray image and train, test data, (a) result of experiment #1, (b) result of experiment #2, (c) result of experiment #3, (d) result of experiment #4
Table 4
Feature variables
4.2 Results of Optimization Using a Genetic Algorithm
Optimization was performed using the variables in Table 5 below based on the algorithm presented in Section 3.2. Table 6 shows all of the chromosomes that were generated during the optimization process. The neural networks were arranged in descending order starting with the neural network that had the highest fitness score. The total number of generated chromosomes was 1216 (256 in the first generation, 128 in the second generation, 64 in generations3-15), and the top 12 are shown in Table 6.
Table 5
Genetic algorithm variables
| Variables | Value |
|---|---|
| Initial number of chromosomes | 256 |
| Minimum number of chromosomes | 64 |
| Maximum generation | 15 |
| mutation probability | 15 % |
Table 6
Neural network optimization results for GMAW experiment data
The maximum value for the number of hidden layers was set at 10 layers, but it was found that the numbers of hidden layers in the top-level chromosomes converged at 3 layers. From this result, it was found that the artificial neural network did not always improve as its number of hidden layers increased. In addition, the results were consistent with previous studies on overfitting15). The batch size was 5000; the learning rate was around 0.02, and the dropout rate tended to be around 0.1. The optimal value was judged to be significant because optimization was performed with a maximum learning rate of 0.2 and a maximum dropout rate of 0.8. Table 7 below shows a comparison of (b) the chromosome with the highest fitness score in Table 6 and (a) the previous study’s DNN artificial neural network structure shown in Table 3. The training was performed on the previous study’s DNN model (a) and the optimized model (b) from Table 7 for 1500 epochs each with the GMAW welding data, and Fig. 9 shows the defect prediction results, bead shapes, and x-ray images, while Table 8 shows the detailed figures for these results.
Table 7
Comparison of hyper parameters, (a) DNN structure from previous research, (b) Optimized GA- DNN structure
Fig. 9
Porosity defects prediction from DNN, (a) Prediction from previous research DNN structure, (b) Prediction from optimized GA-DNN structure
Table 8
Predictive accuracy for each model
| Total test data | True | Error | accuracy | |
|---|---|---|---|---|
| DNN structure from previous research | 361 | 323 | 38 | 89.4737 |
| Optimized GA- DNN structure | 336 | 25 | 93.0748 |
The total number of test data that were used for verification was 361. The previous study’s DNN model accurately classified 323 data and had 38 errors for an accuracy of 89.5%. The DNN model that was optimized with the genetic algorithm accurately classified 336 data and had 25 errors for an accuracy of 93.1%. The optimized model’s prediction accuracy was better by 3.60%. The number of errors was reduced from 38 to 25 for a reduction rate of 34.2%.
4.3 System for Real-time Use After Optimization
This study proposes a system for real-time prediction of porosity defects that occur during GMAW using an optimized GA-DNN structure, as shown in Fig. 10. As in a previous study11), the welding voltage is measured in real-time by a DAQ device, and feature variables are extracted through a data preprocessing step and used as input values. A considerable amount of time is spent performing optimization with the genetic algorithm, but when the input values are entered in the DNN structure that has been pre-optimized with the previous welding process’ data and the genetic algorithm, the porosity defect location prediction values are calculated right away. It is thought that the DNN structure can be periodically optimized and improved using the welding data that are obtained as the welding process is repeatedly performed on-site.
5. Conclusions
This study extracted feature variables from GMAW experiment results and used a genetic algorithm to optimize the hyper parameters of a deep neural network that detects the presence of porosity, and it was confirmed that the results of this approach improved performance. This approach makes it possible to automatically design neural networks that are suitable for each different welding environment, even though the welding equipment that is used or the automobile type, etc. may vary. By using this porosity defect detection algorithm, it will be possible to further maximize improvements in regards to time and cost. Moreover, it is thought that this genetic algorithm approach can be applied not only to welding data but also to other types of training.
Below is a summary of the areas and conclusions that have been developed in this study.
1) Feature variables were derived by measuring the arc voltage waveforms that occur during the GMAW process in real-time and performing data preprocessing.
2) The new genetic algorithm that was proposed by this study reduced calculation time by around 30% compared to before and increased the number of first- generation chromosomes, thus improving the diversity of the initial chromosomes.
3) The results of verifying the optimization results showed that the DNN model that was optimized by the genetic algorithm had a prediction accuracy of 93.1%, which was an increase of 3.60% compared to a previous study’s DNN model prediction rate.
However, in order to increase the algorithm’s generality and use the algorithm in a variety of areas, it will be necessary to make improvements such as automatically selecting specific items like the layer activation functions that are used. These issues will be improved upon in follow-up studies.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
CrossRef TDM
Print
