Research on Classification Method of Film Damage Image Based on Improved ResNet50

When the network is relatively large, that is, the corresponding network parameters are relatively large, and the number of data sets is relatively small. It is not enough to train the whole network, so there will be overfitting, and the training results will be very bad. However, by using the transfer learning method, we can leverage the pre-trained model parameters to train our own smaller datasets. It can also train a better effect [10–11]. Therefore, the method of transfer learning is used in this study. The weights trained on the ImageNet dataset are used to train the film damage dataset. The ImageNet dataset contains over one million images across 1,000 categories, and the number of image categories is sufficient. Although it does not include the film damage image, the judgment of the film damage type is mainly based on its contour and other characteristics. Therefore, transfer learning can be carried out through the ImageNet dataset. The weight file for ResNet50 pre-training using ImageNet can be downloaded from the Tensorflow website.

CBAM (Convolutional Block Attention Module) integrates both channel and spatial attention mechanisms to enhance model performance by emphasizing the importance of different regions in the input image. The channel attention module identifies the most relevant channels in the feature map, while the spatial attention module highlights the spatial locations that contain the most informative content. For the special type of film damage image, its characteristic is that the damage area is often presented in the form of tiling. This means that the distribution of lesions over the entire image space is relatively uniform, so in this case, Increasing the spatial attention mechanism will blur the recognition focus of the model, resulting in a decline in classification accuracy. Based on the above considerations, this study decided to use only the channel attention mechanism to process the film damage images. By increasing the weight of the effective channel and suppressing the weight of the ineffective channel, the model can be designed to focus more on the features that are crucial for the classification task. By more efficiently extracting and utilizing the key information from the image, the model improves its feature representation capabilities, ultimately leading to higher classification accuracy [12–14]. The workflow of the CAM attention mechanism is illustrated in Figure 2.

Figure 2.

CAM Attention Mechanism Diagram

During the feature extraction process in deep learning, global pooling technology plays a vital role. Firstly, two different pooling strategies, global maximum pooling and global average pooling, are used to aggregate the wide and high dimensional features of the feature vector f, The global maximum pooling operation focuses on the maximum position of each channel in the feature map, it can effectively capture the most prominent features in the image. Global average pooling is a comprehensive perception of each pixel in the feature map to obtain more comprehensive information. Next, the features obtained from global maximum pooling and global average pooling are concatenated. This step aims to fuse the complementary features extracted by both pooling methods, and a richer and more comprehensive feature representation is obtained. Then, the concatenated features are then fed into a multi-layer perceptron (MLP), which consists of two fully connected layers and a ReLU activation function. This MLP is used to further compress and abstract the features through the linear transformation of the fully connected layers and the nonlinear mapping provided by the ReLU activation, Highdimensional feature vectors can be compressed to lower dimensions while retaining information useful for classification tasks. Finally, the output of MLP is normalized by sigmoid function, and the normalized features are merged. A C × 1 × 1 dimensional channel feature vector is obtained M_c(f)∈R^1×1×C, M_c(f)∈R^1×1×C which contains the global information of each wide-high channel feature map f_c∈R^1×H×W. The calculation process is illustrated in formula (1). 1Mc(f)=σ(MLP(AvgPool(f))+MLP(MaxPool(f)))=σ(W1(fmaxc))+W1(W0(fmaxc))

In (1): W × H — width and height of the feature map; C — number of feature map channels; ^{f(i, j)} — feature map; MLP-Multilayer Perceptron.

And finally, multiply that attention map with the weight of the channel corresponding f_c∈R^1×H×W to the original feature map point by point, Generate the one-dimensional channel feature attention map containing the global feature information required by the spatial attention module ^f₁, as shown in Equation (2). 2f1=f⊗Mc(f)

The original ResNet50 network model has no Dropout module, and the activation function used is ReLU (Rectified Linear Unit).

To address the issue of overfitting, Dropout is applied when the model is large and the number of parameters is extensive. This technique achieves the regularization effect of the model in the training process by temporarily inactivating some neurons. To enhance the model‘s generalization, however, the distribution of activation values may change after each Dropout application, to solve this problem, researchers proposed AlphaDropout, which is an improvement of Dropout [15]. By using Alpha Dropout, the distribution stability of activation values can be better maintained, and the robustness of the model can be further enhanced. Even in large-scale models and complex tasks, it can effectively deal with the problem of overfitting. At the same time, a new activation function, SeLU (Scaled Exponential Linear Units) [16], is introduced. The combination of AlphaDropout and SeLU ensures that the mean and standard deviation of both the input and output remain consistent, thus maintaining the stability of its normalized properties.

SeLU has a significant advantage over the activation function ReLU in that it does not have a deactivation region, as shown in Equation (3). The SeLU activation function has a saturation region at negative infinity, but this does not adversely affect the expressiveness of the model. In contrast, the SeLU activation function automatically normalizes the sample distribution to 0 mean and unit variance, the stability of the gradient in the training process is ensured, thereby effectively avoiding the problems of gradient explosion and disappearance. This feature of automatic normalization makes the SeLU activation function perform well when dealing with large-scale models and complex tasks. 3SeLU=λ{ x,x>0∂ex−∂,x≤0

Where ^∂ and ^λ are hyperparameters ^λ of about 1.05 and ^∂ about 1.67 and ^x are input quantities.

In this paper, all the original film damage images are numbered, classified and labeled (label 1 is crack, label 2 is dewetting, Label 3 is grain, label 4 is scratch), and finally all the images are imported into the computer in JPG format. The construction of four kinds of film damage image database is completed. All images in this database will be used as the input data set for the CBAM-ResNet50 convolutional neural network. The images in the training set, validation set, and test set are randomly selected by the computer.

The steel defect image dataset from Northeastern University is used to evaluate the generalization ability of the improved CBAM-ResNet50 network model. The images in the steel defect image data set of Northeastern University were collected by several teachers of Northeastern University. The data set includes seven defect types: Rolled-in scale, Patches, Crazing, Pitted, Surface, Inclusion, and Scratches, with 300 images for each defect type. In total, the dataset consists of 1800 grayscale images.

To remove irrelevant information and preserve the real, useful features in the image, some of the film damage images are corrupted with noise. To enhance the model‘s generalization ability, Preprocessing the acquired images is crucial. The main pretreatment steps are as follows:

• a)

  Image scaling, ResNet50 requires the size of the input image to be 224 × 224 × 3. Therefore, the size of the acquired image is scaled to 224 * 224 * 3 pixels to adapt to the training of the model;

• b)

  The main function of filtering and denoising is to suppress the noise of the image as much as possible on the premise of retaining the detailed information of the image [17];

• c)

  Edge extraction, which uses edge detection algorithm to extract image features, can effectively remove redundant information in the image. It has certain advantages in improving the classification accuracy [18];

• d)

  data enhancement, to enhance the generalization capability of the network model and prevent overfitting during training, data augmentation of the film damage image dataset is applied. In this paper, the original dataset is divided into three subsets: a training set with 49,760 images, a validation set with 6,220 images, and a test set with 6,220 images, following an 8:1:1 ratio. Only 80% of the training set undergoes data augmentation, while the validation and test sets do not undergo any data augmentation. In this paper, four kinds of image transformation techniques, including rotation, translation, shear and zoom, are used to enhance the images in the training set for seven times. There are 348320 images in the final training set [19];

In order to avoid the problems of over-fitting and slow convergence, this experiment uses ILSVRC2012 sub-dataset in ImageNet for transfer learning. Figure3 shows the comparison of the relationship curve between the number of iterations of the validation set and the accuracy rate of the CBAM-ResNet50 network model with and without transfer learning. Figure 4 compares the relationship between the number of iterations and the loss value on the training set, with and without transfer learning.

Figure 3.

Curve between the number of iterations and the accuracy with or without transfer learning

Figure 4.

Curve between the number of iterations and the loss value with or without transfer learning

As shown in Figure 3, the ResNet50 + transfer learning model demonstrates higher accuracy and faster convergence compared to the original ResNet50 model on the validation set. Furthermore, the curve becomes more stable after reaching convergence. The ResNet50 + transfer learning model tends to be stable at 30 epochs, while the original ResNet50 model tends to be stable at 68 epochs. The accuracy of ResNet50 + transfer learning model is about 0.85, while the accuracy of ResNet50 model is only about 0.65. This shows that the training accuracy of the network is higher, the training speed is faster, and the network can be stabilized faster after using transfer learning.

As shown in Figure 4, the final loss value of the original ResNet50 network model is relatively high, approximately 0.75. The final loss value of ResNet50 + transfer learning model is 0.4 less than that of ResNet50 +, only 0.3, and the loss curve using transfer learning decreases faster. Achieve stability more quickly.

In the task of image classification, the internal feature transmission mechanism of the traditional convolutional neural network often lacks clear distinction. As a result, in the process of feature extraction and transmission, effective information is easily submerged in a large number of irrelevant information. This type of feature propagation not only impacts the model‘s ability to capture crucial information but may also lead to performance degradation in complex scenarios. To overcome this challenge, this paper incorporates the channel attention mechanism from the CBAM module into the traditional ResNet50 architecture. O as to realize the optimization and improvement of the feature transmission process.

The placement of the attention mechanism plays a crucial role in determining the model’s recognition accuracy. Feature representations at different levels have different levels of abstraction and semantic information, and therefore, Embedding the attention mechanism in different locations of the network will produce different effects. To enhance the model‘s performance without altering its original structure, in this paper, we choose to add attention mechanism after the first and last convolution layer of ResNet50.This design can not only ensure that the attention mechanism can fully play its role, but it will not significantly affect the overall structure of the model. To verify the effectiveness of the attention mechanism‘s placement, four different addition schemes are designed in this paper, as shown in TABLE I.

During the experiment, the same data set and consistent training parameters are used to ensure the fairness of the comparison results. The results of the experiment are displayed in Figure 5.

Figure 5.

Performance comparison of CBAM-ResNet50 model with different addition modes of attention mechanism

As can be seen from Figure 5, when an attention module is added to the ResNet50 network, its recognition performance is slightly improved. This indicates that the attention mechanism can greatly improve the model‘s feature extraction capabilities, allowing it to more effectively capture the essential features within the image. When two attention modules are integrated into the network, the recognition accuracy of the film damage images reaches its optimal performance. This outcome strongly validates the effectiveness of the channel attention mechanism in image recognition tasks. The model can mine the potential information in the image more deeply, and further improve its accuracy on the validation set. It is also important to note that while adding more attention modules may result in higher performance gains, but this will also increase the complexity and computational cost of the model. Therefore, in practical applications, it is necessary to balance the performance and complexity of the model according to the specific task and resource constraints. Select the most appropriate number of attention modules and where to add them.

In this paper, The Alpha Dropout module and SeLU activation function are innovatively incorporated to enhance the performance of film damage image classification. The combination of these two techniques not only helps to maintain the stability of the distribution of the feature map, it can also effectively prevent the occurrence of overfitting phenomenon, this approach also accelerates the training process and improves the model‘s convergence speed.

First of all, the AlphaDropout module is used in the training process. The output of the neurons in the network is set to zero randomly with a certain probability, thereby avoiding the excessive dependence of the model on the output of a specific neuron, the randomness introduced by this approach helps enhance the model‘s generalization ability. To avoid overfitting, where the model performs well on the training set but shows a significant drop in performance on the test set, the AlphaDropout module is employed. This module effectively improves the model‘s robustness. It can maintain stable classification performance. Secondly, the SeLU activation function is selected, which has the characteristic of self-normalization. The distribution of the output value can be automatically adjusted during the activation process to be close to the standard normal distribution, This property helps preserve the stability of the feature map distribution, The information loss or distortion caused by the action of the activation function in the layer-by-layer transmission process is prevented, and meanwhile, The SeLU activation function also has a faster convergence rate and can achieve better classification results in a shorter training period.

In order to verify the effect of AlphaDropout combined with SeLU on the classification of film damage images by ResNet50 network, Based On the ResNet50 network using transfer learning and adding CBAM attention mechanism, AlphaDropout + ReLU is tested respectively. AlphaDropout + SeLU and the original ResNet50. The recognition results of the three models under the same conditions are shown in TABLE II. Compared with using ReLU activation function alone, the combination of AlphaDropout and SeLU achieves the highest accuracy of 90.58%. Compared with the combination of Alpha Dropout and ReLU, its accuracy is increased by 0.13%, which is mainly attributed to the activation function of SeLU. It effectively solves the problem of neuron "inactivation", This, in turn, this enhances the expressive capability of the network model. The normalization feature of SeLU, coupled with AlphaDropout, ensures that the output data retains a mean of 0 and a standard deviation of 1. As a result, the model‘s convergence speed is further accelerated.

A confusion matrix serves as a visual tool to assess the performance of classification models, and it is one of the important indicators to evaluate the results of models. It is often used to evaluate classifier models. In this research, the steel defect image dataset from Northeastern University was employed to evaluate the generalization capability of the CBAM-ResNet50 network model. The enhanced CBAM-ResNet50 model was applied to classify and recognize the test set data, the corresponding confusion matrix is shown in Figure 8.

Figure 8.

Confusion Matrix of Steel Defect Classification Recognition Results on CBAM-ResNet50

In this study, the performance indicators of CBAM-ResNet50 model in the task of steel defect image recognition are calculated by using confuse matrix. As shown in TABLE VI.

TABLE VI demonstrates that the precision, recall, and score for all six types of steel defects exceed 93%. Notably, for defects such as iron oxide scale pressing, cracking, pitting, and scratching, these metrics surpass 96.5%. The overall accuracy rate is 95.56%. ^F₁F₁.

From Figure VI, it is evident that the overall number of misrecognitions is low. This result fully proves that the CBAM-ResNet50 network model proposed in this paper shows excellent performance in the task of defect recognition. It also reflects its good generalization ability, which makes the model in the face of different types and different forms of defects. Can keep higher recognition accuracy, thereby greatly enhancing the universality and the reliability in practical application.