Figure 1
Different fuzzy types
Figure 2
Different CNN for image processing. (a) U-net or Codec network. (b) Multiscale or cascade refinement network. (c) Extended convolution network. (d) Scale recursive network (SRN).
Figure 3
(a) The original remaining network building blocks. (b) Building blocks of the modified network by NAH et al.
Table I
Comparison of characteristics of mainstream data sets
| Data Set | Construction Method | Advantages and Disadvantages |
|---|---|---|
| Levin etc. | Algorithm simulation fuzzy kernel | Easy to obtain; It is easy to obtain without considering local fuzziness; |
| Kupy etc. | Simulated trajectory | Easy to obtain; Only the motion in two-dimensional space is simulated, and the real three-dimensional space is not considered |
| Kohler etc. | The motion track is captured by 6D camera | The motion trajectories in three-dimensional space are collected; Lens distortion, depth of field variation, etc. are not considered |
| GOPRO etc. | Take the average value for continuous shooting by high-speed camera | Closer to the real fuzzy situation; The acquisition process is troublesome and the data scene is single |
| Lai etc. | Real acquisition | Completely real fuzzy pictures; There is no corresponding clear image, which is often used as a test set |
Table II
Blind deblurring algorithm based on deep learning
| Method | Applicable Scenario | Mechanism | Advantage | Limitations |
|---|---|---|---|---|
| Spatial variation RNN[22] | Motion blur, dynamic scene blur | The deblurring process is formulated through the wireless impulse response model | Weights can be learned from another network and different weights can be learned for different fuzzy systems | Large regional and spatial change structures need to be involved at the same time’ |
| SRN[20] | Motion blur | New multiscale cyclic network structure | The number of trainable parameters is reduced and the training efficiency is improved | Limited to fixed data sets and training periods |
| DMPHN[21] | Motion blur | End to end CNN hierarchical model similar to spatial pyramid matching | The required filter is small and can be inferred quickly | Requires large GPU memory |
| DPSR[32] | LR blurred image | A new SISR degradation model is designed | The deep plug and play framework can deal with any fuzzy kernel | For most real images, it does not match the degradation model |
| BIE-RVD[33] | Motion blur | Automatic coding structure of spatiotemporal video screen based on end-to-end differentiable structure | High accuracy and fast network running speed | The task of training is complex and difficult |
| DDMS[34] | Motion blur | A full convolution structure with filtering transformation and characteristic modulation is constructed | Real time filtering completely eliminates multi-scale processing and large filters | Real time filtering completely eliminates multi-scale processing and large filters |
| deblurGAN[27] DeblurGANV2[28] | Motion blur Motion blur | The generated countermeasure network based on perceptual loss [9] (perceptual loss) constraint is used for deblurring | The restored image is more similar to the target image in semantics and closer to people's subjective evaluation of image quality | The influence of different feature layers in the perceptual network on the perceptual loss is not considered, so that the restored image details are still smooth. |
| Deepdeblur[18] | dynamic scene blur | End to end multiscale convolution network | Without estimating the fuzzy kernel, multi-scale CNN can restore clear images directly and flexibly | The multi-scale stacked sub network results in large amount of parameters, large consumption of video memory and great difficulty in training |
| SRN-deblur[20] | Blur of dynamic scene | End to end multiscale cyclic network | Multi-scale structure and parameter sharing alleviate the problem of large amount of parameters, and the learning ability is more stable | The edge is too smooth and there are artifacts |
| DMPHN[21] | Motion blur | The deep-seated multi-facet network based on spatial pyramid matching processes fuzzy images through fine hierarchical representation. | It can solve the problem of performance saturation and run faster than multi-scale method | It can solve the problem of performance saturation and run faster than multi-scale method |
| MPRnet[35] | Deblurring, rain removing and noise removing | A multi-stage progressive image restoration | It can output accurate spatial details and context information. The network structure is simple and the effect is good | The deblurring effect under the dark light line is not good |
| MIMO-Net[29] | Motion blur | Single encoder multiple input single decoder multiple output | Increase the network feeling field and make the training less difficult | The spatial details are lost and the texture is not clear enough |
Figure 4
Visual comparison of image deblurring results of GoPro test set [13]. Patches blurred by key points are displayed in (b), while patches magnified from deblurring results are displayed in (c) – (h).