Figure 1.
Mask datasets
Figure 2.
Image datasets
Figure 3.
Image inpainting methods framework
TABLE I.
Comparison of methods
| Feature | Progressive Image Inpainting | CNN | Transformer | Diffusion Models |
|---|---|---|---|---|
| Core Principles | Multi-stage processing (e.g., structure recovery followed by detail refinement, as in EdgeConnect's edge-prediction-and-filling stages) | Local feature extraction via convolutional kernels (e.g., Partial Conv's mask-aware convolution) | Global dependency modeling via self-attention (e.g., MAT's long-range reasoning) | Iterative denoising process for image generation (e.g., RePaint's stepwise restoration) |
| Key Strengths | 1.High structural integrity 2.Natural texture transitions (e.g., RFR-Net's recursive feature refinement) | 1.Strong local feature extraction 2.High computational efficiency (e.g., DeepFill's real-time performance) | 1.Robust global semantics 2.Effective for large missing regions (e.g., Swin Transformer's multi-scale fusion) | 1.Highest generation quality 2.Exceptional detail recovery (e.g., DiffBIR's realistic textures) |
| Key Weaknesses | 1.High computational complexity 2.Multi-stage training challenges (e.g., PRVS's convergence issues) | 1.Limited receptive field 2.Poor long-range dependency modeling (e.g., structural discontinuities in traditional CNNs) | 1.High resource consumption 2.Overfitting risks with small datasets (e.g., ViT's billion-scale pre-training requirement) | 1.Slow inference 2.High memory usage (e.g., DDPM's 1,000-step iterations) |
| Typical Use Cases | Complex structural restoration (e.g., artifact crack repair) | Small-area fast restoration (e.g., watermark removal from phone photos) | Large-area semantic restoration (e.g., street view occlusion removal) | High-fidelity detail generation (e.g., medical image super-resolution) |
| Computatio nal Efficiency | Moderate (requires multiple forward passes) | High (parallelizable computations) | Low (quadratic attention complexity) | Very Low (hundreds of denoising steps) |
| Training Data Needs | Moderate (requires structural annotations like edge maps) | Moderate (millions of images) | Very High (billion-scale pretraining) | Very High (massive high-quality datasets) |
| Representat ive Methods | EdgeConnect, RFR-Net | Partial Conv, DeepFill | MAT, SwinIR | RePaint, DiffBIR |