Table 1:
Comparative analysis of deepfake detection methods: key contributions, limitations, and relationship to proposed framework
| Reference | Category | Method | Key contribution | Research gaps |
|---|---|---|---|---|
| Rossler et al. [8] | Spatial (Dataset + CNN) | FF++ dataset & CNN baselines | Comprehensive benchmark with multiple manipulation types | Poor cross-dataset generalization |
| Chollet [9] | Spatial (CNN) | Xception architecture | Depthwise separable convolutions for efficient feature extraction | Limited robustness to adversarial attacks |
| Tan and Le [10] | Spatial (CNN) | EfficientNetV2 | Improved training efficiency and accuracy through compound scaling | Operates on raw inputs, vulnerable to compression |
| Nguyen et al. [11] | Spatial (analysis) | Cross-dataset evaluation study | Revealed severe performance drop across datasets | Identified problem but no robust solution |
| Li et al. [12] | Spatial (analysis) | CNN artifact analysis | CNNs exploit compression artifacts instead of semantic forgery | Needs alternative representations |
| Kingma and Welling [13] | Reconstruction | VAE | Foundation for latent representation and reconstruction | Not specific to deepfake detection |
| Durall et al. [14] | Reconstruction | Autoencoder-based detection | Reconstruction error reveals manipulation artifacts | No adversarial robustness or temporal modeling |
| Zhao et al. [15] | Reconstruction + attention | MADD | Combines reconstruction with attention for localization | Sensitive to compression, no temporal modeling |
| Jung et al. [16] | Reconstruction + CNN | DeepVision | Uses reconstruction loss as auxiliary feature | Limited robustness evaluation |
| Nataraj et al. [17] | Reconstruction + ML | Autoencoder + SVM | Uses reconstruction errors with classical ML | Limited to specific GAN types |
| Sabir et al. [18] | Temporal (RNN) | Recurrent CNN + LSTM | Captures temporal inconsistencies | Computationally expensive, uses raw frames |
| Güera and Delp [19] | Temporal (RNN) | ResNet + LSTM | Models inter-frame dependencies | Struggles with high-quality deepfakes |
| Amerini et al. [20] | Temporal (motion) | Optical flow-based detection | Detects unnatural motion patterns | Sensitive to camera motion, expensive |
| Bonettini et al. [21] | Temporal (transformer) | Video transformer networks | Self-attention for spatiotemporal learning | High computational cost |
| Zhao et al. [27] | Hybrid (multi-modal) | Ensemble multi-modal detection | Combines CNN with biological signals | Requires specific signals |
| Haliassos et al. [28] | Hybrid (audio-visual) | LIPS | Uses lip-sync inconsistencies | Requires audio data |
| Shiohara and Yamasaki [29] | Hybrid (frequency) | Frequency-domain analysis | Exploits GAN frequency artifacts | Vulnerable to adversarial attacks |
| Goodfellow et al. [22] | Robustness | FGSM | Fast adversarial attack method | Limited attack strength |
| Madry et al. [23] | Robustness | PGD | Strong iterative adversarial training | Computationally intensive |
| Carlini and Wagner [24] | Robustness | C&W attack | Demonstrates model vulnerabilities | Not specific to deepfake detection |
| Gandhi and Jain [25] | Robustness (evaluation) | Adversarial evaluation | Shows CNN vulnerability under attacks | No robust solution proposed |
| Zhang et al. [26] | Robustness (defense) | Adversarial training | Improves robustness in recognition | Not applied to deepfake detection |
| Proposed work | Hybrid (spatial + temporal + robustness) | Adversarial VAE + TCM + EfficientNetV2 | Unified residual-based spatial-temporal framework with adversarial robustness | Addresses robustness, temporal modeling, generalization, interpretability |
[i] CNN, convolutional neural network; C&W, Carlini-Wagner; FGSM, fast gradient sign method; GAN, generative adversarial networks; LIPS, logical inferences per second; MADD, multi-attentional deepfake detection; ML, machine learning PGD, projected gradient descent; RNN, Recurrent Neural Network; TCM, Temporal consistency module; VAE, variational autoencoder.
Figure 1:
Architecture of the proposed system. CNN, convolutional neural network; FGSM, fast gradient sign method; GRU, gated recurrent unit; PGD, projected gradient descent; VAEs, variational autoencoders.
Table 2:
Experimental setup
| Component | Summary |
|---|---|
| Datasets | FF++ (C40), 4 manipulation types; WildDeepFake for cross-domain evaluation |
| Preprocessing | 30 FPS frame extraction; (256 × 256) resizing; ImageNet normalization; basic augmentations |
| Models | VAE (latent dim 256); EfficientNetV2-S classifier; GRU/1D-CNN temporal module |
| Training settings | Adam optimizer; LR (1 × 10−4); VAE: 40 epochs; Classifier: 25–30 epochs |
| Evaluation metrics | Accuracy, Precision, Recall, F1, AUC |
| Robustness tests | FGSM and PGD applied during evaluation |
| Implementation | PyTorch 2.0, NVIDIA Tesla V100 GPU, mixed-precision (FP16) |
Table 3:
Performance on FF++ test set
| Method | Accuracy (%) | Precision (%) | Recall (%) | F1-score (%) | AUC |
|---|---|---|---|---|---|
| Raw EfficientNetV2 | 90.30 | 89.10 | 88.40 | 88.70 | 0.92 |
| Residual-only (VAE reconstruction error) | 92.10 | 91.80 | 91.00 | 91.40 | 0.94 |
| Proposed: VAE + EfficientNetV2 | 93.80 | 93.10 | 91.90 | 92.30 | 0.96 |
| Proposed (Adversarial-robust VAE) | 93.50 | 92.80 | 92.00 | 92.40 | 0.96 |
Table 4:
Cross-dataset generalization (Train: FF++ → Test: WildDeepFake)
| Method | Accuracy (%) | F1-score (%) | AUC |
|---|---|---|---|
| Raw EfficientNetV2 | 70.40 | 68.20 | 0.74 |
| Residual-only | 76.90 | 75.10 | 0.8 |
| Proposed: VAE + EfficientNetV2 | 80.10 | 78.60 | 0.84 |
Table 5:
Robustness under adversarial attacks
| Method | Clean AUC | Under FGSM | Under PGD |
|---|---|---|---|
| Raw EfficientNetV2 | 0.92 | 0.86 | 0.8 |
| Proposed VAE + EfficientNetV2 | 0.96 | 0.93 | 0.9 |
| Improvement over baseline | 0.04 | 0.07 | 0.1 |
Table 6:
Ablation study on FF++
| Configuration | Accuracy (%) | F1-score (%) |
|---|---|---|
| EfficientNetV2 (raw frames) | 90.30 | 88.70 |
| Residual-only | 92.10 | 91.40 |
| Residuals + Input (6-channel) | 93.80 | 92.30 |
| Proposed (Residual + TCM video fusion) | 95.4 (videos) | 94.20 |
Figure 2:
ROC curves comparing raw EfficientNetV2 and proposed residual-based methods. AUC, area-under-the-curve.
Figure 3:
Classifier loss curves.
Figure 4:
Confusion matrix.
Figure 5:
Spatial vs temporal score scatter plot.