Have a personal or library account? Click to login
Inverse Meets Distillation: Heterogeneous Teacher–Assistant Dual-Path Learning for Unsupervised Defect Detection Cover

Inverse Meets Distillation: Heterogeneous Teacher–Assistant Dual-Path Learning for Unsupervised Defect Detection

Open Access
|Feb 2026

References

  1. J. Ghanim, M. Awad, An unsupervised anomaly detection in electricity consumption using reinforcement learning and time series forest based framework, Journal of Artificial Intelligence and Soft Computing Research 15 (2025).
  2. J. Zhang, H. Dong, J. Gao, R. Yao, G. Li, H. Wu, Self-organized operational neural networks for the detection of atrial fibrillation, Journal of Artificial Intelligence and Soft Computing Research 14 (2024).
  3. W. Zhou, J. Hong, W. Yan, Q. Jiang, Modal evaluation network via knowledge distillation for no-service rail surface defect detection, IEEE Transactions on Circuits and Systems for Video Technology 34 (2024) 3930–3942.
  4. Q. Yu, K. Zhu, Y. Cao, F. Xia, Y. Kang, Tf²: Few-shot text-free training-free defect image generation for industrial anomaly inspection, IEEE Transactions on Circuits and Systems for Video Technology 34 (2024) 11825–11837.
  5. Y. Yang, D. Liu, mkdnad: A network flow anomaly detection method based on multi-teacher knowledge distillation, in: 2022 16th IEEE International Conference on Signal Processing (ICSP), volume 1, IEEE, 2022, pp. 314–319.
  6. P. Bergmann, M. Fauser, D. Sattlegger, C. Steger, Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 4183–4192.
  7. Y. Liu, W. Zhang, J. Wang, Adaptive multi-teacher multi-level knowledge distillation, Neurocomputing 415 (2020) 106–113.
  8. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255.
  9. K. Batzner, L. Heckler, R. König, Efficientad: Accurate visual anomaly detection at millisecond-level latencies, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 128–138.
  10. M. Rudolph, T. Wehrbein, B. Rosenhahn, B. Wandt, Asymmetric student-teacher networks for industrial anomaly detection, in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2023, pp. 2592–2602.
  11. P. Bergmann, M. Fauser, D. Sattlegger, C. Steger, Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9592–9600.
  12. Y. Zou, J. Jeong, L. Pemula, D. Zhang, O. Dabeer, Spot-the-difference self-supervised pre-training for anomaly detection and segmentation, in: European Conference on Computer Vision, Springer, 2022, pp. 392–408.
  13. P. Mishra, R. Verk, D. Fornasier, C. Piciarelli, G. L. Foresti, Vt-adl: A vision transformer network for image anomaly detection and localization, in: 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), IEEE, 2021, pp. 01–06.
  14. M. Ke, C. Lin, Q. Huang, Anomaly detection of logo images in the mobile phone using convolutional autoencoder, in: 2017 4th International Conference on Systems and Informatics (ICSAI), IEEE, 2017, pp. 1163–1168.
  15. Y. Li, W. Zhao, J. Pan, Deformable patterned fabric defect detection with fisher criterion-based deep learning, IEEE Transactions on Automation Science and Engineering 14 (2016) 1256–1264.
  16. D. Abati, A. Porrello, S. Calderara, R. Cucchiara, Latent space autoregression for novelty detection, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 481–490.
  17. Y. Qiu, T. Misu, C. Busso, Unsupervised scalable multimodal driving anomaly detection, IEEE Transactions on Intelligent Vehicles 8 (2022) 3154–3165.
  18. T. Schlegl, P. Seeböck, S. M. Waldstein, U. Schmidt-Erfurth, G. Langs, Unsupervised anomaly detection with generative adversarial networks to guide marker discovery, in: International conference on information processing in medical imaging, Springer, 2017, pp. 146–157.
  19. H. Chung, J. Park, J. Keum, H. Ki, S. Kang, Unsupervised anomaly detection using style distillation, IEEE Access 8 (2020) 221494–221502.
  20. X. Wang, Y. Du, S. Lin, P. Cui, Y. Shen, Y. Yang, advae: A self-adversarial variational autoencoder with gaussian anomaly prior knowledge for anomaly detection, Knowledge-Based Systems 190 (2020) 105187.
  21. D. Gong, L. Liu, V. Le, B. Saha, M. R. Man-sour, S. Venkatesh, A. v. d. Hengel, Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection, in: Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1705–1714.
  22. W. Li, Z. Shang, J. Zhang, M. Gao, S. Qian, A novel unsupervised anomaly detection method for rotating machinery based on memory augmented temporal convolutional autoencoder, Engineering Applications of Artificial Intelligence 123 (2023) 106312.
  23. K. Wu, L. Zhu, W. Shi, W. Wang, J. Wu, Self-attention memory-augmented wavelet-cnn for anomaly detection, IEEE Transactions on Circuits and Systems for Video Technology 33 (2023) 1374–1385.
  24. M. Salehi, A. Eftekhar, N. Sadjadi, M. H. Rohban, H. R. Rabiee, Puzzle-ae: Novelty detection in images through solving puzzles, arXiv preprint arXiv:2008.12959 (2020).
  25. M. Haselmann, D. P. Gruber, P. Tabatabai, Anomaly detection using deep learning based image completion, in: 2018 17th IEEE international conference on machine learning and applications (ICMLA), IEEE, 2018, pp. 1237–1242.
  26. C.-L. Li, K. Sohn, J. Yoon, T. Pfister, Cutpaste: Self-supervised learning for anomaly detection and localization, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 9664–9674.
  27. M. Pei, N. Liu, B. Zhao, H. Sun, Self-supervised learning for industrial image anomaly detection by simulating anomalous samples, International Journal of Computational Intelligence Systems 16 (2023) 152.
  28. Q. Ma, E. Zhang, Y. Chen, J. Duan, L. Shao, Sia-net: Structural information awareness network based on normal samples for surface defect detection, Engineering Applications of Artificial Intelligence 126 (2023) 107131.
  29. Q. Chen, H. Luo, H. Gao, C. Lv, Z. Zhang, Progressive boundary guided anomaly synthesis for industrial anomaly detection, IEEE Transactions on Circuits and Systems for Video Technology (2024) 1–1.
  30. B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, R. C. Williamson, Estimating the support of a high-dimensional distribution, Neural computation 13 (2001) 1443–1471.
  31. D. M. Tax, R. P. Duin, Support vector data description, Machine learning 54 (2004) 45–66.
  32. L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, M. Kloft, Deep one-class classification, in: International conference on machine learning, PMLR, 2018, pp. 4393–4402.
  33. P. Perera, V. M. Patel, Learning deep features for one-class classification, IEEE Transactions on Image Processing 28 (2019) 5450–5463.
  34. W. Mao, G. Wang, L. Kou, X. Liang, Deep domain-adversarial anomaly detection with one-class transfer learning, IEEE/CAA Journal of Automatica Sinica 10 (2023) 524–546.
  35. J. Wang, X. Wang, R. Hao, H. Yin, B. Huang, X. Xu, J. Liu, Incremental template neighborhood matching for 3d anomaly detection, Neurocomputing 581 (2024) 127483.
  36. Z. Hu, Z. Yin, L. Qin, F. Xu, A novel method of fault diagnosis for injection molding systems based on improved vgg16 and machine vision, Sustainability 14 (2022) 14280.
  37. W. Ma, S. Lan, W. Huang, W. Wang, H. Yang, Y. Ma, Y. Ma, A semantics-aware normalizing flow model for anomaly detection, in: 2023 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2023, pp. 2207–2212.
  38. M. Rudolph, B. Wandt, B. Rosenhahn, Same same but differnet: Semi-supervised defect detection with normalizing flows, in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 1907–1916.
  39. M. Salehi, N. Sadjadi, S. Baselizadeh, M. H. Rohban, H. R. Rabiee, Multiresolution knowledge distillation for anomaly detection, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14902–14912.
  40. X. Zhang, S. Li, X. Li, P. Huang, J. Shan, T. Chen, Destseg: Segmentation guided denoising student-teacher for anomaly detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 3914–3923.
  41. S. Yamada, S. Kamiya, K. Hotta, Reconstructed student-teacher and discriminative networks for anomaly detection, in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2022, pp. 2725–2732.
  42. T. Fukuda, M. Suzuki, G. Kurata, S. Thomas, J. Cui, B. Ramabhadran, Efficient knowledge distillation from an ensemble of teachers., in: Inter-speech, 2017, pp. 3697–3701.
  43. S. You, C. Xu, C. Xu, D. Tao, Learning from multiple teacher networks, in: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 2017, pp. 1285–1294.
  44. M.-C. Wu, C.-T. Chiu, K.-H. Wu, Multi-teacher knowledge distillation for compressed video action recognition on deep neural networks, in: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019, pp. 2202–2206.
  45. H. Zhang, D. Chen, C. Wang, Adaptive multi-teacher knowledge distillation with meta-learning, in: 2023 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2023, pp. 1943–1948.
  46. F. Yuan, L. Shou, J. Pei, W. Lin, M. Gong, Y. Fu, D. Jiang, Reinforced multi-teacher selection for knowledge distillation, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021, pp. 14284–14291.
  47. Y. Ma, X. Jiang, N. Guan, W. Yi, Anomaly detection based on multi-teacher knowledge distillation, Journal of systems architecture 138 (2023) 102861.
  48. M. Pei, N. Liu, P. Gao, H. Sun, Reverse knowledge distillation with two teachers for industrial defect detection, Applied Sciences 13 (2023) 3838.
  49. J. Hao, K. Huang, C. Chen, J. Mao, Dual-student knowledge distillation for visual anomaly detection, Complex & Intelligent Systems (2024) 1–13.
  50. Q. Zhou, S. He, H. Liu, T. Chen, J. Chen, Pull & push: Leveraging differential knowledge distillation for efficient unsupervised anomaly detection and localization, IEEE Transactions on Circuits and Systems for Video Technology 33 (2023) 2176–2189.
  51. M. Tan, Q. Le, Efficientnet: Rethinking model scaling for convolutional neural networks, in: International conference on machine learning, PMLR, 2019, pp. 6105–6114.
  52. A. Waswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: NIPS, 2017.
Language: English
Page range: 275 - 291
Submitted on: Sep 26, 2025
|
Accepted on: Jan 29, 2026
|
Published on: Feb 25, 2026
Published by: SAN University
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2026 Xin Zhan, Yaqian Li, Ruihao Chen, Wenming Zhang, Haibin Li, published by SAN University
This work is licensed under the Creative Commons Attribution 4.0 License.