Generating Sea Surface Object Image Using Image-to-Image Translation

Yin, Wenbin; Yu, Jun; Hu, Zhiyi

doi:10.21307/ijanmc-2021-016

Abstract

Sea objects training, the conditional adversarial networks require a large number of images to solve image-to-image translation problems. In the case of insufficient samples, it leads to network overfitting and poor training results. This project proposes a conditional adversarial generative model that retains the original background features in the absence of paired samples. The goal of this project is to reduce the deviation of the corresponding output from the original input. Firstly, the object images of different categories are labeled with color masks. Second, sea objects are generated randomly in the original background using model of this project. Finally, the generated results of this approach are compared with other approaches. The experimental results show that, compared with results from other conditional adversarial generative models, the generated object images using model of this project have the characteristics of richer texture and clearer structure.

References

Goodfellow, J. Pouget-Abadie, M. Mirza, et al., Generative adversarial nets, Advances in neural information processing systems (2014), pp. 2672–2680.
Search in Google Scholar Back to article
M. Mirza, S. Osindero, Conditional Generative Adversarial Nets, Computer Science (2014) pp.2672–2680.
Search in Google Scholar Back to article
P. Isola, J.-Y. Zhu, T. Zhou, et al. Image-to-image translation with conditional adversarial networks, in the IEEE conference on computer vision and pattern recognition (CVPR) (2017), pp. 1125–1134.
Search in Google Scholar Back to article
T. C. Wang, M. Y. Liu, J. Y. Zhu, A. Tao, J. Kautz, B. Catanzaro, High-resolution image synthesis and semantic manipulation with conditional GANs, In the IEEE conference on computer vision and pattern recognition (CVPR) (2018), pp. 8798–8807.
Search in Google Scholar Back to article
M. Zhai, L. Chen, F. Tung, J. He, M. Nawhal, G. Mori, Lifelong gan: Continual learning for conditional image generation. In the IEEE International Conference on Computer Vision(ICCV) (2019), pp. 2759–2768.
Search in Google Scholar Back to article
D. Bau, H. Strobelt, W. Peebles, et al. Semantic photo manipulation with a generative image prior[J]. arXiv preprint arXiv:2005.07727, 2020.
Search in Google Scholar Back to article
X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, Infogan: Interpretable representation learning by information maximizing generative adversarial nets, Advances in neural information processing systems (NIPS) (2016), pp. 2172–2180.
Search in Google Scholar Back to article
J.Y. Zhu, R. Zhang, D. Pathak, T. Darrell, A. A. Efros, O. Wang, E. Shechtman, Toward multimodal image-to-image translation, Advances in neural information processing systems (NIPS) (2017), pp. 465–476.
Search in Google Scholar Back to article
J. Y. Zhu, T. Park, P. Isola, et al., Unpaired image-to-image translation using cycle-consistent adversarial networks, In the IEEE international conference on computer vision (ICCV) (2017), pp. 2223–2232.
Search in Google Scholar Back to article
W. Xian, P. Sangkloy, V. Agrawal, et al. Texturegan: Controlling deep image synthesis with texture patches, In the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 8456–8465.
Search in Google Scholar Back to article
Y. Lu, S. Wu, Y. W. Tai, et al., Image generation from sketch constraint using contextual gan, in the European Conference on Computer Vision (ECCV) (2018), pp. 205–220.
Search in Google Scholar Back to article
A. Gonzalez-Garcia, J. Van De Weijer, Y. Bengio, Image-to-image translation for cross-domain disentanglement, Advances in neural information processing systems (NIPS) (2018), pp. 1287–1298.
Search in Google Scholar Back to article
H. Tang, D. Xu, G. Liu, W. Wang, N. Sebe, Y. Yan, Cycle in cycle generative adversarial networks for keypoint-guided image generation. In the 27th ACM International Conference on Multimedia (2019, October), pp. 2052–2060.
Search in Google Scholar Back to article
Z. Gan, L. Chen, W. Wang, Y. Pu, Y. Zhang, H. Liu, C. Li, L. Carin, Triangle generative adversarial networks. In NIPS. (2017) pp. 5253–5262.
Search in Google Scholar Back to article
Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In CVPR. (2018)
Search in Google Scholar Back to article
Huang X, Liu M Y, Belongie S, et al. Multimodal unsupervised image-to-image translation. In ECCV. (2018)
Search in Google Scholar Back to article
M.-Y. Liu, T. Breuel, and J. Kautz. Unsupervised image-to-image translation networks. In NIPS, 2017.
Search in Google Scholar Back to article
Taigman, Y., Polyak, A., Wolf, L. Unsupervised cross-domain image generation. In ICLR. (2017)
Search in Google Scholar Back to article
K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, D. Krishnan, Unsupervised pixel-level domain adaptation with generative adversarial networks. In CVPR. (2017)
Search in Google Scholar Back to article
E. Hosseini-Asl, Y. Zhou, C. Xiong, R. Socher, Augmented cyclic adversarial learning for low resource domain adaptation (2018). arXiv preprint arXiv:1807.00374.
Search in Google Scholar Back to article
M. Y. Liu, X. Huang, A. Mallya, T. Karras, T. Aila, J. Lehtinen, J. Kautz, Few-shot unsupervised image-to-image translation, In the IEEE International Conference on Computer Vision (ICCV) (2019), pp. 10551–10560.
Search in Google Scholar Back to article
T. C. Wang, M. Y. Liu, A. Tao, G. Liu, J. Kautz, B. Catanzaro, Few-shot video-to-video synthesis, (2019) arXiv preprint arXiv:1910.12713.
Search in Google Scholar Back to article
A. Torralba, Contextual priming for object detection, International journal of computer vision, 2003, 53(2): 169–191.
Search in Google Scholar Back to article
X. Wang and A. Gupta. Generative image modeling using style and structure adversarial networks. In ECCV. (2016)
Search in Google Scholar Back to article
D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. In CVPR. (2016)
Search in Google Scholar Back to article
D. Yoo, N. Kim, S. Park, A. S. Paek, and I. S. Kweon. Pixel-level domain transfer. In ECCV. (2016)
Search in Google Scholar Back to article
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition. In the IEEE con ference on computer vision and pattern recognition (CVPR) (2016), pp. 770–778.
Search in Google Scholar Back to article

Generating Sea Surface Object Image Using Image-to-Image Translation

Abstract

Paradigm

My account