Rendering automatic bokeh recommendation engine for photos using deep learning algorithm

Rakesh Kumar; Meenu Gupta; Jaismeen; Shreya Dhanta; Nishant Kumar Pathak; Yukti Vivek; Ayush Sharma; Deepak; Gaurav Ramola; Sudha Velusamy

doi:10.2478/ausi-2022-0015

Abstract

Automatic bokeh is one of the smartphone’s essential photography effects. This effect enhances the quality of the image where the subject background gets out of focus by providing a soft (i.e., diverse) background. Most smartphones have a single rear camera that is lacking to provide which effects need to be applied to which kind of images. To do the same, smartphones depend on different software to generate the bokeh effect on images. Blur, Color-point, Zoom, Spin, Big Bokeh, Color Picker, Low-key, High-Key, and Silhouette are the popular bokeh effects. With this wide range of bokeh types available, it is difficult for the user to choose a suitable effect for their images. Deep Learning (DL) models (i.e., MobileNetV2, InceptionV3, and VGG16) are used in this work to recommend high-quality bokeh effects for images. Four thousand five hundred images are collected from online resources such as Google images, Unsplash, and Kaggle to examine the model performance. 85% accuracy has been achieved for recommending different bokeh effects using the proposed model MobileNetV2, which exceeds many of the existing models.

References

[1] S.Bakhshi,D.Shamma,L.Kennedy,E.Gilbert, Whywefilter ourphotosand how it impacts engagement, Proc. 9th International AAAI Conference on Web and social media, 2015, pp. 12–21. ⇒25010.1609/icwsm.v9i1.14622
Search in Google Scholar Back to article
[2] S. Dutta, Depth-aware blending of smoothed images for bokeh effect generation. J. Visual Comm. Image Represent. 77 (2021) 103089. ⇒25110.1016/j.jvcir.2021.103089
Search in Google Scholar Back to article
[3] V. Gajarla, A. Gupta, Emotion detection and sentiment analysis of images. Georgia Institute of Technology, 1 (2015) 1–4. ⇒251
Search in Google Scholar Back to article
[4] V. D. Gesu, M. Maccarone, An approach to random images analysis. Proc. of the Springer II Conference In Image Analysis and Processing, 1988, pp 111–118. ⇒25010.1007/978-1-4613-1007-5_10
Search in Google Scholar Back to article
[5] R. Guha. Improving the performance of an artificial intelligence recommendation engine with deep learning neural nets, Proc. 6th EEE International Conference for Convergence in Technology (I2CT), 2001, pp 1–7. ⇒253
Search in Google Scholar Back to article
[6] A. Ignatov, J. Patel, and R. Timofte. Rendering natural camera bokeh effect with deep learning. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp 418–419. ⇒249, 25210.1109/CVPRW50498.2020.00217
Search in Google Scholar Back to article
[7] A. Ignatov, J. Patel, R. Timofte, B. Zheng, X. Ye, L. Huang, X. Tian, S. Dutta, K. Purohit, P. Kandula, et al., Aim 2019 challenge on bokeh effect synthesis: Methods and results. Proc. of IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019, pp 3591–3598. ⇒25010.1109/ICCVW.2019.00444
Search in Google Scholar Back to article
[8] P. Isola, J. Xiao, A. Torralba, A. Oliva, What makes an image memorable?, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp 145–152. ⇒25010.1109/CVPR.2011.5995721
Search in Google Scholar Back to article
[9] R. Jain, M. Gupta, S. Taneja, D. J. Hemanth, Deep learning based detection and analysis of covid-19 on chest x-ray images. Applied Intelligence, 51, 3 (2021) 1690–1700. ⇒25110.1007/s10489-020-01902-1754476934764553
Search in Google Scholar Back to article
[10] H.-H. Li, Y.-H. Liao, Y.-N. Huang, P.-J. Cheng, Based on machine learning for personalized skin care products recommendation engine. Proc. of the IEEE International Symposium on Computer, Consumer and Control (IS3C), 2020, pp 460–462. ⇒252
Search in Google Scholar Back to article
[11] Y. Liu, C. Yuan, Automatic segmentation of background defocused nature image, Proc. of the IEEE 2nd International Congress on Image and Signal Processing, 2009, pp 1–5. ⇒249, 25110.1109/CISP.2009.5304662
Search in Google Scholar Back to article
[12] W. Niu, J. Caverlee, H. Lu, Neural personalized ranking for image recommendation, Proc. of the 11th ACM international conference on web search and data mining, 2018, pp 423–431. ⇒24910.1145/3159652.3159728
Search in Google Scholar Back to article
[13] B. Ramzan, I. S. Bajwa, N. Jamil, R. U. Amin, S. Ramzan, F. Mirza, and N. Sarwar. An intelligent data analysis for recommendation systems using machine learning. Scientific Programming, 2019 (2019). ⇒25110.1155/2019/5941096
Search in Google Scholar Back to article
[14] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.-C. Chen, Mobilenetv2: Inverted residuals and linear bottlenecks. Proc. of the IEEE conference on computer vision and pattern recognition, 2018, pp 4510–4520. ⇒25310.1109/CVPR.2018.00474
Search in Google Scholar Back to article
[15] F. Saxen, P. Werner, S. Handrich, E. Othman, L. Dinges, A. AlHamadi, Face attribute detection with mobilenetv2 and nasnet-mobile, Proc. of 11th the IEEE International Symposium on Image and Signal Processing and Analysis (ISPA), 2019, pp 176–180. ⇒25210.1109/ISPA.2019.8868585
Search in Google Scholar Back to article
[16] X. Shen, A. Hertzmann, J. Jia, S. Paris, B. Price, E. Shechtman, I. Sachs, Automatic portrait segmentation for image stylization. In Computer Graphics Forum, 35, 2 (2016) 93–102. ⇒24910.1111/cgf.12814
Search in Google Scholar Back to article
[17] W.-T. Sun, T.-H. Chao, Y.-H. Kuo, W. H. Hsu, Photo filter recommendation by category-aware aesthetic learning. IEEE Transactions on Multimedia, 19, 8 (2017) 1870–1880. ⇒250, 25210.1109/TMM.2017.2688929
Search in Google Scholar Back to article
[18] N. Wadhwa, R. Garg, D. E. Jacobs, B. E. Feldman, N. Kanazawa, R. Carroll, Y. Movshovitz-Attias, J. T. Barron, Y. Pritch, M. Levoy, Synthetic depth-of-field with a single-camera mobile phone. ACM Transactions on Graphics (ToG), 37, 4 (2018) 1–13. ⇒24910.1145/3197517.3201329
Search in Google Scholar Back to article
[19] F. Wang, Y. Zhang, Y. Ai, W. Zhang, Rendering natural bokeh effects based on depth estimation to improve the aesthetic ability of machine vision. Machines, 10, 5 (2022) 286. ⇒250, 25310.3390/machines10050286
Search in Google Scholar Back to article
[20] X. Xu, D. Sun, S. Liu, W. Ren, Y.-J. Zhang, M.-H. Yang, J. Sun, Rendering portraitures from monocular camera and beyond, Proc. of the European Conference on Computer Vision (ECCV), 2018, pp 35–50. ⇒24910.1007/978-3-030-01240-3_3
Search in Google Scholar Back to article

Rendering automatic bokeh recommendation engine for photos using deep learning algorithm

Abstract

Paradigm

My account