Have a personal or library account? Click to login
Implications of Pooling Strategies in Convolutional Neural Networks: A Deep Insight Cover

Implications of Pooling Strategies in Convolutional Neural Networks: A Deep Insight

By: Shallu Sharma and  Rajesh Mehra  
Open Access
|Aug 2019

References

  1. [1] Reddit, Machine Learning. Available: https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/clyj4jv/
  2. [2] Achille A., Soatto S., Information dropout: Learning optimal representations through noisy computation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 2897-2905.10.1109/TPAMI.2017.278444029994167
  3. [3] Boureau Y.-L., Le Roux N., Bach F., Ponce J., LeCun Y., Ask the locals: multi-way local pooling for image recognition, in Computer Vision (ICCV), 2011 IEEE International Conference on, 2011, 2651-2658.10.1109/ICCV.2011.6126555
  4. [4] Cai M., Shi Y., Liu J., Stochastic pooling maxout networks for low-resource speech recognition, in Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, 2014, 3266-3270.10.1109/ICASSP.2014.6854204
  5. [5] Cheng Y., Zhao X., Cai R., Li Z., Huang K., Rui Y., Semi-Supervised Multimodal Deep Learning for RGB-D Object Recognition, in IJCAI, 2016, 3345-3351.
  6. [6] Dalal N., Triggs B., Histograms of oriented gradients for human detection, in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 2005, 886-893.10.1109/CVPR.2005.177
  7. [7] DeVries T., Taylor G. W., Improved regularization of convolutional neural networks with cutout, arXiv preprint arXiv:1708.04552, 2017, 1-8.
  8. [8] Donahue J., Jia Y., Vinyals O., Hoffman J., Zhang N., Tzeng E., Darrell T., Decaf: A deep convolutional activation feature for generic visual recognition, in International conference on machine learning, 2014, 647-655.
  9. [9] Dumpala S. H., Chakraborty R., Kopparapu S. K., k-FFNN: A priori knowledge infused Feed-forward Neural Networks, arXiv preprint arXiv:1704.07055, 2017, 1-9.
  10. [10] Ellacott S., An analysis of the delta rule, in International Neural Network Conference, 1990, 956-959.10.1007/978-94-009-0643-3_145
  11. [11] Everingham M., Van Gool L., Williams C. K., Winn J., Zisserman A., The pascal visual object classes (voc) challenge, International journal of computer vision,88, 2, 2010, 303-338.10.1007/s11263-009-0275-4
  12. [12] Fei-Fei L., Fergus R., Perona P., Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories, Computer vision and Image understanding,106, 1, 2007, 59-70.10.1016/j.cviu.2005.09.012
  13. [13] Girshick R., Donahue J., Darrell T., Malik J., Rich feature hierarchies for accurate object detection and semantic segmentation, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, 580-587.10.1109/CVPR.2014.81
  14. [14] Goodfellow I., Bengio Y., Courville A., Deep learning1, MIT press Cambridge, 2016.
  15. [15] Goodfellow I. J., Warde-Farley D., Mirza M., Courville A., Bengio Y., Maxout networks, arXiv preprint arXiv:1302.4389, 2013, 1-9.
  16. [16] Graham B., Fractional max-pooling, arXiv preprint arXiv:1412.6071, 2014, 1-10.
  17. [17] Grauman K., Darrell T., The pyramid match kernel: Discriminative classification with sets of image features, in Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, 2005, 1458-1465.10.1109/ICCV.2005.239
  18. [18] He K., Zhang X., Ren S., Sun J., Spatial pyramid pooling in deep convolutional networks for visual recognition, in European conference on computer vision, 2014, 346-361.10.1007/978-3-319-10578-9_23
  19. [19] He K., Zhang X., Ren S., Sun J., Spatial pyramid pooling in deep convolutional networks for visual recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence,37, 9, 2015, 1904-1916.10.1109/TPAMI.2015.238982426353135
  20. [20] Hebb D., “The organization of behavior: a neuropsychological theory. Mahwah, NJ: L,” ed: Erlbaum Associates, 1949.
  21. [21] Hinton G. E., Srivastava N., Krizhevsky A., Sutskever I., Salakhutdinov R. R., Improving neural networks by preventing co-adaptation of feature detectors, arXiv preprint arXiv:1207.0580, 2012, 1-18.
  22. [22] Khan Z. H., Alin T. S., Hussain M. A., Price prediction of share market using artificial neural network (ANN), International Journal of Computer Applications,22, 2, 2011, 42-47.10.5120/2552-3497
  23. [23] Krizhevsky A., Sutskever I., Hinton G. E., Imagenet classification with deep convolutional neural networks, in Advances in neural information processing systems, 2012, 1097-1105.
  24. [24] Lang K. J., Hinton G. E., Dimensionality reduction and prior knowledge in e-set recognition, in Advances in neural information processing systems, 1990, 178-185.
  25. [25] Lazebnik S., Schmid C., Ponce J., Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, in null, 2006, 2169-2178.10.1109/CVPR.2006.68
  26. [26] LeCun Y., Generalization and network design strategies, Connectionism in perspective, 1989, 143-155.
  27. [27] LeCun Y., Bengio Y., Hinton G., Deep learning., Nature521, 2015, 436-444.10.1038/nature1453926017442
  28. [28] LeCun Y., Boser B., Denker J. S., Henderson D., Howard R. E., Hubbard W., Jackel L. D., Backpropagation applied to handwritten zip code recognition, Neural computation,1, 4, 1989, 541-551.10.1162/neco.1989.1.4.541
  29. [29] LeCun Y., Bottou L., Bengio Y., Haffner P., Gradient-based learning applied to document recognition, Proceedings of the IEEE,86, 11, 1998, 2278-2324.10.1109/5.726791
  30. [30] Lee C.-Y., Gallagher P. W., Tu Z., Generalizing pooling functions in convolutional neural networks: Mixed, gated, and tree, in Artificial Intelligence and Statistics, 2016, 464-472.
  31. [31] Lemley J., Bazrafkan S., Corcoran P., Smart Augmentation Learning an Optimal Data Augmentation Strategy, IEEE Access,5, 2017, 5858-5869.10.1109/ACCESS.2017.2696121
  32. [32] Lowe D. G., Distinctive image features from scale-invariant keypoints, International journal of computer vision,60, 2, 2004, 91-110.10.1023/B:VISI.0000029664.99615.94
  33. [33] McCulloch W. S., Pitts W., A logical calculus of the ideas immanent in nervous activity, The bulletin of mathematical biophysics,5, 4, 1943, 115-133.10.1007/BF02478259
  34. [34] Mehdipour Ghazi M., Kemal Ekenel H., A comprehensive analysis of deep learning based representation for face recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016, 34-41.10.1109/CVPRW.2016.20
  35. [35] Nagpal S., Singh M., Vatsa M., Singh R., Regularizing deep learning architecture for face recognition with weight variations, in Biometrics Theory, Applications and Systems (BTAS), 2015 IEEE 7th International Conference on, 2015, 1-6.10.1109/BTAS.2015.7358791
  36. [36] Nowlan S. J., Hinton G. E., Simplifying neural networks by soft weight-sharing, Neural computation,4, 4, 1992, 473-493.10.1162/neco.1992.4.4.473
  37. [37] Piccinini G., The First computational theory of mind and brain: a close look at mcculloch and pitts's “logical calculus of ideas immanent in nervous activity”, Synthese,141, 2, 2004, 175-215.10.1023/B:SYNT.0000043018.52445.3e
  38. [38] Plaut D. C., Experiments on Learning by Back Propagation, 1986, 1-49.
  39. [39] Rumelhart D. E., Hinton G. E., Williams R. J., Learning representations by back-propagating errors, nature,323, 6088, 1986, 533-536.10.1038/323533a0
  40. [40] Rumelhart D. E., McClelland J. L. (1986). Parallel distributed processing: explorations in the microstructure of cognition. volume 1. foundations.10.7551/mitpress/5236.001.0001
  41. [41] Scherer D., Müller A., Behnke S., Evaluation of pooling operations in convolutional architectures for object recognition, Springer, 2010, 92-101.10.1007/978-3-642-15825-4_10
  42. [42] Shallu, Mehra R., Automatic Magnification Independent Classification of Breast Cancer Tissue in Histological Images Using Deep Convolutional Neural Network, Singapore, 2019, 772-781.10.1007/978-981-13-3140-4_69
  43. [43] Shallu, Mehra R., Kumar S., “An insight into the convolutional neural network for the analysis of medical images,” presented at the Nanotechnology for Instrumentation and Measurement Workshop 2017.
  44. [44] Sharma S., Mehra R., Breast cancer histology images classification: Training from scratch or transfer learning?, ICT Express,4, 4, 2018, 247-254.10.1016/j.icte.2018.10.007
  45. [45] Shi Z., Ye Y., Wu Y., Rank-based pooling for deep convolutional neural networks, Neural Networks,83, 2016, 21-31.10.1016/j.neunet.2016.07.00327543927
  46. [46] Springenberg J. T., Dosovitskiy A., Brox T., Riedmiller M., Striving for simplicity: The all convolutional net, arXiv preprint arXiv:1412.6806, 2014,
  47. [47] Srivastava N., Hinton G., Krizhevsky A., Sutskever I., Salakhutdinov R., Dropout: a simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research,15, 1, 2014, 1929-1958.
  48. [48] Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A., Going deeper with convolutions, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, 1-9.10.1109/CVPR.2015.7298594
  49. [49] Szegedy C., Toshev A., Erhan D., Deep neural networks for object detection, in Advances in neural information processing systems, 2013, 2553-2561.
  50. [50] Wu H., Gu X., Max-pooling dropout for regularization of convolutional neural networks, in International Conference on Neural Information Processing, 2015, 46-54.10.1007/978-3-319-26532-2_6
  51. [51] Xu B., Wang N., Chen T., Li M., Empirical evaluation of rectified activations in convolutional network, arXiv preprint arXiv:1505.00853, 2015, 1-5.
  52. [52] Yadav N., Yadav A., Kumar M., Preliminaries of Neural Networks, An introduction to neural network methods for differential equations, 2015, 17-42.10.1007/978-94-017-9816-7_3
  53. [53] Yu D., Wang H., Chen P., Wei Z., Mixed pooling for convolutional neural networks, in International Conference on Rough Sets and Knowledge Technology, 2014, 364-375.10.1007/978-3-319-11740-9_34
  54. [54] Zeiler M. D., Fergus R., Stochastic pooling for regularization of deep convolutional neural networks, arXiv preprint arXiv:1301.3557, 2013, 1-9.
  55. [55] Zeiler M. D., Fergus R., Visualizing and understanding convolutional networks, in European conference on computer vision, 2014, 818-833.10.1007/978-3-319-10590-1_53
  56. [56] Zhai S., Wu H., Kumar A., Cheng Y., Lu Y., Zhang Z., Feris R. S., S3Pool: Pooling with Stochastic Spatial Sampling, in CVPR, 2017, 4003-4011.10.1109/CVPR.2017.426
  57. [57] Zhou B., Khosla A., Lapedriza A., Oliva A., Torralba A., Object detectors emerge in deep scene cnns, arXiv preprint arXiv:1412.6856, 2014, 1-12.
DOI: https://doi.org/10.2478/fcds-2019-0016 | Journal eISSN: 2300-3405 | Journal ISSN: 0867-6356
Language: English
Page range: 303 - 330
Submitted on: Sep 22, 2018
Accepted on: Apr 29, 2019
Published on: Aug 28, 2019
Published by: Poznan University of Technology
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2019 Shallu Sharma, Rajesh Mehra, published by Poznan University of Technology
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.