Have a personal or library account? Click to login
Emerging Modularity During the Evolution of Neural Networks Cover
By: Tomasz Praczyk  
Open Access
|Mar 2023

References

  1. S. Ahmadian, S. Jalali, S. Islam, A. Khosravi, E. Fazli, and S. Nahavandi. A novel deep neuroevolution-based image classification method to diagnose coronavirus disease (covid-19). Comput Biol Med., (139:104994), 2021.10.1016/j.compbiomed.2021.104994855814934749098
  2. A. Baldominos, Y. Saez, and P. Isasi. Evolutionary convolutional neural networks: an application to handwriting recognition. Neurocomputing, 283:38–52, 2018.
  3. C.Y. Baldwin and K.B. Clark. Design Rules: The power of modularity. Chapter 3: What Is Modularity? MIT Press, 2018.
  4. A. Billard and M. J. Mataric. Learning human movements by imitation: evaluation of a biologically inspired connectionist architecture. Robotics and Autonomous Systems, 941:1–16, 2001.
  5. V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics Theory and Experiment, 10:10008, 2008.10.1088/1742-5468/2008/10/P10008
  6. R. Calabretta and J. Neirotti. Adaptive agents in changing environments, the role of modularity. Neural Process Lett, 42:257–274, 2015.10.1007/s11063-014-9355-8
  7. M. Carcenac. A modular neural network applied to image transformation and mental images. Neural Computing and Applications, 17:549–568, 2008.10.1007/s00521-007-0152-4
  8. J. Clune, B.E. Beckmann, P.K. McKinley, and C. Ofria. Investigating whether hyperneat produces modular neural networks. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, pages 635–642, 2010.10.1145/1830483.1830598
  9. J. Clune, J-B. Mouret, and H. Lipson. The evolutionary origins of modularity. In Proceedings of the Royal Society B, 2013.10.1145/2464576.2464596
  10. [10] A.S. Cofino, J.M. Gutierrez, and M.L. Ivanissevich. Evolving modular networks with genetic algorithms: application to nonlinear time series. Expert Systems, 21(4):208–216, 2004.
  11. Y. J. Cruz, M. Rivas, R. Quiza, A. Villalonga, R. E. Haber, and G. Beruvides. Ensemble of convolutional neural networks based on an evolutionary algorithm applied to an industrial welding process. Computers in Industry, 133:103530, 2021.10.1016/j.compind.2021.103530
  12. K. Deb, A. Pratap, S. Agarwal,, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA–II. IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002.
  13. S. Doncieux and J. Meyer. Evolving modular neural networks to solve challenging control problems. In Proceedings of the Fourth International ICSC Symposium on Engineering of Intelligent Systems, 2004.
  14. K. O. Ellefsen and J. Torresen. Evolving neural networks with multiple internal models. In Proceedings of the 14th European Conference on Artificial Life ECAL 2017, volume 14, pages 138–145, 2017.10.7551/ecal_a_025
  15. K.O. Ellefsen, J-B. Mouret, and J. Clune. Neural modularity helps organisms evolve to learn new skills without forgetting old skills. PLoS Computational Biology, 11(4):e1004128, 2015.10.1371/journal.pcbi.1004128438333525837826
  16. C. Espinosa-Soto and A. Wagner. Specialization can drive the evolution of modularity. PLoS Computational Biology, 6(3):e1000719, 2010.10.1371/journal.pcbi.1000719284794820360969
  17. C. Fernando, D. Banarse, M. Reynolds, F. Besse, D. Pfau, M. Jaderberg, M. Lanctot, and D.Wierstra. Convolution by evolution: differentiable pattern producing networks. In Proceedings of the 2016 Genetic and Evolutionary Computation Conference, pages 109–116, 2016.10.1145/2908812.2908890
  18. D. Filan, S. Hod, C. Wild, A. Critch, and S. Russell. Pruned neural networks are surprisingly modular. Technical Report arXiv:2003.04881 [cs.NE], ArXiV, 2020.
  19. J. Gauci and K. Stanley. Generating large–scale neural networks through discovering geometric regularities. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 997–1004, 2007.10.1145/1276958.1277158
  20. S. Han and S. Oh. An optimized modular neural network controller based on environment classification and selective sensor usage for mobile robot reactive navigation. Neural Computation and Application, 17:161–173, 2008.10.1007/s00521-006-0079-1
  21. J. Huizinga, J.B. Mouret, and J. Clune. Evolving neural networks that are both modular and regular: Hyperneat plus the connection cost technique. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pages 697–704, 2014.10.1145/2576768.2598232
  22. M. Hulse, S. Wischmann, and F. Pasemann. Structure and function of evolved neuro– controllers for autonomous robots. Connection Science, 16(4):249–266, 2004.10.1080/09540090412331314795
  23. L. Kirsch, J. Kunze, and David Barber. Modular networks: Learning to decompose neural computation. Technical Report arXiv:1811.05249 [cs.LG], ArXiV, 2018.
  24. J. Koutnik, J. Schmidhuber, and F. Gomez. Evolving deep unsupervised convolutional networks for vision–based reinforcement learning. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pages 541–548, 2014.10.1145/2576768.2598358
  25. V. Landassuri-Moreno and J. A. Bullinaria. Biasing the evolution of modular neural networks. In 2011 IEEE Congress of Evolutionary Computation, 2011.10.1109/CEC.2011.5949855
  26. J. Liang, E. Meyerson, and R. Miikkulainen. Evolutionary architecture search for deep multitask networks. In GECCO ’18: Proceedings of the Genetic and Evolutionary Computation Conference, pages 466–473, 2018.10.1145/3205455.3205489
  27. I. Loshchilov and F. Hutter. CMA–ES for hyper-parameter optimization of deep neural networks. Technical Report arXiv: abs/1604.07269 [cs.NE], ArXiV, 2016.
  28. P. Melin, D. Bravo, and O. Castillo. Fingerprint recognition using the fuzzy sugeno integral for response integration in modular neural networks. International Journal of General Systems, 37(4):499–515, 2008.10.1080/03081070701321910
  29. R. Miikkulainen, J. Liang, E. Meyerson, A. Rawal, D. Fink, O. Francon, B. Raju, H. Shahrzad, A. Navruzyan, N. Duffy, and B. Hodjat. Evolving deep neural networks. Technical Report arXiv abs/1703.00548 [cs.NE], ArXiV, 2017.
  30. J-B. Mouret and S. Doncieux. Evolving modular neural networks through exaptation. In 2009 IEEE Congress on Evolutionary Computation, pages 1570–1577, 2009.10.1109/CEC.2009.4983129
  31. H. Munn and M. Gallagher. Modularity in NEAT reinforcement learning networks, 2022.
  32. N. NourAshrafoddin, A. R. Vahdat, and M. M. Ebadzadeh. Automatic design of modular neural networks using genetic programming. In Proceedings of the 17th International Conference on Artificial Neural Networks ICANN 2007 Part I, pages 788–798, 2007.10.1007/978-3-540-74690-4_80
  33. M. Potter. The Design and Analysis of a Computational Model of Cooperative Coevolution. PhD thesis, George Mason University, 1997.
  34. M. A. Potter and K. A. De Jong. Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evolutionary Computation, 8(1):1–29, 2000.
  35. T. Praczyk. Probabilistic neural network application to warship radio stations identification. Computational Methods in Science and Technology, 13(1):53–58, 2007.10.12921/cmst.2007.13.01.53-57
  36. T. Praczyk. Using augmenting modular neural networks to evolve neuro–controllers for a team of underwater vehicles. Soft Computing, 18(12):2445–2460, 2014.10.1007/s00500-014-1221-0
  37. T. Praczyk. Cooperative co–evolutionary neural networks. Journal of Intelligent & Fuzzy Systems, 30(5):2843–2858, 2016.10.3233/IFS-162095
  38. T. Praczyk. Hill climb modular assembler encoding: Evolving modular neural networks of fixed modular architecture. Knowledge-Based Systems, 232:107493, nov 2021.10.1016/j.knosys.2021.107493
  39. K. Soltanian, A. Ebnenasir, and M. Afsharchi. Modular grammatical evolution for the generation of artificial neural networks. Evolutionary Computation, 30(2):291–327, 06 2022.10.1162/evco_a_0030234878521
  40. S. Sotirov, E. Sotirova, V. Atanassova, K. Atanassov, O. Castillo, P. Melin, T. Petkov, and S. Surchev. A hybrid approach for modular neural network design using intercriteria analysis and intuitionistic fuzzy logic. Complexity, 1:1–11, 2018.10.1155/2018/3927951
  41. K. O. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99–127, 2002.10.1162/10636560232016981112180173
  42. Y. Sun, B. Xue, M. Zhang, and G. G. Yen. Automatically designing CNN architectures using genetic algorithm for image classification. Technical Report arXiv:1808.03818 [cs.NE], ArXiV, 2018.
  43. C. R. Tosh. Can computational efficiency alone drive the evolution of modularity in neural networks? Scientific Reports, 6:31982, 2016.10.1038/srep31982500415227573614
  44. C. R. Tosh and L. McNally. The relative efficiency of modular and non–modular networks of different size. In Proceedings of the Royal Society B: Biological Sciences, volume 282:20142568, 2015.10.1098/rspb.2014.2568434415225631996
  45. A. Turan, S. D. Hinchberger, and M. H. El Naggar. Predicting the dynamic properties of glyben using a modular neural network (MNN). Canadian Geotechnical Journal, 45:1629–1638, 2008.10.1139/T08-054
  46. V. K. Valsalam and R. Miikkulainen. Evolving symmetric and modular neural networks for distributed control. In Proceedings of the Genetic and Evolutionary Computation Conference, 2009.10.1145/1569901.1570002
  47. L. Xie and A. Yuille. Genetic CNN. Technical Report arXiv abs/1703.01513 [cs.NE], ArXiV, 2017.
  48. X. Yao and Y. Liu. A new evolutionary system for evolving artificial neural networks. IEEE Transactions on Neural Networks, 8(3):694–713, 1997.10.1109/72.57210718255671
  49. S.R. Young, D.C. Rose, T.P. Karnowsky, S.H. Lim, and R.M. Patton. Optimizing deep learning hyper–parameters through an evolutionary algorithm. In Proceedings of the Workshop on Machine Learning in High–Performance Computing Environments, number 4, pages 1–5, 2015.10.1145/2834892.2834896
  50. Z. Zhu, S. Guo, and M. Liao. Deep neuroevolution: Evolving neural network for character locomotion controller. In 2021 2nd International Conference on Artificial Intelligence and Information Systems, ICAIIS 2021, New York, NY, USA, 2021. Association for Computing Machinery.10.1145/3469213.3470259
Language: English
Page range: 107 - 126
Submitted on: Feb 9, 2022
Accepted on: Oct 19, 2022
Published on: Mar 11, 2023
Published by: SAN University
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2023 Tomasz Praczyk, published by SAN University
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.