Analysis of the ReSuMe Learning Process For Spiking Neural Networks

Filip Ponulak

doi:10.2478/v10006-008-0011-1

.blurhash-client-img { display: none !important; }

Analysis of the ReSuMe Learning Process For Spiking Neural Networks

International Journal of Applied Mathematics and Computer Science

Volume 18 (2008): Issue 2 (June 2008)

By: Filip Ponulak

Open Access

|Jun 2008

Abstract

In this paper we perform an analysis of the learning process with the ReSuMe method and spiking neural networks (Ponulak, 2005; Ponulak, 2006b). We investigate how the particular parameters of the learning algorithm affect the process of learning. We consider the issue of speeding up the adaptation process, while maintaining the stability of the optimal solution. This is an important issue in many real-life tasks where the neural networks are applied and where the fast learning convergence is highly desirable.

References

Bi G.-Q. (2002). Spatiotemporal specificity of synaptic plasticity: Cellular rules and mechanisms, Biological Cybernetics 87: 319-332.10.1007/s00422-002-0349-7
Search in Google Scholar
CSIM (2002). CSIM: A neural circuit SIMulator. The IGI LSM Group, Technical University, Graz http://www.lsm.tugraz.at
Search in Google Scholar
Freeman J. A. and Skapura D. M. (1991). Neural Networks Algorithms, Applications, and Programming Techniques, Addison-Wesley, Redwood City, CA.
Search in Google Scholar
Gerstner W. and Kistler W. (2002a). Mathematical formulations of Hebbian learning, Biological Cybernetics 87(5-6): 404-415.10.1007/s00422-002-0353-y
Search in Google Scholar
Gerstner W. and Kistler W. (2002b). Spiking Neuron Models. Single Neurons, Populations, Plasticity, Cambridge University Press, Cambridge.10.1017/CBO9780511815706
Search in Google Scholar
Hertz J., Krogh A. and Palmer R. (1991). Introduction to the Theory of Neural Networks, Addison-Wesley, Redwood City, CA.10.1063/1.2810360
Search in Google Scholar
Kangas J. and Kohonen T. (1996). Developments and applications of the self-organizing map and related algorithms, Mathematics and Computers in Simulation 41(1): 3-12(10).10.1016/0378-4754(96)88223-1
Search in Google Scholar
Kasiński A. and Kraft M. (2006). The design of a compact LIF-neuron circuit in FPGA to enable implementation of largescale spiking neuron networks with learning capabilities, Proceesings of the International Conference on Artificial Intelligence and Soft Computing, ICAISC'2006, Warsaw, Poland, pp. 57-64.
Search in Google Scholar
Kasiński A. and Ponulak F. (2005). Experimental demonstration of learning properties of a new supervised learning method for the spiking neural networks, Lecture Notes in Computer Science, Vol. 3696 pp. 145-153.
Search in Google Scholar
Kempter R., Gerstner W. and van Hemmen J. L. (1999). Hebbian learning and spiking neurons, Physical Review E 59(4): 4498-4514.10.1103/PhysRevE.59.4498
Search in Google Scholar
Korbicz J., Obuchowicz A. and Uciński D. (1994). Artificial Neural Networks: Foundations and Applications, Akademicka Oficyna Wydawnicza PLJ, Warsaw. (in Polish).
Search in Google Scholar
Kraft M., Kasiński A. and Ponulak F. (2006). Design of the spiking neuron having learning capabilities based on FPGA circuits, Proceedings of the 3rd International IFAC Workshop on Discrete-Event System Design, Rydzyna, Poland, pp. 301-306.
Search in Google Scholar
Maass W. and Bishop C. (Eds.) (1999). Pulsed Neural Networks, The MIT Press, Cambridge M.A.
Search in Google Scholar
Maass W., Natschlaeger T. and Markram H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Computation 14(11): 2531-2560.10.1162/08997660276040795512433288
Search in Google Scholar
Markram H., Luebke J., Frotscher M. and Sakmann B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs, Science 275(5297): 213-215.10.1126/science.275.5297.2138985014
Search in Google Scholar
Natschlaeger T., Maass W. and Markram H. (2002). The "liquid computer", a novel strategy for real-time computing on time series, Foundations of Information Processing of TELEMATIK 8(1): 32-36.
Search in Google Scholar
Papik K., Molnar B., Schaefer R., Dombovari Z., Tulassay Z. and Feher J. (1998). Application of neural networks in medicine—A review, Medical Science Monitor 4(3): 538-546.
Search in Google Scholar
Ponulak F. (2005). ReSuMe—New supervised learning method for Spiking Neural Networks, Technical Report, Institute of Control and Information Engineering, Poznań University of Technology. Available at http://d1.cie.put.poznan.pl/~fp/
Search in Google Scholar
Ponulak F. (2006a). ReSuMe—Proof of convergence, Technical Report, Institute of Control and Information Engineering, Poznan University of Technology. Available at http://d1.cie.put.poznan.pl/~fp/
Search in Google Scholar
Ponulak F. (2006b). Supervised Learning in Spiking Neural Networks with ReSuMe Method, Ph.D. thesis, Institute of Control and Information Engineering, Poznań University of Technology. Available at: http://d1.cie.put.poznan.pl/~fp/
Search in Google Scholar
Ponulak F., Belter D. and Kasiński A. (2006). Adaptive central pattern generator based on spiking neural networks, Proceedings of EPFL LATSIS Symposium 2006, Dynamical Principles for Neuroscience and Intelligent Biomimetic Devices, Lausanne, Switzerland, pp. 121-122.
Search in Google Scholar
Ponulak F. and Kasiński A. (2005). A novel approach towards movement control with spiking neural networks, Proceedings of the 3rd International Symposium on Adaptive Motion in Animals and Machines, Ilmenau, Germany. (Abstract).
Search in Google Scholar
Ponulak F. and Kasiński A. (2006a). Generalization Properties of SNN Trained with ReSuMe, Proceedings of the European Symposium on Artificial Neural Networks, ESANN'2006, Bruges, Belgium, pp. 623-629.
Search in Google Scholar
Ponulak F. and Kasiński A. (2006b). ReSuMe learning method for spiking neural networks dedicated to neuroprostheses control, Proceedings of EPFL LATSIS Symposium 2006, Dynamical Principles for Neuroscience and Intelligent Biomimetic Devices, Lausanne, Switzerland, pp. 119-120.
Search in Google Scholar
van Hemmen J. (2001). Theory of synaptic plasticity, in F.Moss and S.Gielen (Eds.) Handbook of Biological Physics, Neuro-informatics, Neural Modelling, Elsevier, Amsterdam, Vol. 4 pp. 771-823.
Search in Google Scholar