Short introduction to neural networks

This page supplements my lectures in the Methods in Computational Neuroscience summer course at Woods Hole.

Please contact me if you find any bugs in the following:

• Some problems in nonlinear and linear network theory.
• XPP code for the flip-flop and conductance-based autapse model.

The Hebbian paradigm

• W. James.  Psychology: the briefer course (1892).
• D. O. Hebb.  The organization of behavior (1949).
• D. J. Amit.  The Hebbian paradigm reintegrated: local reverberations as internal representations. Behav. Brain Sci. 18:617-26 (1995).

Linear network theory

• I don't know of a good treatment of this topic in the context of neural networks.
• H. S. Seung, D. D. Lee, B. Y. Reis, and D. W. Tank.  The autapse: a simple illustration of short-term analog memory storage by tuned synaptic feedback.  J. Comput. Neurosci. (2000).

Nonlinear network theory

• Lyapunov theory
• For symmetric networks, a Lyapunov function is known.
• J. J. Hopfield.  Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA 81:3088-92 (1984).
• A Lyapunov function is also known for a network consisting of two populations, assuming symmetric connections within each population, and antisymmetric connections between populations.
• H. S. Seung, T. J. Richardson, J. C. Lagarias, and J. J. Hopfield. Minimax and Hamiltonian dynamics of excitatory-inhibitory networks.  Advances in Neural Information Processing Systems 10, 329-35 (1998).
• general
• J. J. Slotine and W. Li.  Applied nonlinear control.  Prentice-Hall, 1991.
• Associative memory models
• J. J. Hopfield.  Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79:2554-8 (1982).
• M. V. Tsodyks. Associative memory in neural networks with the Hebbian learning rule. Modern Physics Letters B3:555-60 (1989).
• J. Hertz, A. Krogh, and R. G. Palmer.  Introduction to the Theory of Neural Computation, Addison-Wesley, 1991

Network models of persistent neural activity

• delayed-match-to-sample tasks
• M. Griniasty, M. V. Tsodyks, and D. J. Amit. Conversion of temporal correlations between stimuli to spatial correlations between attractors. Neural Comput. 5, 1-17 (1993).
• D. J. Amit, N. Brunel, and M. V. Tsodyks. Correlations of cortical Hebbian reverberations: theory versus experiment. J. Neurosci. 14, 6435-45 (1994).
• oculomotor integrator
• H. S. Seung. How the brain keeps the eyes still.  Proc. Natl. Acad. Sci. USA 93:13339-44 (1996).
• H. S. Seung, D. D. Lee, B. Y. Reis, and D. W. Tank.  Stability of the memory of eye position in a recurrent network of conductance-based model neurons. Neuron (2000).
• head direction system
• K. Zhang. Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory.  J. Neurosci. 16:2112-26 (1996).
• oculomotor delayed response task
• M. Camperi and X.-J. Wang. A model of visuospatial working memory in prefrontal cortex: recurrent network and cellular bistability.  J. Comput. Neurosci. 5:383-405 (1998).

Hybrid computation

• For an alternative to the attractor and amplifier ideas, see:
• R. Hahnloser et al. Digital selection and analog amplification coexist in a cortex-inspired silicon circuit. Nature (2000).

Sebastian Seung
seung@mit.edu