9.641 Neural Networks

Lecture schedule

• Lecture 1 (9/7): Class introduction and description (PDF) and the rate equations (PDF)
  • Introduction to Neurons and Classical Neurodynamics by Prof. Seung (PDF)
  • Older notes by Prof. Seung (PDF)
  • Christof Koch, Biophysics of Computation, Section 14.2: 335-341, Oxford University Press, New York, Oxford (1999).
  • Bard Ermentrout, Reduction of Conductance-Based Models with Slow Synapses to Neural Nets, Neural Computation. 6,679-695 (1995).
  • Dayan & Abbott, Chapter 5, Theoretical Neuroscience (MIT only).
• Lecture 2 (9/14): Synaptic Organization (PDF) and Perceptrons (PDF)
• Lecture 3 (9/19): Vision and Computation. (Multi-layer Perceptrons: PDF, Convolutional Neural Networks / Vision: PDF)
  • D. Marr, Vision, Section 2.2, 54-79, W.H. Freeman and Company, New York (1982).
  • D.H. Hubel, Eye, Brain, and Vision, Chapter 3, 39-46, Scientific American Library, New York, (1988-1995).
• Lecture 4 (9/21): Hierarchical Visual Processing and Convolutional Networks
  • Kunihiko Fukushima, A Hierarchical Neural Network Capable of Visual Pattern Recognition, Neural Networks. (1988).
  • Yann Lecun, Gradient-Based Learning Applied to Document Recognition, Proc. of the IEEE. (1998).
  • Yann Lecun, Fu-Jie Huang, and Leon Bottou, Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting, Proceedings of CVPR'04. (2004).
• Lecture 5 (9/26): Conditioning (PDF), Hebbian Learning (PDF), and the Delta Rule (PDF)
• Lecture 6 (9/28): Backpropagation (PDF)
  • Older notes on delta rule (PDF) and backpropagation (PDF) by Prof. Seung
  • Yann LeCun, Leon Bottou, et al. Efficient backprop, in G. Orr and K. Muller, Neural networks: tricks of the trade, Springer, 1998.
  • David A. Robinson, Implications of neural networks for how we think about brain function, Behav. Brain. Sci., 15, 644-55 (1992).
  • Review of the multidimensional chain rule
  • C. M. Bishop's derivation of backpropagation, with an example and some notes on the computational complexity of backprop. from Neural Networks for Pattern Recognition, Oxford, 1995.
• Lecture 7 (10/3): More Backpropagation, Backpropagation in Recurrent Networks (PDF)
  • F. Pineda, "Generalization of Backpropagation to Recurrent Neural Networks", Physical Review Letters, Vol. 59, No. 19, 1987.
• Lecture 8 (10/5): Symmetric Networks (PDF)
  • Summary of Lyapunov analysis: Chapter 3 from Slotine,1991
• Lecture 9 (10/12): Contrastive Hebbian Learning (PDF); Stochastic Networks (Boltzmann Machine) and the Mean-Field Approximation (PDF)
  • Movellan, J. (1990). Contrastive Hebbian learning in the continuous Hopfield model. In D. Touretzky, J. Elman, T. Sejnowski,&G. Hinton (Eds.), Proceedings of the 1990 Connectionist Models Summer School, San Mateo, CA: Morgan Kaufmann. (PDF)
  • Movellan, J. (1990). Contrastive Hebbian Learning in Interactive Networks. (unpublished). (PDF)
  • X. Xie and H. S. Seung. Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural Computation 15, 441-54 (2003). (PDF)
• Lecture 11 (10/17):
• Lecture 12 (10/19): Computer Vision; Edge Detection
• Lecture 13 (10/24): Viren "Convolution" Jain on Convolutional Networks (PDF)

  ---

  Last Updated on Monday, 30-Oct-2006 13:05:38 EST .