Pulsed Neural Networks

Pulsed Neural Networks

by Wolfgang Maass and Christopher M. Bishop (editors)
Published by MIT Press, Cambridge, MA, 1999
377 pages, $45
ISBN 0-12-443863-6

reviewed by
Dan Simon
Innovatia Software
dansimon@innovatia.com

Most artificial neural networks are based on the propagation of signals from one neuron to the next without regard to timing. This is not necessarily made explicit in most derivations, but the underlying assumption of most neural network theories is that the firing strength of a neuron is based on its mean firing rate. Data from neurobiological experiments make it clear, however, that biological neurons use not only mean rates but also the timing of the signals to transmit information and perform computations. For instance, the timing of neural action potentials is now a recognized means for encoding information in the sensory system of electric fish, the auditory system of bats, and the visual system of flies. Also, a rat's position in a maze has been reconstructed on the basis of neural spike timing.

Neural nets that depend on the timing of inter-neuron signals are called pulsed neural nets, and that is the topic of this book. The volume comprises the highlights of a two-day workshop held in 1997, and contains contributions by 28 leading experts in the field. The 14 chapters are roughly divided into three sections called Basic Concepts and Models, Implementations, and Design and Analysis of Pulsed Neural Systems.

The first chapter discusses and presents evidence for possible neural coding schemes. It has traditionally been thought (since the 1920s) that most neural information is contained in the mean firing rate of neurons. Recent experiments, though, have shown that this is too simple of a model to accurately describe brain activity. Reaction times in behavioral experiments are sometimes too short to be explained by temporal averaging. For instance, a fly can respond to a stimulus and change the direction of flight within 30 ms. This chapter also introduces several mathematical neuron models that use pulse coding.

The second chapter examines some ways that computing can be accomplished with pulsed neural nets. In addition, it presents theoretical evidence that indicates a pulsed neural net may possess more computational power than a traditional neural net of comparable size.

The third chapter describes the capabilities and limitations of Metal Oxide Silicon VLSI technology as related to pulsed neural nets. The justification for analog implementation comes from both size and power considerations. In addition, an artificial neural net typically interfaces with an analog outside world, meaning that a digital neural net must use interface and conversion circuitry that an analog neural net does not require.

The fourth chapter presents some real-world biological data on the firing properties of neurons and infers from that data some possible coding schemes. Examples are presented for which the mean firing rate hypothesis does not seem to be sufficient for information coding. It is clear that the mean firing rate does indeed contain information, but it also seems that the temporal firing rate contains additional information.

The next four chapters discuss the implementation of pulsed neural nets using neuromorphs, or networks of neural nets. The limited area of silicon chips constrains the amount of computation and the number of inputs and outputs that are available to a single neural net. Several methods for performing general communication between analog chips implementing neural nets have been developed and are discussed in these chapters.

Chapter 9 delves into some issues associated with the digital simulation of pulsed neural nets. This includes discussions of various mathematical models and ways to speed up simulations via algorithmic improvements, special-purpose computers, and parallel computing.

Chapter 10 discusses populations of neurons. Earlier in the book it was shown that biological experiments disproved the notion that neurobiological information is transmitted solely by mean firing rates. However, instead of an average over time, rate can be defined as an average over a population of neurons with similar properties.

Chapter 11 investigates the collective behavior of pulsed neurons, especially as related to the synchronization between the firing of different neurons. This chapter presents an example of edge detection and scene segmentation for image processing using a pulsed neural net.

Chapter 12 discusses dynamic weights. Most models of neural nets assume the inter-neuron weights are static; that is, they change only on the slow time scale of learning. This chapter presents experimental evidence that shows biological neurons have dynamic weights. With the introduction of this complexity, models of learning in neural nets need to be revised. This chapter shows that dynamic weights can be easily simulated by electronic circuits, and can therefore in principle be integrated into VLSI.

Chapter 13 introduces the use of stochastic bit streams for the simulation of pulsed neural nets. A neuron's action potentials are represented by individual bits being set, and the value of the potential is represented by the frequency with which the bits appear. This approach is demonstrated on a graph coloring problem, and FPGA's are proposed and discussed for hardware implementation.

The final chapter considers neural learning using pulse timing in the auditory system of barn owls. Barn owls use interaural time differences in the microsecond range for sound source localization.

This volume is interesting in that it explores a somewhat new and obscure subset of neural network theory. It places a strong emphasis on biology and on hardware implementations. The contributions are highly technical and research-oriented and do not attempt to provide a concise overview of the subject. Even those who are already familiar with neural nets will need to undertake additional study to gain prerequisite knowledge in order to exploit the information presented in this book.

Home Credentials Publications White Papers

Last Revised: March 13, 2001