Neural networks are models of nerve cells (neurons) or networks of neurons. While originally inspired by biology, they are often not realistic models of biological systems, nor do they try to be.
The basic idea is that a neuron receives signals from many other neurons or sensory cells via synaptic connections. These signals, weighted with the strength of the respective connection, determine whether the cell becomes active, i.e. starts sending signals itself. In the simplest mathematical model, synaptic connections are components of the so-called weight vector; the states of the connected neurons are components of the pattern vector; and the state of the cell is a function of the scalar product between weight and pattern.
The simplestg neural network, the perceptron, simply gives the sign of this scalar product as its output (which is also called the hidden field). The perceptron can therefore be used for simple classification tasks (the pattern belongs/ does not belong to a given category).
To make things more flexible, one can either use functions other than the sign, such as the error function erf, or connect several perceptrons in two or more layers. The amazing thing is that, by means of suitable algorithms, the neural network can "learn" from given data. That is, not only can it reproduce given combinations of inputs and outputs, it can also classify previously unseen data correctly if it is similar to examples shown before. For example, neural networks are used for identifying hand-written numbers, for sorting out interesting events in elementary particle detectors, and for identifying tumors on ultra-sonic scans.
Why is this interesting for physicists? On one hand, statistical physics offers many tools to make statements about the limits (storage capacity, generalization error etc.) of network architectures and the convergence of learning algorithms. On the other hand, neural networks are an example of systems in which complex phenomena arise from the interaction of simple units, which can also be found in many "traditional" systems, but is important enough that everyone should be interested in it.