## Search the FAQ Archives

3 - A - B - C - D - E - F - G - H - I - J - K - L - M
N - O - P - Q - R - S - T - U - V - W - X - Y - Z

# comp.ai.neural-nets FAQ, Part 1 of 7: IntroductionSection - What is a neural network (NN)?

( Part1 - Part2 - Part3 - Part4 - Part5 - Part6 - Part7 - Single Page )
[ Usenet FAQs | Web FAQs | Documents | RFC Index | Business Photos and Profiles ]

Top Document: comp.ai.neural-nets FAQ, Part 1 of 7: Introduction
Previous Document: May I copy this FAQ?
Next Document: Where can I find a simple introduction to NNs?
```
The question 'What is a neural network?' is ill-posed.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Pinkus
(1999)

First of all, when we are talking about a neural network, we should more
properly say "artificial neural network" (ANN), because that is what we mean
most of the time in comp.ai.neural-nets. Biological neural networks are much
more complicated than the mathematical models we use for ANNs. But it is
customary to be lazy and drop the "A" or the "artificial".

There is no universally accepted definition of an NN. But perhaps most
people in the field would agree that an NN is a network of many simple
processors ("units"), each possibly having a small amount of local memory.
The units are connected by communication channels ("connections") which
usually carry numeric (as opposed to symbolic) data, encoded by any of
various means. The units operate only on their local data and on the inputs
they receive via the connections. The restriction to local operations is
often relaxed during training.

Some NNs are models of biological neural networks and some are not, but
historically, much of the inspiration for the field of NNs came from the
desire to produce artificial systems capable of sophisticated, perhaps
"intelligent", computations similar to those that the human brain routinely
performs, and thereby possibly to enhance our understanding of the human
brain.

Most NNs have some sort of "training" rule whereby the weights of
connections are adjusted on the basis of data. In other words, NNs "learn"
from examples, as children learn to distinguish dogs from cats based on
examples of dogs and cats. If trained carefully, NNs may exhibit some
capability for generalization beyond the training data, that is, to produce
approximately correct results for new cases that were not used for training.

NNs normally have great potential for parallelism, since the computations of
the components are largely independent of each other. Some people regard
massive parallelism and high connectivity to be defining characteristics of
NNs, but such requirements rule out various simple models, such as simple
linear regression (a minimal feedforward net with only two units plus bias),
which are usefully regarded as special cases of NNs.

Here is a sampling of definitions from the books on the FAQ maintainer's
shelf. None will please everyone. Perhaps for that reason many NN textbooks
do not explicitly define neural networks.

According to the DARPA Neural Network Study (1988, AFCEA International
Press, p. 60):

... a neural network is a system composed of many simple processing
elements operating in parallel whose function is determined by
network structure, connection strengths, and the processing performed
at computing elements or nodes.

According to Haykin (1994), p. 2:

A neural network is a massively parallel distributed processor that
has a natural propensity for storing experiential knowledge and
making it available for use. It resembles the brain in two respects:

1. Knowledge is acquired by the network through a learning process.
2. Interneuron connection strengths known as synaptic weights are
used to store the knowledge.

According to Nigrin (1993), p. 11:

A neural network is a circuit composed of a very large number of
simple processing elements that are neurally based. Each element
operates only on local information. Furthermore each element operates
asynchronously; thus there is no overall system clock.

According to Zurada (1992), p. xv:

Artificial neural systems, or neural networks, are physical cellular
systems which can acquire, store, and utilize experiential knowledge.

References:

Pinkus, A. (1999), "Approximation theory of the MLP model in neural
networks," Acta Numerica, 8, 143-196.

Haykin, S. (1994), Neural Networks: A Comprehensive Foundation, NY:
Macmillan.

Nigrin, A. (1993), Neural Networks for Pattern Recognition, Cambridge,
MA: The MIT Press.

Zurada, J.M. (1992), Introduction To Artificial Neural Systems, Boston:
PWS Publishing Company.

```

## User Contributions:

Top Document: comp.ai.neural-nets FAQ, Part 1 of 7: Introduction
Previous Document: May I copy this FAQ?
Next Document: Where can I find a simple introduction to NNs?

Part1 - Part2 - Part3 - Part4 - Part5 - Part6 - Part7 - Single Page

[ Usenet FAQs | Web FAQs | Documents | RFC Index ]

Send corrections/additions to the FAQ Maintainer:
saswss@unx.sas.com (Warren Sarle)

Last Update March 27 2014 @ 02:11 PM