A neural network is a computational model made of many very simple units, arranged in layers and joined by weighted connections. Each unit adds up what reaches it and passes a signal on. The network’s knowledge sits not in lines of code but in the numbers on those connections, the weights, which are adjusted during training until the output fits.
The borrowing from the brain lives in the name and the basic idea, and ends there. An artificial neuron is an arithmetic rule, not a replica of a nerve cell. Read the network as biology and you will expect abilities it does not have.
How it is built
A single neuron
A neuron receives several numbers as input. Each is multiplied by its weight, everything is added up, and a fixed offset called the bias comes on top. A weight near zero means: this input does not interest me. A large negative weight means: it argues against. Nothing more happens here.
Layers
Neurons sit side by side in layers, and the layers one behind the other. The input layer takes in the data, one figure per feature or one value per pixel. The output layer delivers the result, for instance one number per possible category. In between sit the hidden layers, where the actual work happens. Having many such intermediate layers is what turns a network into a case of deep learning.
The activation function
Behind the sum sits a small function that decides what the neuron passes on. It is the most inconspicuous and most important part. Without it, all the layers together would amount to a single weighted sum, and the network could only describe straight relationships. Only that kink in the function makes it possible to represent curved and tangled ones.
The most widespread choice sets negative values to zero and lets positive ones through unchanged. It is cheap to compute and interferes with training less than the smooth curves that used to be standard.
How training works
At the start all the weights are random. The network is guessing, and guessing badly. The rest is a systematic shrinking of the error.
Measuring the error
The network receives a training example, runs it through and produces an output. That output is compared with the correct answer, and the deviation is condensed into a single number, the yardstick everything else follows. How it is formed is a substantive decision: a network meant to catch rare cases needs an error measure that punishes a miss more harshly than a false alarm.
Correcting backwards
Now the calculation runs backwards through the network. For every weight it works out in which direction and how strongly it influenced the error, then nudges it a small step the right way. The size of that step is the learning rate. Too large and the network overshoots every time. Too small and training takes forever.
This pass repeats for all the training data, many times over. One complete run through the data is called an epoch.
When to stop
Not when the error on the training data is at its lowest. Beyond a certain point the network memorises the examples along with their accidents and gets worse on new cases. So a second measurement runs on held-back data, and training ends when that stops improving. This overfitting is the most common reason for models that shine in the demo and disappoint in production.
Common network types
The architecture follows the structure of the data. A network for images is wired differently from one for text.
| Type | Built for | Principle |
|---|---|---|
| Feedforward | tables, key figures | signals travel once from front to back |
| Convolutional (CNN) | images, video | small filters slide across the image and detect local patterns wherever they sit |
| Recurrent (RNN, LSTM) | time series, speech | the network hands a state to the next step, giving it a short-term memory |
| Transformer | text, increasingly everything | all parts of the input are considered at once and weigh each other up |
| Autoencoder | anomalies, compression | the network learns to reconstruct its input through a bottleneck |
The transformer has displaced the other architectures for text and underpins today’s large language models. For images, convolutional networks remain widespread because they get by on less data.
Why traceability is difficult
A neural network has no single place where a rule is written down. What it learned is spread across all the weights, each one meaningless on its own. A decision emerges from the interplay of very many very small contributions. You can follow every step of the arithmetic and still not say why the network decided as it did.
There are tools that soften this. They show which region of an image or which feature tipped the balance, or what would have happened had an input looked different. Those are hints, not justifications, and they can mislead: a network that recognises ships reliably may be keying on the water behind them and fail inside a shipyard hall.
In practice that means three things. First, every model needs documentation of what data it was trained on and what it is not meant for. Second, fixed test cases that cover the awkward situations too. Third, where decisions affect people, somebody stays responsible and able to overrule. The EU AI Act turns this good practice into an obligation for some of these applications.
Frequently asked questions about neural networks
Does a network think? No. It maps a statistical relationship between input and output. That the results sometimes look considered comes from the volume of learned patterns, not from any understanding of the subject.
How many layers and neurons does a network need? As few as possible. An oversized network memorises, an undersized one cannot represent the relationship. The usual route is to adopt an architecture proven for the data type and change only what measurably helps.
Can I add new knowledge to a network afterwards? Only with care. Training a finished network further on new data can partly overwrite the old. The usual approach is to use old and new data together, or to keep the knowledge out of the network entirely and supply it at runtime through retrieval augmented generation.
What separates a neural network from classic machine learning? Methods such as decision trees or linear models produce a structure you can read. A neural network produces weights. In exchange it represents relationships the simpler methods fail on. There is more on the wider picture under machine learning.
Conclusion
Neural networks are surprisingly simple in construction. Weighted sums, an activation function, a procedure that distributes the error backwards. Their strength comes from sheer quantity, and that same quantity is their biggest weakness: nobody can cleanly justify an individual decision. Anyone deploying them should plan for that rather than discover it later. Where models like these genuinely help in projects and where we advise against them is shown in the overview of our work as an AI agency. How we build software, and where we use AI while doing it, is described on our page about custom software development. Our experience with AI in day-to-day development is written up in the article AI in software development, and for anything beyond that there is a free consultation.