Neural networks · Explained

Deep Learning Fundamentals

How stacks of simple artificial neurons learn to see, listen, read and write, and why they're changing cybersecurity.

Start with a neuron Explore architectures

The big idea

What is Deep Learning?

Deep learning is a branch of machine learning that uses artificial neural networks with many layers. That depth is where the name comes from.

Classic machine learning needs a human to decide which features matter. A deep network learns its own features: early layers detect simple patterns like edges or character sequences, and deeper layers combine them into whole concepts like faces, words or a malware family. Given enough data and computing power (usually GPUs), this layered learning beats hand-built approaches at vision, speech and language.

Building block

The Artificial Neuron

Every network is built from one simple unit, loosely inspired by brain cells. A neuron:

takes several inputs (x₁, x₂, …)
multiplies each by a weight (w), which sets how important that input is
adds a bias (b), a learnable offset
passes the sum through an activation function (f), which adds non-linearity

Training a network simply means finding good values for millions, or even billions, of these weights and biases.

output of one neuron

y = f( w₁x₁ + w₂x₂ + … + wₙxₙ + b )

x = 0.9 (failed logins), w = 1.7

x = 0.2 (new device), w = 0.8

b = -1.0 → sum = 0.69

f = sigmoid → y ≈ 0.67 → "suspicious"

Forward pass

Feed a batch of examples through the network, layer by layer, to get predictions.

Measure the loss

A loss function scores how wrong the predictions are compared to the true labels.

Backpropagation

Work backwards through the network to find how much each weight contributed to the error.

Gradient descent

Nudge every weight a small step (the learning rate) in the direction that reduces the loss.

ReLU

f(x) = max(0, x)

The default for hidden layers: simple, fast, and it keeps gradients healthy in deep networks.

Sigmoid

f(x) = 1 / (1 + e^-x)

Squashes any value into 0–1, so it suits output layers that give a probability, such as "malicious or benign".

Tanh

f(x) = tanh(x)

Like sigmoid but centred on zero (−1 to 1). Common in older recurrent networks.

Images

CNN

Convolutional Neural Network. Slides small filters across the input to detect local patterns such as edges, textures and shapes.

Security use: classifying malware binaries rendered as images, CAPTCHA and screenshot analysis.

Sequences

RNN / LSTM

Recurrent networks keep a memory of earlier steps, so order matters. LSTMs add "gates" to remember longer.

Security use: modelling log sequences, network flows over time, and detecting DGA domains.

Language & more

Transformer

Uses self-attention to weigh every part of the input against every other part, in parallel. It powers today's LLMs.

Security use: phishing email detection, log summarisation, and AI assistants for SOC analysts.

Compression

Autoencoder

Learns to squeeze data into a small representation and rebuild it. Inputs it can't rebuild well are unusual.

Security use: unsupervised anomaly detection in traffic and user behaviour.

Generation

GAN

Generative Adversarial Network. A generator and a discriminator compete until the fakes look real.

Security angle: deepfakes, and also synthetic training data for rare attacks.

Generation

Diffusion

Learns to reverse a gradual noising process, turning random noise into detailed images, audio or video.

Security angle: realistic fake media that makes verifying content harder.

See it in code

A Tiny Network in PyTorch

import torch
from torch import nn

# 20 features in (e.g. URL / traffic stats) -> 1 probability out
model = nn.Sequential(
    nn.Linear(20, 64), nn.ReLU(),     # hidden layer 1
    nn.Linear(64, 32), nn.ReLU(),     # hidden layer 2
    nn.Linear(32, 1),  nn.Sigmoid(),  # output: P(malicious)
)
loss_fn = nn.BCELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

for epoch in range(10):
    for x, y in train_loader:
        pred = model(x)                 # 1. forward pass
        loss = loss_fn(pred, y)         # 2. measure the loss
        optimizer.zero_grad()
        loss.backward()                 # 3. backpropagation
        optimizer.step()                # 4. gradient descent

The four commented lines in the loop are the training loop shown above. train_loader stands for your own labelled dataset.

Speak the language

Key Terms

Epoch: one full pass over the training data.

Batch: the group of examples processed before each weight update.

Learning rate: the size of each weight update. Too high diverges, too low crawls.

Overfitting: memorising the training data instead of learning patterns that generalise.

Dropout: randomly switching off neurons during training to fight overfitting.

Parameters: the learned weights and biases. Large language models have billions.

Adversarial examples

Tiny, invisible changes to an input, like a few pixels or bytes, that flip the model's answer, for example making malware look benign.

Data poisoning

Slipping crafted samples into training data to plant blind spots or hidden backdoor triggers.

Model extraction

Querying a model's API repeatedly to build a copy of it, or to find inputs that slip past it.

Privacy leakage

Membership-inference and inversion attacks can reveal whether specific, possibly sensitive, records were in the training set.