What Is Deep Learning?

A receipt photograph contains coloured pixels. You see prices, a discount and a total. For software, getting from one to the other is a substantial part of the problem.

Someone could write rules for where the total usually appears, what a decimal point looks like and which letters spell “TOTAL”. Then a shop changes its layout. Or the photograph is tilted. Or the ink has faded.

Deep learning takes another approach: a neural network learns useful ways to represent the input through multiple processing layers. Those learned representations help it perform a task, such as extracting the total from a receipt image. [1][2]

But there is a catch worth understanding before we admire the result. Reading the printed total correctly does not mean checking the arithmetic correctly. Our receipt will make that distinction visible.

What makes learning “deep”?

Deep learning is a branch of machine learning built around neural networks with multiple layers of learned processing. A layer transforms numbers into another set of numbers; subsequent layers work with those transformed representations. [1]

The word “deep” refers to successive processing layers, not profound thought. Depth concerns how many transformations are stacked; width concerns the size of a layer. The parameter count counts adjustable values, such as weights. These are related design choices, not interchangeable measurements.

There is no single layer count that is useful as a universal dividing line across every architecture. For a beginner, the important idea is the composition: later transformations build on earlier ones.

The relationship is:

Artificial intelligence includes machine learning. Machine learning includes neural-network methods. Deep learning uses deep neural networks.

A small neural network need not be deep. And machine learning includes methods that are not neural networks at all.

If you have just read What Is Machine Learning?, the next question is not whether examples can change a model. It is what the model can learn between its input and its answer.

From choosing features to learning representations

Imagine building a receipt reader with a few measurements you chose in advance: the darkest region, the largest digits, or the number closest to the bottom of the image.

Those features might help. They also make assumptions about what matters. A receipt with a large loyalty-points balance beneath its total could upset the plan.

A deep network can learn intermediate features from training data instead of relying entirely on features someone has specified manually. This is called representation learning. A representation is simply a way of expressing information in a form the next computation can use. [1]

For an image, early processing might respond to local contrasts or short edges. Later processing can combine simpler patterns into more useful structures. For our receipt task, useful information could involve digit shapes, neighbouring text and their arrangement.

The network does not need to store an English instruction saying “look below the discount”. Its adjustable numerical operations can come to respond to patterns that help it produce the required output.

That does not remove human choices. Someone still chooses the task, training material, architecture and measures of success. Learned features are not automatically the right features for every purpose.

It is tempting to turn that picture into a tidy story: one layer finds edges, the next finds letters, the next understands the receipt. Real learned representations are less neatly labelled. Features can be distributed across many units, and a unit can respond to more than one kind of pattern. Visualisations help us investigate them; they do not make every layer a transparent human concept. [3]

Nor must every document model first run a separate text-recognition system. For example, the Donut research model was designed to extract document information without a separate optical character recognition stage. That is evidence of a possible design, not a claim that our receipt illustration is a Donut test. [4]

A correctly read receipt can still contain a mistake

Take another look at the three amounts:

For this invented receipt, there are no taxes, fees or other adjustments. All amounts use the same currency unit.

Byte Wonders

Byte asks: “If it reads ‘19.00’ perfectly, has it got the receipt right?”

It depends on what you asked it to do.

If the task is “extract the printed total”, 19.00 is correct. Returning 20.00 would silently alter the document.

If the task is “calculate the subtotal minus the discount”, the answer is 20.00. Returning 19.00 would be an arithmetic error.

If the task is “check whether the receipt adds up”, a useful answer is: “The printed total is 19.00, but 24.00 minus 4.00 is 20.00. The receipt differs by 1.00.”

One document, three tasks, three different standards of success.

This matters when training a system. If every target asks for the printed total, the training signal rewards extracting that value. It does not, by itself, reward noticing whether the receipt is internally consistent.

A sensible receipt-checking application might combine a learned document reader with ordinary arithmetic and explicit checks. Deep learning does not need to perform every operation just because it performs one difficult part.

Why multiple layers help—and why more is not a guarantee

Working through several learned transformations lets a network build useful combinations rather than rely only on a direct mapping from the original input to the answer. This is particularly valuable for complex inputs such as images, audio and language. [2]

But stacking transformations is not magic. If all the layers only performed linear operations, their combined calculation could be collapsed into a single linear transformation. Nonlinear operations are a crucial part of what makes many-layer networks more expressive. [5]

Our companion article on neural networks opens up one of those calculations. Here, the important point is what composition makes possible: a model can learn useful intermediate descriptions as part of learning the final task.

Byte’s Reality Check

Byte asks: “So why not keep adding layers until it gets everything right?”

Because the ability to represent a useful function and the ability to learn it successfully are different problems.

Deeper networks can be harder to optimise. Residual networks were developed to make very deep networks easier to train, using connections that let information bypass some transformations. That research demonstrated a useful architectural solution; it did not establish a rule that any extra layer improves any model. [6]

More capacity also cannot repair a badly specified task. Our extraction model could become an excellent receipt reader while still never being evaluated on arithmetic checking.

What does training require?

For a supervised receipt-extraction task, training examples would pair document images with the fields the system should return. The model’s outputs are compared with those targets, and training adjusts parameters to reduce a chosen loss—a numerical measure of error. Many layers can be adjusted as part of that process. [2]

The practical requirements depend on the job. Training a large model from scratch is different from adapting an existing model, and both differ from running an already trained model on a new receipt.

With transfer learning, a model starts from representations learned previously. One approach keeps a pretrained base fixed and trains a new task-specific part. Fine-tuning instead updates some or all of the pretrained parameters for the new task. These approaches can reduce the amount of new training needed; they do not remove the need for suitable data and evaluation. [7]

For our receipt reader, useful evaluation questions include:

• Can it extract totals from shops and layouts absent from training?
• What happens with blur, folded paper or unfamiliar number formats?
• Does it confuse a discount, tax amount or loyalty balance with the total?
• If the product promises to check receipts, does the evaluation actually test checking?

These are proposed checks, not results from an experiment we ran. They connect the model’s stated purpose to evidence we would need before relying on it.

Where the same idea becomes useful

The input need not be a receipt. Learned representations underpin deep-learning systems for recognising objects in images, processing speech and working with language. [2]

The details vary. A speech system receives numerical representations of sound, not printed characters. A language model works with tokens represented numerically, not a receipt photograph. Their architectures and objectives can differ too.

What connects them is the use of multiple learned transformations to make information useful for a task. What separates them is the task—and the evidence needed to show they perform it well.

A model that transcribes speech accurately has not thereby established whether the speaker’s claims are true. A model that produces a fluent answer has not thereby verified its contents. The receipt distinction travels well: recognising, producing and checking are different jobs.

Key takeaways

• Deep learning uses neural networks with multiple layers of learned processing.
• Its central advantage is learning intermediate representations, rather than relying entirely on manually selected features.
• Depth, width and parameter count describe different aspects of a model.
• More layers do not guarantee better training, better evidence or the right objective.
• Extracting a printed total, calculating an expected total and checking the difference require different definitions of success.

When you see a deep-learning demonstration, ask two questions: What transformation has it learned? And what does the demonstrated result actually prove?

Continue learning

Previous: What Is Machine Learning? — how training examples, features and objectives shape a model.

Next: What Are Neural Networks? — follow the numbers inside a small network and separate using it from changing it.

Then: What Are Large Language Models? — how learned language patterns become a generated answer.

Explore the AI Fundamentals learning path.

Sources

1. Goodfellow, Bengio and Courville, Deep Learning, Introduction. Representation learning, layered composition and the relationship between AI, machine learning and deep learning.
2. LeCun, Bengio and Hinton (2015), Deep learning. Public abstract supports multilayer representations, training through error signals and application areas.
3. Olah, Mordvintsev and Schubert (2017), Feature Visualization. Investigating learned features and the limits of simple semantic labels.
4. Kim and colleagues (2022), OCR-free Document Understanding Transformer. Primary research describing Donut; our receipt is not an evaluation of that model.
5. Google Machine Learning Crash Course, Nodes and hidden layers. Why stacking linear transformations alone is insufficient.
6. He and colleagues (2015), Deep Residual Learning for Image Recognition. Optimisation difficulties and residual architectures.
7. TensorFlow, Transfer learning and fine-tuning. Frozen feature extraction versus updating pretrained parameters.