Sapienza University of Rome · PhD Soft Skills 2026
Artificial Intelligence, from the Inside
A one-day course for doctoral students of every discipline.
No mathematics. No programming. No prior knowledge.
Module 1 · 9:00 – 12:00 · Fabrizio Silvestri
The promise
By the end of today you will have built a working
research assistant that reads your documents and answers questions about them.
You will not write a single line of code.
You will describe what you want.
The machine will write the program. Your job is to know what to ask for — and to know when the answer is wrong.
The deal we are making
What this course is, and is not
It is
A working understanding of how these systems actually behave
A method for getting a machine to build tools for you
A trained instinct for when the output is nonsense
Seven hands-on exercises you keep
It is not
A programming course
A mathematics course
A tour of the newest products
A promise that AI will do your research for you
Why we are not teaching you to code
The skill that used to matter, and the one that does now
Until recently
To make a computer do something new, you had to learn its language.
That took months, and most researchers reasonably decided it was not worth it.
Now
The computer learned ours. The bottleneck moved from
syntax to specification: saying precisely what you want, and checking what you got.
Why this matters for you
Precise specification and rigorous verification are the two things a doctorate trains you to do.
You are not starting from zero here — you are starting from an advantage.
But there is a catch
You cannot ask for what you cannot imagine,
and you cannot catch a lie you don't know is possible.
That is why the morning is about how these systems work —
not for its own sake, but because every failure you will meet this afternoon
has a cause you will have already seen.
The day
Where we are going
Time
Morning · understanding
9:00
What "learning from examples" actually means
9:40
Exercise 1 Teach a machine to sort your texts
10:10
Neurons, images, sequences — and what each one cannot see
10:50
Break
11:05
Transformers: why 2017 changed everything
11:35
Exercise 3 Make a language model hallucinate on purpose
Time
Afternoon · building
14:00
From a model to a chatbot, and prompting as a craft
15:20
Grounding the machine in your documents
16:00
Capstone Build your own research agent
We are doing this online — so, four rules
How today works
Use the chat, freely. If a word goes past that you don't know, put it in the chat.
That is my mistake, not yours. I will stop and answer.
Hands-on is optional. Every notebook ships with the results already in it, and I will
run each exercise on screen myself. If you'd rather watch, watch — you will lose nothing.
Nothing you do can break anything. The notebooks run on Google's computers, not yours.
Worst case, close the tab and reopen it.
There is no "getting it right". Half of today is deliberately making the machine fail.
Nobody is checking your screen.
Two minutes, everyone
Setup
Open a browser and go to fabsilvestri.github.io/ai-from-the-inside
Click Warm-up under Notebooks. It opens in Google Colab.
Sign in with any Google account.
Click Copy to Drive in the toolbar. Every notebook opens
read-only — this one click makes it yours, and is the difference between keeping today's
work and losing it.
Click the ▶ button on the first cell. If you see a green tick, you are ready.
Nothing to install. Nothing to pay for. Nothing to configure.
Hold on to this
Everything you will use today — every chatbot, every assistant, every "AI" —
is a machine that guesses the next fragment of text, over and over,
wrapped in a loop.
If that sounds too simple to be true, good. Let's spend three hours earning it.
1
Rules, or examples
The only fork in the road that matters
A word of warning about the word
"Artificial intelligence" has meant something different
in every decade since 1956.
Chess engines were AI. Then they were "just search".
Spam filters were AI. Then they were "just statistics". Whatever works stops being called AI.
This is sometimes called the "AI effect": AI is, by definition, the part we haven't demystified yet. Today we demystify some of it.
A definition we can actually use
A working definition
A system is doing "AI" when it produces useful behaviour that nobody
explicitly programmed, because it derived that behaviour from data.
Not a philosophical claim. An engineering one. It says nothing about
understanding, consciousness, or intention — and neither will we.
Two ways to make a machine do anything
The fork in the road
1 · Write the rules
You, the human, work out the logic and state it exactly.
The machine follows it, forever, without deviation.
Example. "A sonnet has 14 lines." Count the lines. Done.
2 · Show examples
You collect cases where you already know the answer.
The machine works out its own rules to reproduce them.
Example. 2,000 letters you have already marked angry or polite.
Everything in this course lives on the right-hand side.
Where rules win
Rules are wonderful — when the world is tidy
Is this year a leap year? A four-line rule. Perfect, forever, no data needed.
Is this a valid tax code? A checksum. Exactly right or exactly wrong.
Does this manuscript have 14 lines? Count them.
Do not forget this
If you can write the rule, write the rule. It will be faster, cheaper, auditable and correct.
Using AI where a rule would do is not sophistication — it is waste.
Where rules collapse
Now write me the rules for "this letter is angry"
Attempt
Contains an exclamation mark → angry
Contains the word "unacceptable" → angry
Written in capitals → angry
Immediately breaks on
"Congratulations!!! Wonderful news!"
"I find the delay unacceptable, but understandable."
Any 18th-century letter, where capitals mean emphasis, not fury
Any letter that is furious and impeccably polite — the most common kind
The pattern
Every rule you add fixes one case and breaks two others.
The exception list grows faster than the rule list.
This is not a failure of effort. For anything involving human meaning,
the rules genuinely do not exist in a writable form.
So stop writing rules. Collect examples instead, and let the machine find the rule.
The definition, now earned
Machine learning
A program that improves at a task by being shown examples,
rather than by being given instructions.
Examples2,000 letters, each already marked angry or polite
→
LearningThe machine hunts for patterns that separate the two piles
→
ModelA thing that can now judge a letter it has never seen
Note what the machine is not given: any theory of anger, any dictionary,
any instruction about what to look at.
Getting the vocabulary straight
Four words, nested inside each other
Artificial intelligence — the whole ambition. Includes rule-based systems from the 1960s.
Machine learning — the part that learns from examples. Includes very simple methods.
Deep learning — many-layered neural networks. Since ~2012.
Large language models — deep learning applied to text, at enormous scale.
Since ~2018.You are here
Before we go further
What "learning" does not mean here
No understanding
The system has no concept of a letter, a person, or anger.
It has statistical regularities over symbols.
No goals
It does not want to be right. There is no intention, preference, or
strategy anywhere in it.
No memory
Once trained, it does not learn from you. Your conversation today
changes nothing about it tomorrow.
The word "learning" is a technical term of art, borrowed loosely.
It is the single biggest source of misunderstanding about these systems.
Section 1 · what to carry forward
Takeaway
If you can write the rule, write the rule. Use machine learning only when the
rule cannot be written — which, for anything involving human meaning, is almost always.
Playbook line 1: "Before reaching for AI, ask whether a rule would do."
2
Data, training, evaluation
The three parts every AI system is made of — including the ones you'll use this afternoon
The whole of machine learning, on one slide
Three parts. That is all there is.
① DATAExamples where the answer is already known
→
② TRAININGAdjusting the machine until it reproduces those answers
→
③ EVALUATIONTesting it on examples it has never seen
Every argument you will ever have about an AI system is an argument about
one of these three. Usually the first.
① Data
A dataset is a table where somebody
already wrote down the answer.
That "somebody" is a person. Remember them — we come back to them in four slides.
① Data
What a dataset actually looks like
Text (the input)
Label (the answer)
"I am writing to express my dismay at the handling of…"
angry
"Thank you so much for your prompt and generous reply…"
polite
"This is the third time I have had to ask."
angry
"I trust this finds you well."
polite
… 1,996 more rows
…
Two columns. Left: what the machine sees. Right: what it should say.
This shape — inputs and answers — is the shape of nearly all machine learning.
① Data
"Ground truth" is a remarkably confident name
Ground truth (n.) — the labels in the right-hand column,
treated by the system as if they were facts about the world.
What it sounds like
Objective reality, measured.
What it usually is
The opinion of some people, under time pressure, following a guideline document
that was itself a compromise.
① Data · the question to always ask
Who labelled it, and by what definition?
Who? Domain experts? Students? Crowdworkers paid per item?
Under what definition? Was "angry" defined, or left to intuition?
Did they agree? If two labellers disagree 30% of the time, no model can be more than 70% "right".
Who is missing? Which languages, periods, genres, and people are not in the table at all?
Playbook line 2
"Whose judgement is encoded in this system's answers?" — ask it of every AI tool you are asked to trust.
① Data · where it goes wrong
The machine learns the data, not the world
Hiring
A system trained on a decade of a company's hiring decisions learns
to reproduce that decade's hiring decisions — including whom it declined to hire, and why.
Medical imaging
A skin-lesion classifier learns that images containing a ruler are malignant —
because clinicians only reach for a ruler when they are already worried.
Language
A model trained mostly on English and mostly on the last twenty years
is quietly terrible at everything else — including most of the material in your theses.
Archives
A model trained on digitised holdings learns the biases of what was
digitised first: the famous, the well-funded, the already-canonical.
A model cannot be fairer, broader, or more current than the table it was trained on.
No amount of computing power fixes a missing column.
② Training
What "training" actually does
Guess. Check. Adjust. Repeat — a few million times.
The machine makes a guess about one example. At the start, the guess is random.
It compares its guess to the known answer. The gap is called the error.
It nudges its internal settings slightly, in the direction that would have reduced that error.
Next example. Repeat until the nudges stop helping.
That is the entire algorithm. It is astonishingly dumb, and it works
because it is repeated on a scale humans cannot picture.
② Training · a picture
A wall of dials
Picture a machine with a wall of dials. Text goes in one end, a judgement comes out the other.
Training turns the dials — a fraction of a degree at a time — until the outputs
match the answers you already know.
How many dials?
~10³A simple text classifier (Exercise 1)
~10¹²A frontier language model (this afternoon)
A billion times more dials.
Same procedure, turning every one of them.
② Training · the deflating truth
"The model" is nothing but the final positions of the dials.
A large file full of numbers. No rules, no facts, no sentences stored anywhere —
just settings that happen to produce useful behaviour.
③ Evaluation
Now the part that everybody gets wrong, including people who should know better.
③ Evaluation
You must hide some of the data from it
TRAINING SET · 80%The machine sees these examples and their answers.
TEST SET · 20%Locked in a drawer until the very end.
Then the only honest question: how well does it do on the 20% it has never seen?
Performance on the 80% tells you nothing at all. A machine that memorised
every example scores 100% there and is completely useless.
③ Evaluation · the cardinal sin
Testing on data the machine has already seen
It sounds too obvious to be a real problem. It is the most common serious error in applied AI.
The same document appears in both piles under a different filename.
You tuned twelve settings by checking the test score each time — the test set has now leaked into your choices.
Your data has a time order and you split it randomly, so the model sees the future.
You are evaluating a language model on a benchmark that was published online — and it read the internet.
That last one is a live, unsolved problem for every LLM benchmark you will read about.
③ Evaluation
Overfitting: the student who memorised past papers
Learning
Extracts the pattern that generalises. Handles a question it has never seen.
Slightly imperfect on the examples it studied.
Overfitting
Memorises the specific examples, including their accidents and noise.
Flawless on the past papers. Helpless on a new one.
The counter-intuitive consequence: a model that is perfect on its training
data is usually a bad model.
③ Evaluation · the number that lies
"95% accurate" can be worthless
Suppose 5 letters in every 100 in your archive are anonymous denunciations, and you build a
detector for them.
My detector says "not a denunciation" every single time.
It is a single line of code. It contains no intelligence whatsoever.
Its accuracy is 95%.
Whenever you meet an accuracy figure, the first question is:
what would a system that does nothing score?
③ Evaluation · two better numbers
Precision and recall, in archive terms
You ask a system to find every letter in the archive that mentions a certain family.
Precision
Of the letters it handed you, how many were actually relevant?
Low precision = you waste weeks reading rubbish.
Recall
Of the relevant letters in the archive, how many did it hand you?
Low recall = you never learn what you missed. Far more dangerous.
You can always trade one for the other. Which one matters more is a
research decision, not a technical one — and it is yours.
③ Evaluation · the four outcomes
Every mistake is one of two kinds
Truly relevant
Truly irrelevant
System says relevant
Correct hit what you wanted
False alarm costs you reading time
System says irrelevant
Miss costs you the finding
Correct rejection silent, invisible, the majority
One table. Ask for it every time somebody shows you a single accuracy number.
Section 2 · the referee's checklist
Six questions for any AI claim you meet
What exactly was the training data, and who is missing from it?
Who assigned the labels, under what definition, and did they agree with each other?
Was the test data genuinely held out — including from your own tuning decisions?
What would a do-nothing baseline score on this same test?
Precision or recall — which one does this application actually need?
On what kind of input does it fail, and did anyone look?
You now know enough to referee an AI paper. Genuinely — that is not flattery.
Section 2 · what to carry forward
Takeaway
Data is a human artefact with human limits. Training is dumb repetition at
vast scale. Evaluation is only honest on data the machine has never seen.
Playbook line 3: "Never accept an accuracy number without the baseline
and the failure cases."
Exercise 1 · 30 minutes · Notebook 1
Teach a machine to sort your texts
You are about to build, train and evaluate a working
machine-learning system — and then break it on purpose.
Lines of code you will write: zero.
How the exercises work — read this once
Colab writes the code. You write the request.
Each cell in the notebook has a grey instruction box above it. That is the prompt.
Click into the empty code cell and press Ctrl+Shift+Enter
(or click Generate) to open Colab's AI.
Paste or retype the prompt. Colab writes the code.
Read what it wrote — even if you don't understand it, look at it — then press ▶.
If it errors, don't panic and don't fix it by hand. Say so to the AI: that is step 6 of every exercise.
Every notebook also has the code already filled in further down,
in case you would rather just run it.
Exercise 1 · the six prompts
Your sequence
Load the file letters.csv into a table.
Show me its shape, the column names,
and 10 random rows.
— see what you actually have
Split the data 80/20 into a training set
and a test set. Train a simple text
classifier on the training set only.
Report its accuracy on the test set.
— the whole of machine learning, in one prompt
What accuracy would I get by always
predicting the most common label?
Print both numbers side by side.
— the baseline. Never skip this.
Show me 8 test examples the model got
wrong, with the true label and the
predicted label.
— where does it fail, and does that make sense?
Which 15 words push the model most
strongly towards each label?
Show them in a bar chart.
— what did it actually learn?
Now retrain using only 20 training
examples instead of the full set.
Report the new test accuracy.
— how much data does this actually need?
Exercise 1 · while it runs
Four things to watch for
Is the accuracy real?
Compare it to the do-nothing baseline. If they're close, your model has learned nothing.
Are the errors sensible?
Read the mistakes. Would you have got them right? Sometimes the label is simply wrong.
Are the top words embarrassing?
If the model keys on a signature, a date format, or a scanning artefact, it found a shortcut, not a concept.
Did 20 examples nearly work?
If so, your task is easier than you thought. If accuracy collapsed, you now know data volume matters.
Exercise 1 · the twist
Now prove that it memorises
Randomly shuffle all the labels in the training set so
they no longer match the texts, then train the same model
again. Report the training accuracy AND the test accuracy.
Training accuracy
Will be high. The model can memorise nonsense.
Test accuracy
Will collapse to the baseline. There was never anything to learn.
This is overfitting made visible in ninety seconds. It is also the single
best demonstration of why held-out evaluation is not optional.
Exercise 1 · debrief
You just did the whole thing: collected data, trained a model,
evaluated honestly, inspected what it learned, and broke it.
Playbook line 4
"Ask the machine to show me what it got wrong, and what features it used." — the two prompts that
turn a black box into something you can actually judge.
3
Neurons, images, sequences
Three architectures, three things they made possible, three things they could not do
First, a warning about a word
"Neural network" is a sixty-year-old metaphor that has long since outlived its accuracy.
These systems were loosely inspired by neurons in the 1950s.
They are not models of the brain, they are not how you think, and no neuroscientist would recognise them.
Keep this in mind whenever you read that an AI system "thinks like a human". It does not.
The atom of the whole field
One "neuron" is a weighted vote
It receives some numbers.
It multiplies each by an importance weight.
It adds them up.
If the total passes a threshold, it fires.
That is the entire mechanism. There is nothing else inside it.
Is this letter angry?
exclamations
× 0.7
"unacceptable"
× 0.9
"kind regards"
× −0.4
Total above threshold → angry
Training is nothing but finding those three numbers — 0.7, 0.9, −0.4 — automatically.
Why one is not enough
Stack them, and something new appears
Inputthe raw text or image
→
Layer 1simple patterns
→
Layer 2combinations of those patterns
→
Layer 3combinations of those
→
Outputa judgement
A single neuron can only draw a straight line through the data. Stacked layers can
carve out any shape at all — which is why complicated things become learnable.
The word "deep"
"Deep learning" means nothing more glamorous than a lot of layers. That is the whole etymology.
Architecture 1
Feed-forward: information goes one way
Good at
Fixed-size inputs: a form, a set of measurements, a table row
"Given these 30 numbers, predict this one"
Being fast, small, and easy to train
Hopeless at
Anything where position matters — an image, where a pixel means nothing alone
Anything where order matters — a sentence, where word sequence is the meaning
Inputs of varying length
Those two blind spots are exactly what the next two architectures were invented to fix.
Architecture 2 · the problem
An image is a million numbers with no meaning
Pixel 400,318 is dark grey. So what?
Meaning in an image lives in local arrangement: this dark pixel next to that light one
makes an edge; these edges together make a curve; these curves make an eye.
And it must work wherever the thing appears. A face in the top-left corner
is still a face. A feed-forward network has to learn "face" separately for every possible position — hopeless.
Architecture 2 · the idea
CNN: slide a small window over everything
Take a small window — say 3×3 pixels — that detects one simple thing, like a vertical edge.
Slide it across the entire image, checking every position.
You now have a map of "where are the vertical edges?"
Do that with hundreds of different windows. Then run more windows over those maps.
Because the same window is reused everywhere, the network learns
"edge" once instead of once per position. That single trick is what made computer vision work.
Layer 4 — whole objects: a face, a column, a signature
Nobody programmed this progression. It appears on its own, from nothing but
examples and the guess–check–adjust loop. When researchers first visualised it, it was a genuine surprise.
Architecture 2 · in your disciplines
What CNNs are actually used for here
Palaeography
Transcribing handwritten manuscripts; dating and attributing hands; grouping scribes across a corpus.
Art history
Finding iconographic motifs across tens of thousands of images that no one could look through by hand.
Archaeology
Spotting structures in aerial and LiDAR survey data; classifying ceramic fragments.
Conservation
Detecting damage, overpainting, and craquelure patterns in high-resolution scans.
In every case the value is the same: triage at a scale a human cannot reach,
followed by human judgement on what it surfaces.
Architecture 2 · where it breaks
It finds shortcuts, not concepts
Wolves and huskies
A famous classifier separated wolves from huskies almost perfectly.
It had learned to detect snow in the background.
Adversarial patches
A small printed sticker, meaningless to you, placed in the corner of an image,
flips the classification with high confidence.
Out of distribution
Trained on clean museum photography, then given a damaged, badly lit archive scan:
accuracy collapses, confidence does not.
Scanner signature
If one collection was digitised on one machine, the model can learn the machine
rather than the content — and score beautifully on your test set.
Live demo · 5 minutes · watch, don't type
Watch one get built, and then fooled
Load the built-in handwritten digits dataset. Show me 10 example
images. Train a small image classifier on 80% of it and report
the accuracy on the remaining 20%. Show me a grid of the images
it got wrong, labelled with what it guessed.
Now add faint random noise to the test images and report the
accuracy again. Show me the confidence of its wrong answers.
Runs in under a minute on Colab's free tier. No GPU, no waiting, no installation.
Architecture 3 · the problem
A sentence is not a bag of words
"The senate condemned the consul."
"The consul condemned the senate."
Identical words. Opposite events. Any system that ignores order cannot tell these apart —
and the classifier you built in Exercise 1 cannot tell these apart.
Go back and try it later. It is a genuinely useful humiliation.
Architecture 3 · the idea
RNN: read one word at a time, and carry a memory
"The"memory: —
→
"senate"memory: subject seen
→
"condemned"memory: subject + action
→
"the consul"memory: who did what to whom
The network reads left to right, and after each word updates a small internal summary
of everything so far. Order now matters, because the summary depends on it.
This is how machine translation, early speech recognition and predictive
keyboards worked for most of the 2010s.
Architecture 3 · where it breaks
The summary is small, and the sentence is long
The letters that the archivist we met last spring in the
reading room of the small provincial library near Bologna had catalogued were…
By the time it reaches were, the network must still remember that the subject was
letters, plural, from twenty-five words ago — through a summary that has been overwritten
twenty-five times.
Everything gets squeezed through one small memory, one word at a time.
Distant information fades. Worse: because it must read strictly in order, it cannot be sped up
by adding more computers.
Two walls: it forgets what is far away, and it cannot be made bigger.
In 2017 one architecture knocked down both at once.
Section 3 · what to carry forward
Takeaway
Each architecture is a bet about the shape of the data: fixed-size (feed-forward),
spatial (CNN), sequential (RNN). And each learns whatever shortcut works — including ones you would
never accept if you could see them.
Playbook line 5: "When a model performs suspiciously well, look for the shortcut before celebrating."
4
Transformers, and the machine that guesses
Everything you will use this afternoon is built out of this section
2017
A paper with an arrogant title and, as it turned out, an accurate one.
"Attention Is All You Need"Vaswani et al., 2017 — the paper that every system in this course descends from
Its proposal: throw away the sequential reading entirely.
Let every word look at every other word, all at once.
The core idea, in one sentence
Every word gets to ask every other word a question
For each word, the model computes: which other words in this text do I need to
look at in order to be understood correctly?
Processing the word "angry":
Thelettershewrotetohersisterwasangry
Strong attention to letter (what is angry?) and to sister (angry at whom?).
Weak attention to the, to, her.
"Attention" is just a set of weights saying how much each word should
influence each other word. Nothing more mystical than that.
Why this is more than a trick
It resolves things a sequence cannot
"The trophy would not fit in the suitcase because it was too big."
"it" = the trophy
"The trophy would not fit in the suitcase because it was too small."
"it" = the suitcase
One word changes, and the referent of it flips. Nothing local tells you this —
you need the whole sentence at once, and you need to know about trophies and suitcases.
Attention makes "look at the whole sentence at once" the default rather than the exception.
The commercially decisive part
All at once means all in parallel
RNN
Word 2 cannot be processed until word 1 is done. Ten thousand words means ten
thousand steps, in order. Adding computers does not help.
Transformer
All words processed simultaneously. Add a thousand computers and it really does
go a thousand times faster.
This is why 2017 mattered commercially, not just intellectually: it made the models
scalable, and once something is scalable, money can be poured into it.
Remember this for the afternoon
Everything about the last eight years of AI follows from the fact that this architecture rewards
spending more money on more computers. That is also the bridge to the HPC session.
Now the part that surprises everyone
The architecture is only half the story. What we train it to do is the other half — and it is almost insultingly simple.
Before words, a detour
The model does not see words. It sees tokens.
Text is chopped into fragments — common words stay whole, rarer ones get split.
"The archivist catalogued an unremarkable manuscript."
The archivist catalogued an unremarkable manuscript.
12 tokens for 6 words. Roughly: 1 token ≈ 4 characters of English.
Why? A fixed vocabulary of ~100,000 fragments can build any word, including
ones never seen before, without needing a dictionary of every word in every language.
Tokens · the consequences are real
Three famous failures, all caused by this one slide
It cannot spell
Ask how many r's are in "strawberry". It often gets it wrong —
it never saw letters, only the chunks straw and berry.
Some languages cost more
Italian, and far more so Greek, Arabic or Chinese, need more tokens per word than
English. Same text: more expensive, and more of your context window consumed.
It stumbles on rare names
An unusual surname or place shatters into many fragments, and the model
handles fragments it has barely seen much less reliably.
If your research involves non-English sources, pre-modern orthography, or
unusual proper nouns — that is your material, and this is your warning.
From tokens to meaning
Each token becomes a point in a space of meaning
Every token is turned into a long list of numbers — its coordinates. The model learns
these coordinates so that tokens used in similar contexts end up near each other.
Nearby
letter · epistle · correspondence · note
Far away
letter · basalt
Nobody wrote a thesaurus. Proximity emerged from the statistics of usage alone —
a computational version of "you shall know a word by the company it keeps".
Why this matters more than it looks
Meaning-space has directions
Paris − France + Italy ≈ Rome
The direction from a country to
its capital turns out to be a consistent direction in the space — not because anyone put it there.
This is the property that makes search by meaning possible: you can find
documents that are about the same thing without sharing a single word.
Remember this at 15:20. It is how your research assistant will find the right page.
Now, the training objective
Take a piece of text. Hide the last token. Guess it.
Repeat approximately a trillion times.
The training objective, concretely
What "guess the next token" looks like
Input: "The archivist opened the wooden"
box
31%
door
24%
chest
18%
cabinet
11%
crocodile
0.001%
The output is never a word. It is always a probability for every one of the
~100,000 tokens. One is then picked, appended to the text, and the whole thing runs again.
Say it plainly
That is the whole trick. There is no other mechanism.
Everything else — answering questions, writing code, translating, summarising,
reasoning step by step — is this one operation, repeated, on text that was arranged to make the
useful continuation the likely one.
The obvious objection
"But it answers my questions!"
Consider what the most likely continuation of this text is:
Q: What was the capital of the Kingdom of Italy in 1861? A:
Across everything ever written, the fragment most likely to follow "A:" in that context
is the correct answer. Answering correctly and predicting text well are the same behaviour —
as long as the correct answer really was the common continuation in the training data.
And when it wasn't?
Then the model produces the plausible continuation instead of the true one. Hold that thought
for four slides.
The dice
Temperature: how adventurous the picking is
Low · 0.0–0.3
Almost always takes the most likely token. Repetitive, predictable, stable.
Use for extraction, classification, structured output, anything you will check.
Medium · 0.7
The usual default. Natural-sounding, varied.
Use for drafting, explaining, conversation.
High · 1.2+
Frequently picks unlikely tokens. Creative, surprising, and increasingly unhinged.
Use for brainstorming. Never for facts.
This is why the same question can give you different answers.
It is a dice roll, and by default the dice are loaded but not fixed.
Where the knowledge comes from
Stage 1 · Pre-training: read almost everything
Web pages, books, code repositories, scientific papers, forums, subtitles, digitised archives.
Trillions of tokens.
Months of computation on tens of thousands of specialised processors.
Tens of millions of euros for a frontier model.
For scale
A person reading continuously, day and night,
at 250 words a minute, would need on the order of tens of thousands of years
to get through the training set of a modern model.
The result of this stage is a machine that continues text superbly —
and does not answer questions, follow instructions, or refuse anything.
Where the helpfulness comes from
Stages 2 and 3 · Being taught to behave
① Pre-trainingRead everything. Learn language and, incidentally, facts.
→
② Instruction tuningFine-tune on examples of instructions followed well.
→
③ Preference tuningHumans rank pairs of answers; the model is nudged toward the preferred ones.
Stage 1 gives it capability. Stages 2 and 3 give it manners — and the manners are
a product decision, made by a company, encoding somebody's judgement about what a good answer is.
Worth stating clearly
When a model refuses, hedges, flatters you, or adopts a particular tone, that is not the architecture.
That is stage 3, and it is a choice somebody made.
The model's desk
The context window is everything it can see
Before generating each token, the model looks at one thing only: the text currently in
front of it. That text is the context window, and it has a fixed maximum size.
8ktokens ≈ 15 pages (models of 2022)
200ktokens ≈ 400 pages (common today)
1M+tokens ≈ 2,000 pages (largest available)
Outside that window, nothing exists. Not your earlier conversation,
not your documents, not this morning.
Context · what "it forgot" really means
It did not forget. It never had it.
Long conversations. When the transcript exceeds the window, the earliest parts are dropped
or summarised. It is not forgetfulness; the text is simply gone.
Lost in the middle. Even inside the window, material in the middle of a very long text
is used less reliably than material at the start or end. This is measurable and it is not fixed.
Between sessions. A new chat starts empty. Any apparent memory is a product feature that
silently pastes saved notes back onto the desk.
Practical consequence
Do not bury your most important instruction in the middle of a five-page prompt. Put it at the
beginning, and repeat it at the end.
The one you came for
Hallucination
Not a bug. Not a glitch. Not something the next version fixes.
A direct consequence of everything on the last ten slides.
Hallucination · the mechanism
Why it is inevitable
There is no database inside. The model is a wall of dials, not a store of facts.
Its only operation is producing a plausible continuation of the text so far.
A plausible-looking citation and a real citation look identical to that machinery —
same shape, same style, same rhythm.
So when it doesn't have the fact, it does not fail. It succeeds at its actual task
and produces something that looks exactly right.
The system is working perfectly. Your expectation of it was the thing that was wrong.
Hallucination · why it sounds so sure
Fluency is not knowledge, and confidence is a style
In a human
Fluent, specific, confident prose is weak evidence of expertise. We are trained
from childhood to read it that way.
In a model
Fluency is the only thing it optimises. It is exactly as fluent when
inventing as when recalling. The signal you rely on has been severed.
Worse: stage 3 trained it to sound helpful and assured, because human raters
preferred that. Hedging was rated lower and trained out.
Hallucination · the case that will hurt you
Fabricated citations
Ask for sources on a niche topic and you will get: a plausible
author who works in that field, a plausible co-author, a plausible title, a real journal,
a plausible year, plausible page numbers, and a well-formed DOI.
And the paper does not exist.
Every element is individually likely. The combination is fiction. This has ended careers
and produced court sanctions, and it is the single largest risk of these tools in academic work.
Non-negotiable rule
Never cite anything an AI gave you that you have not personally opened. Not the abstract — the paper.
So what is it actually for?
The distinction that makes all of this usable
Weak · asking it to recall
"What did Gramsci say about hegemony?"
"Give me five sources on this topic."
"When was this manuscript catalogued?"
You are querying a lossy compression of the internet, and you cannot see
where it is lossy.
Strong · asking it to transform
"Here is the passage. Summarise the argument in three claims."
"Here are my notes. Turn them into a table with these columns."
"Here is my paragraph. What would a hostile reviewer attack?"
You supplied the material. You can check the output against it in seconds.
Give it the material. Ask it to transform, not to remember.
That single habit removes most of the risk and keeps almost all of the value.
Exercise 3 · 20 minutes · Notebook 3
Look inside the language model
Tokens, probabilities, temperature, and a hallucination
you will produce deliberately.
Exercise 3 · the sequence
Five prompts
Split this sentence into tokens and print
each one with its number. Then do the same
for the Italian translation, and compare
how many tokens each one needs.
— feel the tokenizer, and the language tax
Take the phrase "The archivist opened the
wooden" and show me the model's top 10
predictions for the next token, with their
probabilities, as a bar chart.
— see the probability distribution directly
Generate a continuation of that phrase 5
times at temperature 0.2, and 5 times at
temperature 1.5. Print them side by side.
— watch the dice
Count how many times the letter "r" appears
in "strawberry", first by asking the model,
then by counting it directly in code.
Compare the two answers.
— the tokenizer's revenge
Give me five peer-reviewed articles about
[an extremely narrow topic in your own
field], with authors, journals, years and
DOIs.
— then check every single one
The last prompt is the exercise. Take the citations it gives you and search for
every one. Write down how many were real. Keep that number — it is the most useful
thing you will learn today.
Exercise 3 · the debrief question
Now ask it whether it was sure
For each of the five citations you just gave me, tell me how
confident you are that it exists, and mark any you may have
constructed rather than recalled.
Sometimes this works remarkably well. Sometimes it confidently defends fiction.
The fact that you cannot tell which in advance is the entire point.
Worth a moment's thought: if you had been tired, and in a hurry, and the
citation had looked right — would you have checked?
Playbook line 7
Asking a model to check itself is worth doing and is not verification. Only a source you
opened yourself is verification.
Morning close
What you now know
Three hours ago most of this was a black box
The morning in one picture
From examples to a machine that writes
Examplesinstead of rules
→
Dialsturned by guess–check–adjust
→
Layerssimple patterns combined into complex ones
→
Attentionevery token sees every other, in parallel
→
Next tokena probability over 100,000 fragments
→
Repeatand text appears
Every system you will use this afternoon, and every one you will read
about this year, is somewhere on this line.
Vocabulary you now own
Ten words that unlock the literature
label — the human-supplied answertraining — guess, check, adjustparameter — one dialtest set — data hidden from the modeloverfitting — memorising instead of learningbaseline — what doing nothing scorestoken — the fragment it actually seesembedding — a token's coordinates in meaning-spaceattention — which tokens look at whichcontext window — the desk it can see
You can now read the methods section of an AI paper, and — more usefully —
interrogate anyone who tries to sell you one.
Your playbook so far
Seven lines, earned this morning
Before reaching for AI, ask whether a rule would do.
Ask whose judgement is encoded in the system's answers.
Never accept an accuracy number without the baseline and the failure cases.
Ask the machine to show what it got wrong and which features it used.
When a model performs suspiciously well, look for the shortcut before celebrating.
Give it the material. Ask it to transform, not to remember.
Self-checking is worth a prompt and is never verification.
Page 1 of your playbook is now full. The afternoon fills pages 2 and 3.
One honest slide before lunch
What I have not told you
Nobody, including the people who build them, can explain why a specific output appeared.
Interpretability is an open research problem, not a solved engineering detail.
Whether these systems "understand" anything is genuinely contested among serious people.
I have avoided the question all morning because you do not need it to use them well.
The training data of every major commercial model is undisclosed. You cannot audit what you
cannot see, and that includes whether your own work is in there.
The environmental and labour costs are real, unevenly distributed, and largely externalised.
You should know the shape of what is unknown. That is also scholarship.
Coming back at 14:00
This morning: what the machine is.
This afternoon: what you can make it do.
