LLMs: Algorithms that break the rules
Code should be correct, clear and efficient.
LLMs are none of these, yet very useful.
A large language model (LLM) is the algorithm behind ChatGPT, Claude, Lumo and similar AI assistants. It can be defined in the same way we define any other algorithm.
A large language model is a stepwise procedure of well-defined executable instructions (including billions of learned numeric parameters) that given a sequence of symbols as input, outputs a probability distribution over possible continuations.
When an LLM is used, the displayed answer is generated one symbol at a time, typically by sampling from this distribution.
That’s it. The algorithm is statistical prediction: given text so far, guess likely text to come. Presented with typical user input, it answers questions, writes programs and apparently reasons.
This is behavior that emerged from training on enormous amounts of text. Very little of this ultimate behavior follows transparently from the training procedure.
Algorithms of this kind come with an unusual set of properties:
- Strengths. No classical algorithm can draft text, summarize a messy document or translate between arbitrary languages. LLMs handle them surprisingly well. They also adapt to new problems from examples in the prompt alone.
- Limitations. They state falsehoods with the same confident tone as facts (hallucination), answers are not reliably reproducible and performance cannot be guaranteed. The model may fail spectacularly on inputs that appear easier than ones it aced.
Now, let’s hold the model against the motto of this site:
Code should be correct, clear and efficient.
Correct?
An algorithm is correct if it stops for each problem instance and returns the required output. Classical algorithms come with proofs: an algorithm such as Quicksort guarantees the result for every input of every length.
An LLM comes with no such guarantee. Worse, the failure mode is invisible: a hallucinated fact and a remembered fact can arrive in identically fluent prose. Correctness, if it exists at all, is statistical: often true, never provable.
Clear?
Good code can be read and audited by a human. A trained LLM is billions of numbers with no human-readable explanation of why any particular answer took the shape it did. We cannot inspect the code, because there is effectively no code – only a compressed statistical residue of the training text, of which we may not even have a trustworthy record.
Efficient?
Compare the cost of the two algorithms for the same task. Sorting a million numbers with Quicksort runs in seconds on a laptop. Training an LLM consumed megawatts over months. As a sorting algorithm, this would be a joke.
The efficiency question is genuine, just different: the only fair comparison is with no executable algorithm at all – with leaving the task undone, or doing it by hand. Against that baseline, the expense may well be worth paying.
What to do
“Correct” is quietly doing two jobs in the motto: provable correctness, and the looser notion of reliability. Classical algorithms can be proven correct. For an LLM, the best available deal is calibrated use:
- Verify. Treat every factual claim as a hypothesis.
- Supply context. The model can only predict from what is in the prompt; it cannot guess your situation.
- Keep control. Use it for drafts and suggestions, not for decisions.
Further reading
The free Swedish book Tänkarens guide till AI features one such algorithm describing itself.