A modern LLM is built in three stages. Understanding the stages is the cleanest way to understand what the model is and isn't.
Stage 1: Pre-training
Take a transformer with billions of parameters and train it to predict the next token on a large fraction of the public internet — Common Crawl, Wikipedia, Reddit, books, code from GitHub. This stage is where 99% of the compute goes. A frontier model in 2025 might use $50M-$200M of GPU time. The output is a model that completes text plausibly but doesn't yet 'follow instructions'.
Stage 2: Supervised fine-tuning (SFT)
Take the pre-trained base model and fine-tune it on a much smaller dataset of high-quality demonstrations: prompts paired with the kind of responses a helpful assistant would give. After SFT, the model can follow instructions. Quality of this data matters more than quantity.
Stage 3: RLHF (or DPO)
Reinforcement learning from human feedback. Humans rank multiple model outputs to the same prompt. A separate 'reward model' is trained on these rankings. The main model is then fine-tuned to maximise the reward model's score. This is what makes ChatGPT pleasant to talk to. DPO (Direct Preference Optimisation) is a 2023 replacement that achieves similar results more directly.
Scaling laws
Kaplan et al. (2020) at OpenAI demonstrated that loss falls predictably as you scale compute, model size, and data. Hoffmann et al. (2022) at DeepMind refined the relationship in the Chinchilla paper, showing that earlier models had been under-trained for their size. The scaling laws turned LLM training from an art into engineering: pick a compute budget, look up the optimal parameter count and data size, train, get a predictable model.
Why hallucinations happen
The model is trained to predict the most likely next token given the context. It has no separate mechanism for tracking truth, no internal database to consult, no way to refuse when it doesn't know. When asked a question it doesn't have a confident answer to, it produces the most plausible-looking continuation. That continuation is often wrong, and the model has no way to tell.
The structural cost of next-token prediction
Hallucination isn't a bug in any specific model. It's a structural feature of training a model to predict tokens that read well, not tokens that are true. Mitigations (RAG, verification, constitutional AI, training on 'I don't know') reduce hallucinations but cannot eliminate them. Build verification into any production deployment.
Why Claude, GPT-4, Gemini behave differently
The base architectures are similar. The differences come from: training data mix, post-training data (RLHF preferences), Constitutional AI (Anthropic's alternative to standard RLHF), reasoning training (OpenAI's o-series, Anthropic's extended thinking), and safety policies. Two frontier models can have identical architecture and very different personalities and capabilities.
Exercise
An analyst asks an LLM 'what was Safaricom's reported FY2025 EBITDA?' The model confidently answers 'KES 165 billion' — but the actual figure (which you can verify on Safaricom's investor-relations page) is different. (1) Mechanistically, why did the model do this rather than refusing? (2) Even if the answer happened to be correct, why is treating an LLM as a primary-source database structurally unsafe for analyst work? (3) What is the right deployment pattern if you want LLM help on factual financial questions?