In the previous chapter, we established that a language model learns patterns from text rather than following hand-written rules. But that raises an obvious question: how does the learning actually happen? Where does the data come from? What does the training process look like? And why does it cost hundreds of millions — sometimes billions — of dollars?

This chapter pulls back the curtain on the training process. You do not need a computer science degree to follow along. If you can understand how a student learns by studying textbooks, you can understand how an AI model learns by studying data.

The Training Data: What AI Reads

Before a language model can learn anything, it needs material to study. For the largest models, that material is a staggering amount of text.

The training data typically includes large portions of the public internet — web pages, articles, forum discussions, and blog posts. It includes digitized books spanning centuries of human knowledge. It includes open-source code from platforms like GitHub. It includes Wikipedia, academic papers, legal documents, recipes, song lyrics, instruction manuals, and much more.

To give you a sense of the scale: the training data for a modern frontier model might contain trillions of words. If you tried to read it yourself at a pace of one word per second, eight hours a day, it would take you tens of thousands of years to get through it all.

This breadth of data is what gives language models their remarkable range. The reason a model can discuss medieval history, debug a Python script, and explain a cookie recipe is that it has seen text about all of these topics and millions more during training.

The Data Quality Problem

Not all data is created equal, and AI researchers have learned this the hard way. Early models trained on unfiltered internet text picked up all kinds of problems — toxic language, factual errors, spam, and biases baked into the text they studied.

Think of it this way: if a student only studied conspiracy theory websites, they would come away with a very distorted view of the world. Similarly, if a language model trains on low-quality data, it produces low-quality outputs.

Modern training pipelines invest enormous effort in data curation. Teams of researchers and engineers filter out spam, remove duplicate content, balance the mix of topics, and try to ensure that the training data represents a broad and reasonably accurate view of human knowledge. This curation process is one of the most important and least glamorous parts of building an AI model.

The quality of training data has become such a competitive advantage that most AI companies treat the details of their data pipelines as closely guarded secrets. What we do know is that data quality often matters more than data quantity. A model trained on a carefully curated dataset can outperform one trained on a larger but messier collection.

Pre-Training: The Foundation

The training process for a modern language model happens in stages, and the first and most expensive stage is called pre-training.

During pre-training, the model is shown massive amounts of text and learns to predict the next word (or, more precisely, the next token — we will cover tokens in detail in the next chapter). It starts with random guesses and gradually improves, adjusting millions or billions of internal settings called parameters.

Here is an analogy. Imagine you are learning a new language by reading thousands of books in that language. At first, nothing makes sense. But gradually, you start recognizing common words. Then you notice grammatical patterns. Then you start understanding sentence structure. Eventually, you can read a new sentence and have a good sense of what word is likely to come next, because you have internalized the patterns of the language.

Pre-training works similarly, except it happens across dozens of languages simultaneously, across every domain of human knowledge, and at a scale that no human could ever achieve.

How the Learning Actually Works

The model starts with randomly initialized parameters — essentially, a massive collection of numbers that mean nothing. It reads a chunk of text, tries to predict the next word, and then checks whether it was right. If it was wrong, it adjusts its parameters slightly in a direction that would have made the correct prediction more likely. Then it moves on to the next chunk and does it again.

This process is called gradient descent, but you do not need to remember that term. The key idea is simple: the model makes a prediction, checks the answer, and nudges its settings to do better next time. Repeat this billions of times across trillions of words, and the model's parameters settle into a configuration that captures an extraordinarily detailed map of how language works.

What is remarkable is that this simple process — predict, check, adjust, repeat — produces a system that appears to understand grammar, facts, logic, humor, tone, and countless other aspects of language and knowledge. Nobody programs these capabilities in. They emerge from the training process.

Fine-Tuning: Making the Model Useful

Pre-training gives the model a broad understanding of language and knowledge, but the result is not yet a useful assistant. A pre-trained model is like a brilliant student who has read every book in the library but has never had a conversation with another person. It can complete text beautifully, but it does not know how to follow instructions, answer questions helpfully, or have a back-and-forth dialogue.

This is where fine-tuning comes in. After pre-training, the model is trained further on a smaller, more carefully curated dataset of examples that demonstrate the kind of behavior you want.

For a chatbot, this means training on examples of helpful, clear, honest conversations. The model sees thousands of examples where a human asks a question and a knowledgeable assistant provides a good answer. It learns to mimic that pattern — to take on the role of a helpful assistant rather than just predicting the next word in whatever text it is given.

Fine-tuning is dramatically cheaper than pre-training because it uses much less data and computation. But it is where the model's personality, helpfulness, and safety characteristics are shaped.

RLHF: Teaching AI to Be Helpful

One of the most important innovations in modern AI is a technique called Reinforcement Learning from Human Feedback, or RLHF. It sounds complicated, but the core idea is intuitive.

After the model has been fine-tuned, humans evaluate its responses. They might be shown two different answers to the same question and asked which one is better — more helpful, more accurate, more clearly written, safer. These human preferences are collected and used to train the model further.

Think of it like training a new employee. You do not just give them a manual (that was pre-training and fine-tuning). You also give them feedback on their actual work. "This report was great." "This email was too formal for the situation." "You answered the customer's question but missed the real concern behind it." Over time, the employee learns to produce work that meets your standards.

RLHF is what transforms a language model from something that generates plausible-sounding text into something that generates genuinely helpful, safe, and appropriate text. It is also how companies align their models with human values — teaching them to refuse harmful requests, acknowledge uncertainty, and be honest about their limitations.

This alignment process is an area of active research and debate. Different companies take different approaches, and the question of exactly what values an AI should be aligned with is as much a philosophical question as a technical one.

The Hardware: Why Training Requires Supercomputers

Training a frontier language model is one of the most computationally intensive tasks humans have ever undertaken. It requires thousands of specialized processors running continuously for weeks or months.

The processors used for AI training are typically GPUs (Graphics Processing Units) or TPUs (Tensor Processing Units). GPUs were originally designed for rendering video game graphics, but it turns out that the kind of math they do — performing many simple calculations simultaneously — is exactly what neural network training requires. TPUs are custom chips designed by Google specifically for AI workloads.

A single high-end AI GPU costs tens of thousands of dollars. Training a frontier model requires clusters of tens of thousands of these chips, connected by specialized high-speed networking, housed in massive data centers with enormous cooling systems.

The Cost of Training

The numbers are staggering. Training a frontier language model in 2025 costs somewhere between $100 million and $1 billion or more, depending on the model's size and how much experimentation was involved. This includes the cost of the hardware, the electricity to run it (a single training run can consume as much electricity as a small city uses in a month), and the salaries of the highly specialized researchers and engineers who design and oversee the process.

These costs have been roughly doubling every year as companies push for bigger and more capable models. This raises serious questions about who can afford to build frontier AI — increasingly, only the largest and best-funded companies and labs in the world.

The expense also means that training is not something you do casually. Companies plan training runs months in advance, carefully design their data mixtures and model architectures, and run extensive smaller-scale experiments before committing to a full-size training run. A failed training run can waste tens of millions of dollars.

The People Behind the Models

It is easy to focus on the data and the hardware and forget that training an AI model is fundamentally a human endeavor. Behind every frontier model is a team of hundreds or thousands of people.

Researchers design the model architecture and training methodology. Engineers build and maintain the massive computing infrastructure. Data specialists curate and clean the training data. Human evaluators provide the feedback used in RLHF. Red teamers try to find and fix the model's failure modes. Legal teams navigate the complex questions around training data and intellectual property. Product teams figure out how to make the model useful for actual people.

The talent required for this work is extremely specialized, and the competition for AI researchers and engineers has driven salaries to extraordinary levels. Top AI researchers can command compensation packages worth millions of dollars per year, which has created a significant brain drain from academia. Universities are losing their best computer science professors and researchers to industry, which raises concerns about the long-term health of academic AI research.

What the Model Actually Learns

After all this training — the massive datasets, the billions of dollars of compute, the months of processing — what exactly has the model learned?

This is one of the most fascinating and unsettled questions in AI research. The model's knowledge is stored in its parameters — billions of numbers that collectively encode everything it learned during training. But these numbers are not organized like a database or an encyclopedia. There is no single parameter that stores the capital of France, no module dedicated to understanding grammar.

Instead, the knowledge is distributed across the entire network in a way that researchers are still working to fully understand. We can probe the model and observe that it "knows" things — it can answer questions, make logical inferences, and demonstrate knowledge of the world. But exactly how this knowledge is represented internally is an active area of research called mechanistic interpretability.

What we can say is that the model has built an internal representation of language and knowledge that is far more sophisticated than simple memorization. It can combine facts it learned separately, apply concepts to new situations it was never trained on, and generate text that, while informed by its training data, was never explicitly present in that data.

The Training Pipeline in Summary

Let us bring it all together with a simplified view of how a model goes from nothing to the AI assistant you interact with:

1. Data collection and curation: Assemble trillions of words of text, carefully filtered for quality and balance.

2. Pre-training: Train the model on this data to predict the next word. This takes weeks or months on thousands of specialized processors and costs hundreds of millions of dollars. The result is a model with broad knowledge and language ability but no particular helpfulness.

3. Fine-tuning: Train the model further on curated examples of helpful, high-quality conversations and task completion. This shapes the model into an assistant rather than just a text predictor.

4. RLHF and alignment: Use human feedback to further refine the model's behavior, making it more helpful, honest, and safe. This is where the model learns to be the kind of AI you would actually want to interact with.

5. Evaluation and testing: Extensively test the model across thousands of benchmarks and real-world scenarios, with red teams trying to find problems before users do.

6. Deployment: Release the model behind an API or in a product, with ongoing monitoring and updates.

Each of these steps involves difficult technical decisions, tradeoffs, and hard-won expertise. The fact that the end result is a system you can chat with casually on your phone belies the enormous effort behind it.

Why This Matters to You

Understanding how AI models are trained helps you make sense of the headlines. When you read that a company raised $6 billion for AI, now you know where that money is going. When you hear debates about AI training data and copyright, you understand what is at stake. When someone claims that bigger models will keep getting better, you can appreciate both why that has been true and why it might not continue forever.

Training is not a one-time event, either. AI companies are continuously working on the next generation of models, and the decisions they make about data, scale, and alignment will shape how AI affects your life in the years to come.

See This in the News

The enormous cost of training AI models has real consequences beyond the tech industry. The salaries required to attract top AI talent are creating a brain drain from universities and research institutions, threatening the academic science ecosystem that produced many of AI's foundational breakthroughs:

AI Salaries Are Draining Academic Science