From Base Model to Assistant: What Post-Training Actually Changes

Why this matters

A freshly pretrained base model is a very good text completer and nothing more: give it 'The capital of France is' and it keeps going in whatever style the internet taught it, including finishing a question with more questions instead of an answer. Post-training is the set of stages — supervised fine-tuning, reward modeling, and reinforcement learning — that reshapes that raw completer into something that follows instructions, stays in a conversational voice, and refuses what it shouldn't answer. What makes this surprising the first time you see it is how little the weights move relative to pretraining: labs report post-training on a small fraction of the pretraining token count, yet the behavioral shift is enormous. Understanding that post-training is a behavior change bolted onto a mostly-frozen capability base is the single idea that makes every later module in this course make sense, because it explains why you can break behavior badly with a tiny amount of bad post-training data, and why capability and alignment can be improved almost independently.

Demo

Below is a minimal illustration of the completion-vs-instruction-following gap using a small open base model and its instruction-tuned counterpart from the same family, so you can see the shift with your own eyes instead of taking it on faith.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

BASE = "Qwen/Qwen2.5-0.5B"          # raw pretrained completer
INSTRUCT = "Qwen/Qwen2.5-0.5B-Instruct"  # same family, post-trained

def complete(model_name, prompt, max_new_tokens=40):
    tok = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float32)
    ids = tok(prompt, return_tensors="pt")
    out = model.generate(**ids, max_new_tokens=max_new_tokens, do_sample=False)
    return tok.decode(out[0], skip_special_tokens=True)

prompt = "What is the capital of France?"
print("BASE:     ", complete(BASE, prompt))
print("INSTRUCT: ", complete(INSTRUCT, prompt))

# The base model often keeps pattern-matching ("... France? What is the
# capital of Germany? ...") because it learned "this is what text like this
# continues with," not "answer the question." The instruct model was
# post-trained to treat the prompt as a request and produce a direct answer.

Try it yourself

1. Run the demo with a factual question, then rerun with an open-ended one ('Write a short poem about rain') and compare how much the base-vs-instruct gap changes.
2. Wrap the base model's prompt in the instruct model's chat template (system + user turns) before generating, and check whether that alone narrows the gap.
3. Count the parameters that differ between the two checkpoints using a state-dict diff on a couple of layers — confirm the architecture is identical and only the weights differ.
4. Try a prompt the instruct model should refuse (something mildly unsafe) and compare how each model responds.

References

• Ouyang et al. — Training language models to follow human feedback (InstructGPT)
• Meta — Llama 3 model card (see the post-training section)