Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
The single most common mistake newcomers make is assuming more post-training machinery is always better, when in reality most real projects should stop before full RLHF. If your task is narrow and you have good demonstrations, SFT alone (often with LoRA) gets you most of the value at a fraction of the cost and risk. If you need the model to prefer one style or safety posture over another and you can collect or synthesize pairwise preferences, DPO gives you most of RLHF's benefit without a reward model or rollout loop. Full PPO-based RLHF earns its cost when you need a reward signal that generalizes far beyond your labeled preference pairs, or when the reward is genuinely non-static (like RLVR's rule-based rewards in Module 6, which need on-policy sampling to work well). Building the decision tree explicitly — instead of defaulting to whatever's trendy in a paper — is the actual professional skill this module is building toward, and you'll use it again for your capstone.
The decision function below turns the tradeoffs from this whole module into a single callable so you can plug in your project's specifics and get a recommendation you can then argue with.
has_verifier check for code correctness specifically, and route it to RLVR even when needs_verifiable_reward wasn't explicitly set.def choose_post_training_path(has_demonstrations, has_preference_pairs,
needs_verifiable_reward, compute_budget):
if needs_verifiable_reward:
return "RLVR / GRPO -- your reward is a rule or verifier, not a learned model."
if not has_demonstrations:
return "Start with SFT -- you need a policy that produces the right SHAPE of answer first."
if not has_preference_pairs:
return "SFT only (consider LoRA) -- no ranking signal to optimize toward yet."
if compute_budget == "low":
return "DPO -- most of the preference-optimization benefit, no reward model or rollouts."
return "Full RLHF (reward model + PPO) -- justified only if DPO's ceiling has been tested and found insufficient."
print(choose_post_training_path(True, False, False, "low"))
print(choose_post_training_path(True, True, False, "low"))
print(choose_post_training_path(True, True, False, "high"))
print(choose_post_training_path(True, True, True, "high"))python3 main.py