Pick a depth. Each prompt opens in your AI pre-loaded with the lesson. Click a row to preview the prompt.
Before you write a single line of ranking code you have to answer an uncomfortable question: what is this system actually for? 'Maximize clicks' is measurable in an hour and is almost never what the business wants — a feed optimized purely for clicks converges on outrage and clickbait, because those generate clicks people regret five minutes later. The real objective is usually something slower and harder to measure: long-term engagement, subscription retention, purchase satisfaction, or revenue net of returns. Every metric you can compute quickly — CTR, watch-start rate, add-to-cart rate — is a proxy standing in for that real objective, and proxies drift from the truth exactly where they're easiest to game. Naming your true objective and your proxy explicitly, and being honest about the gap between them, is what stops a recommender from technically succeeding at the wrong thing.
The demo simulates two ranking policies — one that greedily maximizes a click-probability proxy, one that maximizes a (noisier, delayed) satisfaction signal — and shows how optimizing the easy proxy alone can produce a worse outcome on the metric that actually matters.
regret field to clickbait items (probability the user reports feeling manipulated) and print an expected-regret total per policy.p_click * satisfaction ** 2 and see how squaring satisfaction shifts weight further toward solid articles.import random
random.seed(1)
# Each item has a click probability (easy to measure) and a satisfaction
# score conditional on being clicked (only observable later, e.g. next-day
# retention). Clickbait items have high click prob but low satisfaction.
ITEMS = [
{"id": "clickbait_1", "p_click": 0.35, "satisfaction": 0.10},
{"id": "clickbait_2", "p_click": 0.30, "satisfaction": 0.15},
{"id": "solid_article_1", "p_click": 0.12, "satisfaction": 0.80},
{"id": "solid_article_2", "p_click": 0.10, "satisfaction": 0.85},
{"id": "solid_article_3", "p_click": 0.09, "satisfaction": 0.75},
]
def rank_by(items, key):
return sorted(items, key=key, reverse=True)
def simulate_session(ranked, n_users=10_000):
total_clicks = 0.0
total_satisfaction = 0.0
for item in ranked[:3]: # top-3 shown
expected_clicks = item["p_click"] * n_users
total_clicks += expected_clicks
total_satisfaction += expected_clicks * item["satisfaction"]
return total_clicks, total_satisfaction / max(total_clicks, 1)
click_ranked = rank_by(ITEMS, key=lambda i: i["p_click"])
satisfaction_ranked = rank_by(ITEMS, key=lambda i: i["p_click"] * i["satisfaction"])
for name, ranked in [("click-maximizing", click_ranked), ("value-weighted", satisfaction_ranked)]:
clicks, avg_sat = simulate_session(ranked)
print(f"{name:>17}: top-3 = {[i['id'] for i in ranked[:3]]}")
print(f"{'':>17} clicks={clicks:,.0f} avg_satisfaction={avg_sat:.2f}")python3 main.py