Skill rating jumps 12% after a win — losses teach nothing

A win lifts your visible skill rating by about 12%. A loss barely moves it. That's the pattern plenty of Australian product teams have noticed in their own dashboards, and it raises an awkward question: if the number only rewards wins, what exactly is it measuring — skill, or the appearance of it? This isn't a quirk of one platform. It falls out of how rating systems, feedback loops and human motivation interact. Worth pulling apart, because the design choices behind that number shape behaviour in ways most of us never audit.

The maths only knows outcomes

Most skill ratings — Elo, Glicko, TrueSkill, the ladder systems in competitive games, the "proficiency" scores in corporate learning platforms — share a core assumption: your rating moves when you win, stalls when you lose. Elo's original chess formulation nudges ratings after every game, but the psychology of how people read those numbers is lopsided. A win reads as evidence of skill. A loss reads as bad luck, a laggy connection, an unfair matchup.

There's a name for the underlying bias: self-serving attribution. We credit ourselves for successes and blame circumstance for failures. The rating system doesn't cause that, but it amplifies it, because the number itself behaves the same way. A jump after a win feels earned. The flat line after a loss feels like the system failing to notice your improvement.

If you build anything with a skill score attached — a sales leaderboard, a code review rating, a customer support quality metric — you're inheriting this asymmetry whether you intended to or not.

Variable rewards and the pull of the next win

B.F. Skinner's work on reinforcement schedules is the obvious reference here, and it's obvious because it keeps being right. Variable-ratio reinforcement — rewards delivered after an unpredictable number of actions — produces the most persistent behaviour of any schedule tested. Not the highest volume of reward, the most persistent behaviour.

A skill rating that jumps after a win is a variable reward. You don't know which win will produce the jump. Some wins are worth 3 points, some 15, depending on the opponent's rating, the match context, whatever the algorithm weighs. That unpredictability is what makes the next match feel urgent.

The loss side is where it gets interesting. In a well-designed system, a loss should teach something — the opponent was stronger, your approach had a gap, the matchup exposed a weakness. But if the rating barely moves, the feedback is muted. The system has, perhaps unintentionally, told you the loss doesn't count. So you don't examine it. You queue again.

Natasha Dow Schüll's work on machine design and "the machine zone" is instructive here, though she was studying a very different domain. Her observation was that the design of feedback — the pace, the clarity, the near-miss quality — shapes attention more than the content of the feedback does. A rating that jumps on wins and sits still on losses is a feedback design choice. It happens to produce engagement. Whether it produces skill is a separate question.

Loss aversion cuts both ways

Kahneman and Tversky's loss aversion research is usually invoked to explain why people hate losing. Fair enough — losses feel roughly twice as painful as equivalent gains feel good. But there's a second-order effect that matters more for skill development.

If a loss barely dents your rating, the pain is reduced. That sounds humane. In practice, it removes the signal that would prompt reflection. The rating system has effectively said: this event doesn't matter enough to record. Meanwhile the win — the same event in reverse — is recorded in bold.

Consider a concrete case. In 2019, a group of researchers studying chess improvement (not a game-of-chance context, a pure skill domain) found that players who reviewed their losses with an engine improved measurably faster than players who reviewed their wins. The gains came from the losses. The wins were mostly confirmation. The rating system in chess, notably, does move after losses — Elo is symmetric by design. Yet players still gravitate to studying wins, because the emotional asymmetry is stronger than the numerical one.

Now imagine a system where the numbers are asymmetric too. The pull toward studying wins becomes almost total.

What a 12% jump actually rewards

If wins produce a visible jump and losses produce nothing, the rating is measuring something like "recent win rate weighted by opponent strength" rather than skill. Those are correlated, but not the same. A player who wins 60% of matches against weak opponents can out-rate a player who wins 45% against strong ones, depending on the algorithm's K-factor and the pool composition.

For businesses building skill-tracking into products — onboarding scores, certification levels, performance dashboards — the design question is sharp: are you measuring capability, or are you measuring the behaviour your reward structure incentivised? They diverge whenever the reward is asymmetric.

Some teams have started weighting losses. A few competitive platforms now show "what you lost" alongside "what you gained" — a small UX change that costs nothing but forces the loss into view. Others have moved to rolling-window ratings that decay slowly, so a bad week doesn't vanish but also doesn't crater your score. The common thread: make the loss legible, not punishing.

Designing for the loss side

The practical move isn't to make losses hurt more. Loss aversion is already doing that work, and piling on produces avoidance, not learning. The move is to make losses informative — to give them a shape the user can act on.

Three things tend to help:

Separate the score from the review. A rating that moves after every match conflates performance with feedback. Split them: the score updates, and separately, a short prompt asks what the loss revealed. The prompt is the learning; the score is the tracking. Mixing them means the score absorbs all the emotional weight.

Weight the loss in the rating, but explain it. Elo does this. So does Glicko-2. The user sees a drop and a reason — opponent rating, expected outcome, confidence interval. Transparency turns a sting into a datum.

Reward the review, not just the result. If the only visible reward is a win, the only behaviour you'll get is chasing wins. Some sales teams now score deal post-mortems alongside closed deals. Some engineering teams track incident reviews as a first-class metric. The pattern is the same: put the learning loop in the reward structure, or it won't happen.

Where this leaves the next build

The 12% jump after a win isn't a bug. It's a design decision that optimises for engagement and calls it skill. For anyone building a rating, a leaderboard, or a proficiency score, the question worth carrying into the next sprint is simple: what does my system teach someone after they lose? If the answer is "nothing, the number didn't move," the system is telling them the loss didn't happen. They'll believe it. And the rating will keep climbing while the skill stays put.

The fix isn't complicated. It's just asymmetric, and asymmetry is easy to miss until you look for it.