Coaches Reward Goalscorers. But Should They?

/
Goalscorers2.png

By Eliot McKinley & John Muller

On March 30, 2019, the 16-year-old midfielder Gianluca Busio came on for Sporting Kansas City in a rout of Montreal. He didn’t do a whole lot in his half hour on the pitch—seven of his eight completed passes went backwards—but in the 78th minute he poked the ball away from a center back and slotted home his team’s sixth goal. The next week Busio was rewarded with a full 90 minutes and he scored again. The week after that, another appearance, a third straight goal. Coach Peter Vermes was sticking with the red-hot kid and it was paying off.

Alas, not all breakthroughs go as smoothly as Busio’s. On July 17, a teenage striker named Theo Bair earned his second career start for Vancouver. He made a couple of promising runs where he held off a New England defender and found a shot from a low cross, but neither chance connected. The first hit the far post and ricocheted out. Two minutes later, Bair reached back for a bouncing pass at the top of the six-yard box but couldn’t quite corral it. The shot sailed over the crossbar from embarrassingly close range and Bair tumbled head over heels into the goal, where he slapped the grass in frustration. He was subbed off, and next game he only appeared for the last 14 minutes.

Would Bair have gotten more playing time next game if he had scored? Would Busio have earned less if he’d missed? On a case by case basis it’s hard to say. But the data suggests that, in general, MLS coaches reward attackers for scoring or punish them for missing memorable chances—and those lineup decisions may be hard to justify.

Coaches — They're Just Like Us!

If a decade of analytics work on shot metrics has taught us anything, it's that soccer-watching humans are hardwired to care too much about finishing. Is it a skill? Yeah, of course. Anyone who tells you they're as good as Lionel Messi at kicking a ball into a net should be assigned to either a Champions League roster or Arkham Asylum for the Criminally Insane. But the differences in professional attackers' likelihood of scoring from similar situations are slight enough that finding good shots is a more valuable skill than finishing them. In fact, to even guess at differences in shooting ability you'll need data on hundreds of shots per player.

Unfortunately, that's just not a natural way to think about soccer. We watch sports to find out which players can do really difficult things marginally better than their fellow pros. It's hard not to see everything as proof of skill. People like heuristics, shortcuts that help us draw practical inferences right away instead of waiting for data to pile up. And we like narratives, running dramas where the young forward who cost our team last week's game seals his fate as a bottler or finds redemption depending on whether this weekend's two shots go in. As Joan Didion probably wrote somewhere, we tell ourselves stories in order to live with our choice to spend two hours watching the Vancouver Whitecaps.

But what about professional coaches—you know, the people whose jobs depend on getting their team to score. Surely they select players based on proven predictors of future goals, not the same cognitive quirks that make the rest of us misjudge strikers, right?

The Scorers' Reward

When it came to determining next game's playing time, though, the outcome of that one shot made all the difference. The players who missed their big chance in Game One averaged 61 minutes in Game Two. For the players who scored their big chance, that number shot up to 67 minutes, a 10% playing time bonus.

To understand how coaches think about goals, we looked at a naturally occurring split: next game's minutes for two comparable groups of players who either missed or converted a shot with an expected goal (xG) value of at least 0.3 in their team’s last game.

It’s worth mentioning that a shot with a 30% probability of scoring is a pretty big chance. If it doesn't sound like one, that’s because people overestimate the likelihood of scoring from close to goal. In practice, 0.3 xG is roughly the threshold where, if you miss, TV commentators start saying extremely British things like "He simply has to do better there, for me," and "He's fluffed his lines!" If you want to get a feel for what it looks like, Busio's goal against Montreal had an xG value of 0.31.

We tried to make sure there was no reason to suspect our scorers and missers might be different in other ways. The hundreds of players in each group were the same age on average, had the same position breakdown, and got similar rest before Game Two. To keep the goals we weren’t interested in from skewing our results, we excluded anyone who happened to score in Game One from a shot worth less than 0.3 xG. Our scorer group put up slightly higher Game One xG per 96 minutes than the missers, as you might expect since there was no upper limit on the xG of the > 0.3 xG shot, but this difference, though statistically significant, wasn’t large enough to predict a performance difference in the next game. In the end, the main factor that separated one set of players from the other was whether their big chance went in.

When it came to determining next game's playing time, though, the outcome of that one shot made all the difference. The players who missed their big chance in Game One averaged 61 minutes in Game Two. For the players who scored their big chance, that number shot up to 67 minutes, a 10% playing time bonus. This difference was highly statistically significant (p = 0.00057)—in other words, probably not an accident.

At first we thought this effect might be linked to age. If coaches were rewarding goals or punishing misses as evidence of finishing skill, you might expect the playing time gap to be confined to young players like Bair and Busio, who have fewer career shots to go on. (Think about it: if Carlos Vela misses a big chance at age 31, Bob Bradley’s not going to worry that he’s lost the ability to shoot.) When we subdivided our big-chance missers and scorers by age, though, older players got a significant minutes bump for scoring just like U23s did.

Screen Shot 2020-03-30 at 10.55.21 AM.png

The effect wasn't limited to big chances. We next looked at a similar split between players who made or missed a shot with less than a 5% chance of scoring, which usually means it was taken from outside the box. Here, too, scorers got a statistically significant playing time bonus over missers. That colored our interpretation of what was going on. With the big-chance (> 0.3 xG) groups it was easy to imagine that coaches might be punishing players for memorable misses, but who would bench a player for failing to score on one shot from distance? It seemed more likely that these small-chance (< 0.05 xG) scorers were being rewarded for a beautiful strike, even if their shot selection wasn’t great. Perhaps the type of shot didn’t matter and the equation was simple: you score a goal from anywhere, you play more next week. 

At first glance this all sort of makes sense, doesn’t it? If you were a coach, wouldn't you bench your blooper-reel schlubs and give your highlight-reel guys a chance to do it again? Scoring goals is good, and good results deserve rewards. Maybe the missers missed because they're worn out. Maybe the scorers scored because they're enjoying a run of good form. If you want goals in your next game, the player who just proved he can provide them seems like the obvious place to start. 

Except—here’s the thing—the data doesn't really bear that out. In practice, the big-chance missers and big-chance scorers were statistically indistinguishable in their next game.  They generated chances of identical quality, converted them at a comparable rate, and produced about the same number of goals per 96 minutes. (In fact, the players who’d missed a big chance in Game One put up slightly better attacking numbers in Game Two than the scorers, but those differences weren’t significant.) The same held true for the small-chance groups: the only thing separating players who scored a long-distance banger from those who found the stands from the same range was that the scorers got significantly more minutes next week. 

It turns out that knowing whether a player finished his chances tells you very little about how he'll play in the future, but it tells you a lot about whether his coach will let him. Why is that?

Shot of xG > 0.3 and no other goals N Age Days Rest Game 1 xG per 96’ Game 1 Mins Game 2 Mins Game 2 Goals per 96’ Game 2 xG per 96’ Game 2 G/xG
Missers 450 26.64 7.25 0.72* 78.25 60.75* 0.25 0.19 1.26
Scorers 437 26.7 7 0.83* 81.43 66.56* 0.2 0.18 1.08
Shot of xG < 0.05 and no other goals N Age Days Rest Game 1 xG per 96’ Game 1 Mins Game 2 Mins Game 2 Goals per 96’ Game 2 xG per 96’ Game 2 G/xG
Missers 4925 26.64 7.14 0.14* 82.52 62.45* 0.11 0.1 1.13
Scorers 189 26.68 6.77 0.19* 82.77 69.64* 0.07 0.1 0.72>
*Statistically significant at p ≤ 0.05

As the table above illustrates, players who scored in Game One got a significant playing time bonus in Game Two over shooters who missed from the same range, even though the missers were just as likely to score in Game Two.

The Hot Foot Problem

About forty years ago, a basketball-obsessed Stanford grad student named Thomas Gilovich approached the famous behavioral economist Amos Tversky with an idea: Everyone who’s ever played pickup or watched an NBA game knows that shooters sometimes get hot, hitting streaks where it feels impossible to miss. But what if everyone was wrong?

In other words (and probably better words, since xG isn’t really designed for small samples): Don’t reward good goals. Reward good chances.

The study they did together, The Hot Hand in Basketball: On the Misperception of Random Sequences, was a landmark in sports research. Digging through NBA field goals, free throws, and a controlled experiment with Cornell basketball players, Gilovich and his coauthors couldn’t find evidence that shooters get hotter than chance would predict, even though surveys of pro players and fans showed widespread belief that it happens. The paper ignited an academic firestorm, but through decades of peer-reviewed debate evidence for a “hot hand” effect in sports has been shown to be modest at best, no matter how strongly we feel it.

And yet the game changing study didn’t do that much to actually, you know, change the game. Players who drained a few shots kept believing they couldn’t miss. Teammates kept feeding the ball to the “hot” shooter, often at a coach’s instruction. The legendary Celtics coach Red Auerbach summed up the reaction around the league to Gilovich’s work: “Who is this guy? So he makes a study. I couldn’t care less.”

It could be that what we’re seeing in soccer goalscorers’ minutes is a belief in the hot foot. The related but slightly different explanation that coaches treat goals as evidence of finishing skill isn’t as satisfying, partly because we found a playing time bonus even for veterans with years of shots behind them. But there’s a popular conviction around the game that players sometimes find “good form” and enjoy non-random, skill-based hot streaks that rise above their usual ability for a few matches. As the saying goes, form is temporary, class is permanent.

Whether coaches are rewarding scorers for finishing or form, there’s not much in the numbers to suggest they’re behaving rationally. On one hand, it may be harmless, since scorers and missers of similar-quality chances were equally likely to score in the next game. On the other hand, players who scored from distance got the most next-game minutes of any of our four groups, enjoying 15% more playing time than those who missed from close to goal, even though the latter group was more than three times as likely to score in the next game. There’s a risk to making lineup decisions based on memorable strikes instead of expected goals.

In other words (and probably better words, since xG isn’t really designed for small samples): Don’t reward good goals. Reward good chances.

As for Busio and Bair, their seasons played out like some kind of cautionary analytics fable. After his three-game streak in the spring, Busio went the rest of the year without scoring, flickering in and out of Kansas City’s lineup. He wound up with three goals on 3.2 xG over 1014 minutes—a strong scoring rate for a midfielder, but right in line with league finishing norms. And Bair, the bottler, went on to score a debut goal that quieted the “But can he finish?” conversation in a hurry.

Needless to say, he played every minute of the next game.