A couple weeks ago I took a look at the role that penalties play in adding luck to the outcome of an MLS match. In the process of writing that article, something came up that warranted further investigation, but didn’t quite fit into my initial piece. We wondered whether the introduction of Video Assistant Referee (VAR) in 2017 had changed the distribution of penalty calls in MLS. With that in mind, I decided to take another look at the penalty data to see what I could find about the nature of penalty calls themselves, rather than how they affect the outcome of a match, and whether the introduction of new technology affected referees’ decisions.Read More
Penalty Kick Statistics
Back in 2017, Vox published a video summarizing research from Michael Mauboussin’s book The Success Equation, which ranked the major team sports on a scale of luck to skill using a formula that included games played, player size, number of possessions, chances, and various other factors. This research wasn’t intended to measure player skill—surprise! professional athletes tend to be very skillful at their chosen sport—but rather how well their sports “capture” that skill. in other words, the study sought to show how well results in those sports could be predicted by player skills. Soccer—specifically, the Premier League—came out as the second most “skill-based” of the major sports, ranking behind only basketball in terms of its non-randomness. Still, as anyone who’s watched any CONCACAF matches can attest, luck is an, um, “relevant” factor in the outcome of a match.
Still, beyond the obvious instances of human fallibility (and the question of if and how much the introduction of VAR has reduced this “luck factor” is a question that should be explored in more depth) the video brings up the question of what aspects of the sport are “lucky” vs. “skilled”, and whether the existing balance of those two is the most desirable.Read More
xN is our weekly look at what you can expect to read, write, and discuss about Major League Soccer this week. We take a look at each prospective narrative and rate it based on its strength and whether or not it has any actual merit.
Last week, I asserted that if Toronto failed to secure three points this week that the talk of the league would be whether or not last year’s all conquering heroes would even manage to make the playoffs this season. Unsurprisingly, they failed to acquire those three points, surprisingly nobody is really talking about it all that much. Well, since I’m CNO (Chief Narrative Officer) of this league now (self appointed, the term is lifelong meaning it remains until such a time as I die or get bored. Smart money on the latter.), I’m going to go ahead and make it a narrative because a) it’s important and b) I can’t really think of another thing to write about this week. I mean I guess we could cover VAR again, but NAH.Read More
Note: If you're not interested in the math, skip down to “With that in mind". Alternatively, if you're especially interested in math, checkout my github repo with the data and a jupyter notebook.
What if we wanted to rank MLS penalty kick takers? What would be the best way to go about it?
We could look at historical PKs, and take the players who have converted the highest percentage of their chances. Here are a handful a players who have scored 100% of their regular season penalties, going back to 2011:Read More
According to an article on BBC, there is a great difference in penalty kick conversion rates at various points in a World Cup shootout. When a kick would win the shootout, players in the World Cup have converted 93 percent of their opportunities, but when facing elimination with a miss, players have converted just 44 percent of the time. The article doesn't cite sample sizes for these situations, but we do know there were 204 penalties taken over 23 shootouts in the data set. Every shootout has to include at least one such chance--either a chance to clinch or a chance to choke--so a conservative estimate would be sample sizes of 10. And in fact, all we'd need for statistical significance are sample sizes of 10. Check!
Even with statistical significance covered, there could very well be some selection bias here, as perhaps the best PK takers are saved for clinching moments. The combination of small sample sizes and selection bias might explain a lot of the discrepancy in conversion rates, but that's just not a fun conclusion. So let's assume there is some effect of pressure.
In my mind, a PK to clinch a shootout should have some pressure associated with it, just as a PK to avoid elimination would. But what this data suggests is that it's the pressure to avoid elimination that really gets to players.
So I thought I'd check it out in MLS. The only problem is that we don't have nearly enough shootouts that I can access. So instead I will look at in-game PK conversion rates in scenarios where the shooting team is either down one or tied, controlling for which half and whether or not the kick taker is at home.
The results of a logit binomial regression led me to a few conclusions. First, taking a PK at home doesn't significantly alter its chances of going in, but there is significant interaction between the gamestate and the half. There are four scenarios that seem to matter for PK conversion: tied in the first half, tied in the second half, down one in the first half, and down one in the second half. Here's a chart that summarizes those outcomes in MLS:
From our knowledge of World Cup shootouts it was predictable that the highest conversion rate belongs to the situation with the least pressure. Tied in the first half, a miss still leaves the team with a lot of time to win, and there the PK probabilities are highest. What's somewhat baffling to me is the rest of the chart. Like for instance the incredibly low conversion rate when down a goal in the first half. Though a sample size of 13 is small, the difference between 88.4 and 38.5 percent is still very statistically significant (p = 0.0002). Or how about why facing a deficit seems to matter in the first half but not the second.
I find it hard to blame selection bias here for our findings. Teams that go down a goal in the first half are likely to be worse teams with potentially worse penalty kick takers, but then that wouldn't explain why they are able to perform well from the spot in the second half. And teams that are tied in the first half have no reason to be the better teams on average, though it's that group that has converted 88.4 percent of its penalties. I'm left to wonder if I don't understand psychology, or if this is all a type-I error. After all, if we ignore deficits in the first half, then there is no statistical significance between the other three scenarios.