The Evolution of MLS Penalty Kicks (and How to Fix Them) / by Alex Bartiromo

By Alex Bartiromo

Back in 2017, Vox published a video summarizing research from Michael Mauboussin’s book The Success Equation, which ranked the major team sports on a scale of luck to skill using a formula that included games played, player size, number of possessions, chances, and various other factors. This research wasn’t intended to measure player skill—surprise! professional athletes tend to be very skillful at their chosen sport—but rather how well their sports “capture” that skill. in other words, the study sought to show how well results in those sports could be predicted by player skills. Soccer—specifically, the Premier League—came out as the second most “skill-based” of the major sports, ranking behind only basketball in terms of its non-randomness. Still, as anyone who’s watched any CONCACAF matches can attest, luck is an, um, “relevant” factor in the outcome of a match.

Still, beyond the obvious instances of human fallibility (and the question of if and how much the introduction of VAR has reduced this “luck factor” is a question that should be explored in more depth) the video brings up the question of what aspects of the sport are “lucky” vs. “skilled”, and whether the existing balance of those two is the most desirable.

Ok, you got me: I’m talking about penalties. Because man are they frustrating. Picture this: team A (your team, the team you follow week in, week out and probably project an unhealthy amount of your internal conflicts onto) is absolutely dominant, racking up shot after shot, but only has a one goal lead from a header off a corner kick in the 12th minute. And in the 80th minute, team B, which hasn’t had an attacking possession in the second half, somehow makes it to the edge of your box. There’s a collision, legs are tangled, your right back is gesturing like Paulie Walnuts at the ref, and next thing you know you’re walking out of Colorado with only one point. Can you visualize that, or a situation just like it? Of course you can. You’ve seen it a million times.

At least it feels that way. Lest you think that I want to upend FIFA rules and regulations just because the Red Bulls haven’t been winning too much this season, I looked into some penalty shot data to try and figure out just how lucky teams are getting.

Year PK xGt Non-PK xGt xPKt PK xGp Non-PK xGp xPKp PK Goals Non-PK Goals Succ. PK Succ. PK / Goals * 100 xGKeeper
2011 719.7 698.8 21.0 775.0 708.8 66.2 775 706 69 8.9% 0.79
2012 780.1 763.2 16.9 840.7 784.2 56.5 836 779 57 6.8% 0.78
2013 765.8 746.9 18.9 821.0 758.5 62.5 821 758 63 7.7% 0.80
2014 812.7 780.9 31.8 901.0 796.8 104.2 901 798 103 11.4% 0.79
2015 850.7 824.4 26.3 922.0 837.4 84.6 922 838 84 9.1% 0.76
2016 858.9 833.9 25.0 933.7 849.9 83.8 933 849 84 9.0% 0.79
2017 1002.0 972.8 29.2 1085.0 991.6 93.4 1085 994 91 8.4% 0.79
2018 1120.3 1087.4 32.9 1215.2 1107.9 107.3 1214 1105 109 9.0% 0.82
PK: all shots from penalty kicks. Non-PK: all shots not from penalty kicks. xGt: Team Model Expected Goals. xGp: Player Model Expected Goals. xGKeeper: Goalkeeper Model Expected Goals. You can read about the differences between these models here.

Starting from the left hand side of the table we can see the difference in xG caused by penalties. The first set of numbers represents team xG (xGt), while the second is player xG (xGp). As you can see, the gap between the two is substantial. Why? As Drew writes in the article linked above, it’s because xGt “was built to measure a team’s quality”, whereas xGp simply measures the actual chances expected. That means that the two measures value the xG derived from penalties differently. Whereas xGp accounts for the actual probability of scoring from a penalty (0.78), xGt attempts to account for luck—remember, the point of this measure of value is to account for how much a team “deserved”—by lowering the xG value to 0.25, depending on the season. Since this article is attempting to analyze the luck/reward balance of penalties, these two numbers are invaluable in helping put a number on luck, however elusive that may be. If the penalty xGp is .78 vs. the penalty xGt of 0.25, then some basic math tells us that the difference is 0.53. We can call this the “luck factor” (or the stathead version, “xLuck”).

On the right hand side of the table, we have keeper xG (xGk) data. What’s important to keep in mind here is that xGk takes into account the angle and placement of the shot, in contrast to both player and team xGk,which measure factors before the shot is taken. This makes sense: it’s obviously harder for a keeper to save a shot bound for the top corner than one that’s spanked down the middle. The upshot is that an increased xGk would indicate that the probability of a keeper saving the shot is lower, i.e. that the shot has a better chance of being a goal. However, what this table shows is actually a relatively stable xGk, with a range of about .06 in the years measured. This came as a surprise to me, as I expected that with the influx of GAM and TAM, and thus, higher-quality attacking players into the league, the xG numbers on penalties would go up. It is worth nothing that 2018 had the highest xGk on record, and the only one above 0.8. Perhaps this means that it has taken a few years for the money spent to “filter down” into penalty kick taking. We will need more years of data to know for sure. However, absent that information, we have to wonder why xGk hasn’t gone up by more. One possible reason is that there has been an increase in the quality of keepers concomitant to increased spending on attacking players in the league. This seems unlikely, given that spending on this position has not kept apace spending on other positions. Unless the MLS keeper pool simply increased its talent through hard work across the board, it’s hard to believe that they became better at saving penalties as a group. A more plausible explanation is that more skillful attacking players are not necessarily better penalty takers, or not significantly so. Sebastian Giovinco, who scored free kicks seemingly at will, struggled from the spot. The qualities of a good penalty kick taker may not correlate as much with other shooting skills as one might assume. Either way, more data is needed to resolve this issue.

Let’s stick to the task at hand, though. On average, 8.79% of goals scored are penalties, chances expected to be scored at a 78.88% clip (avg. 2011-2018). Let’s put that a bit more dramatically: almost 10% of goals come from chances that are scored almost 4/5ths of the time. Given the scarcity of goals in soccer (last season, which had the highest goal scoring environment of the decade and highest in MLS since the wild west days of tiebreaker shootouts in the 90s, featured 3.10 goals per game), and the measurable effects of game states on a match, this is not an insignificant amount of relatively easy chances that are given away. Penalties have a huge impact on the game. But are the outcomes of this process fair? I think you know my answer to that.

If the fouls that lead to penalties could be proven to have prevented chances that were on par with a penalty, it might be, but that is clearly not the case. Penalties are just fouls that occur somewhat close to the goal, and how many chances have an xG>=0.78, really? (Hint: they don’t, really.)

And if penalties were evenly distributed, it wouldn’t be much of an issue, either. Everyone gets the same amount of luck. But then it wouldn’t really be luck anymore, would it? You don’t need to be a statistics nerd to realize that Atlanta, who had 14 penalty attempts in 2018, was given a massive boost over Minnesota, who only had 1. But since you’re here, let’s just assume you are and say that the luck involved in a penalty is “worth” 0.532 xG, still way above the vast majority of shots.  Should we just accept that kind of disparity? With the spirit of the recent FIFA rule changes in mind, let’s take a look at four ways the penalty rule could be changed to lessen the influence of luck on the game and better reward skill.

 1. Move the penalty spot back

One way of reducing the amount of luck awarded by a penalty is simply make the penalty shot itself harder. If penalties results in goals ~80% of the time, why not try to create a shot that will result in a goal ~60% of the time by moving the penalty spot farther back than the current 12 yards? This would still be a huge reward for the attacking team, while reducing the overall amount of penalties scored in a season. Since there is no data on unobstructed spot kicks except from the current location, we can’t know what the exact distance is that would lead to this sort of reduction in success rate, but that concern is secondary. Setting the spot with a particular conversion rate in mind would allow governing bodies to determine how much they want to reduce the impact of a penalty on the overall match, i.e. think about what would be an ideal outcome for competitive balance. This isn’t an either/or proposition, but one that could be adjusted until the “optimal” spot is found.

 2. Require the fouled player to take the penalty

In order to do research for this article, I did some very extensive, definitely scientific surveying of The People to figure out what they wanted changed about the penalty rule, and one idea that came up was to make the player who earned the penalty take the penalty (shoutouts Luis from Dizzy’s). This makes sense in terms of karma: if your team is going to get a gilt-edged chance to score, it shouldn’t be able to choose who is taking it. Basketball teams don’t get a special free throw shooter every time someone is fouled. But beyond that, this rule change might not really affect the game all that much, since attacking players (who are generally considered the best penalty kick takers) are the vast majority of players who draw penalties to begin with. If the decrease in conversion rate ended up being small, it wouldn’t be noticeable over the course of a match, or even a season, considering the role that randomness plays. Nonetheless, having, say, Daniel Steres taking a crucial spot kick in a playoff match with Zlatan Ibrahimovic attempting to telepathically manipulate his technique from five yards away could be a thrilling experience for fans.

3. Vary the amount of points awarded on a goal

 Speaking of basketball, another way of reducing the amount of luck on the outcome of a match would be to have different amounts of points awarded by type of goal. Penalties would remain the relatively common and easy tool for scoring they are now, but they simply wouldn’t count for as much. The strength of this option is that it would not change the game as it’s played on the pitch, but simply the way points are distributed (though this could obviously change the tactics and mentality of the teams playing). If you wanted to get really spicy, you could make the edge of box into a sort of “three-point line”, with goals scored from outside worth three points, goals scored inside two, and penalties one. This would reward higher skill goals (though not of all types): the 40 yard screamer into the top corner that’s scored 1.3% of the time would no longer be equivalent to a shot 12 yards away that is scored ~80% of the time. These two types of goals already feel like they are different. Why not make it reality? This would also make referee decisions less controversial, as their impact on the game would be reduced.

4. Make the standard for the penalty a clear denial of a goalscoring opportunity

One of the most frustrating things you see in soccer is when a player is in the corner of the box, facing away from goal and looking to lay the ball off to a teammate, and she/he gets fouled and “earns” a shot that is successful more than eight out of ten times. It feels unfair because it is unfair. The spirit of the penalty rule, as I see it, is to award a high-percentage shot because one was denied already due to a foul, handball, etc. In practice, penalties are regularly awarded on borderline fouls and technicalities that have little to do with preventing goal scoring opportunities directly. In light of that, why not adopt the standard for red cards when the last defender commits a foul: for a penalty to be awarded, the foul must deny a clear goal-scoring opportunity. This would limit the amount of “soft” but technically correct penalties that leave supporters of all teams feeling hard done. Of course, it could also lead to even more difficulties of interpretation, because what does and doesn’t constitute a “clear goal-scoring opportunity” when we are talking about fouls inside the box isn’t very, well, clear.

Despite what some purists might argue, the rules of the game should always be looked a critically and with an eye toward improvement. None of these rule changes will ever be implemented, but it is always good to look at alternative timelines to see where the current one is lacking. What changes do you think should be made to the penalty rule? Leave your comments below.