Measuring Goalkeepers: The Limitations of Goals - xGoals / by Bill Reno

By Bill Reno (@letsallsoccer)

Last month we took a look at how bad the Goals Against Average (GAA) stat is for evaluating goalkeepers and how the save percentage (S%) is slightly better. To show the inefficiencies of the GAA and S%, I compared them to ASA’s own unique stat of Goals Minus Expected Goals (GmxG). The GmxG looks at where shots are taken, calculating the likelihood of a shot going in from that distance and angle to goal, which ends up telling us if a goalkeeper is giving up too many goals given the circumstances. This is great for a couple of reasons, the main one being that we have a more accurate reading on shots on goal than the ol’ shot percentage. If one goalkeeper consistently sees shots from distance while another is routinely left out to dry by his defense, the GmxG will let us know how many goals each goalkeeper should be conceding even if their SOG and goals are similar. However there are some shortcomings with any stat and the GmxG isn’t any different.

The most obvious limitation of the stat is that it only values shot stopping. It does not take into account how many goals have come from poor distribution, bobbled crosses, or even weak rebounds a goalkeeper has gifted an opposition. For example, Bill Hamid leads the table by quite a margin but it doesn’t take into account his poor rebound against New York earlier in the season. Sure, it’s just one example, but it doesn’t affect a goalkeeper’s GmxG.

The GmxG is also hindered by the fact it needs a large sample size to truly even out. After about one hundred interactions (saves + goals), based off my own observations not anything scientific, it starts to accurately tell of a goalkeeper’s performance. However before then, there are too many variances that can occur to sway the numbers. Aspects the GmxG does not take into account are shot speed, a goalkeeper’s view, and the movement of the ball prior to the shot being taken. All of these affect how difficult it is to save the shot. Perhaps the most annoying usage of an GmxG is after one game. You may apply expected goals to an offense after one game, but there are too many unknowns to judge a goalkeeper on after half a dozen shots.

Lastly, and probably the most damning, goalkeepers are typically benched for mistakes that do not have to deal with shot stopping. Raïs M'Bolhi is a good example of a goalkeeper who struggled in the air and with his own distribution. In fact, he was so miserable that he hardly had a chance to save any shots before he was benched because his mistakes were so numerous. And, of course, let’s not forget Carlo Cudicini. Saving shots is clearly needed in a good goalkeeper, but it doesn’t let us know how else the goalkeeper is helping or hurting their team.

The GmxG is not a WAR (wins above replacement) for goalkeepers. It’s an intense look on one dimension of goalkeeping. Although it is obviously an important one, and one we can learn a lot from, it does have it’s limitations. Is it better than the GAA? Absolutely. Is it better than the Save Percentage? Yes, significantly. Should it be the only thing taken into account in gauging goalkeepers this season? Definitely not. At least for now. The more the stat is smoothed out and missing pieces are marked down, the more confidence we can place in it.

Keeper Team Min SOG GA xGA G - xG
Bill Hamid DCU 1341 71 12 19.9 -7.9
Nick Rimando RSL 1348 59 16 21.29 -5.29
David Ousted VAN 1829 79 20 25.06 -5.06
Josh Saunders NYC 1725 91 26 29.43 -3.43
Tim Melia SKC 764 31 5 8.28 -3.28
Stefan Frei SEA 1663 89 18 21.06 -3.06
Clinton Irwin COL 1633 72 18 20.95 -2.95
Chris Konopka TOR 861 51 15 17.68 -2.68
David Bingham SJ 1633 83 18 20.28 -2.28
Jeff Attinella RSL 388 29 9 10.76 -1.76
Bobby Shuttleworth NE 1718 79 29 30.17 -1.17
Tally Hall ORL 767 40 10 11.17 -1.17
Jaime Penedo LA 1337 69 19 19.79 -0.79
Sean Johnson CHI 859 33 11 11.11 -0.11
Jon Busch CHI 672 29 11 10.48 0.52
Tyler Deric HOU 1643 68 21 20.13 0.87
Brian Sylvestre PHI 758 41 14 13.07 0.93
Evan Bush MTL 1239 84 23 21.82 1.18
Adam Larsen Kwarasey POR 1721 63 20 18.32 1.68
Chris Seitz FCD 672 27 10 8.15 1.85
Steve Clark CLB 1617 92 29 26.44 2.56
Dan Kennedy FCD 1048 35 14 11.12 2.88
Joe Bendik TOR 574 36 11 8.08 2.92
Donovan Ricketts ORL 961 34 14 11.07 2.93
Luis Marin SKC 771 30 11 7.96 3.04
Luis Robles NYRB 1618 64 23 19.75 3.25
*Minimum 500 minutes played. Does not include own goals.

All that said, let’s finish on MLS Goalkeeper of the Year Power Rankings

  1. David Ousted - Week after week, Ousted is widening the gap between him and the rest of the crowd. Can anyone catch the Danish? It seems unlikely at this point as he has now won MLS’s Save of the Week three times in a row.

  2. Stefan Frei - Could possibly win the award just about any other year. His penalty save was great but it’s not like Ousted didn’t do that twice in one game the week before.

  3. Nick Rimando - The top three stay the same from last month and Rimando continues to defy my early year prediction. I still think he’s playing low percentage guesses but right now he’s keeping RSL relevant despite his team’s lackluster performances from earlier in the season.

  4. Bill Hamid - The lack of a Gold Cup could have played in Hamid’s favor but he can't quit these nagging injuries. Now out for 4-6 weeks, backup Andrew Dykstra could see himself on the shortlist if he continues strong play.

  5. Tim Melia - What’s crazier? Unseating a foreign player specifically brought in for a starting role and barely making the roster in the first place? Or jumping into GOTY discussion despite missing the first third of the season? It's All Crazy! It's All False! It's All a Dream! It's Alright.

  6. Tally Hall - The Lions are glad to have Mr. Hall back in goal. His body is still looking a little stiff since returning to the field but his mind is clearly in a good place as he holds down the fort in Orlando.

  7. Steve Clark - Could very well be Clark’s last time on this list as we drop down to six next month. Inconsistency is the problem and as Columbus pushes towards playoffs, that’s the main thing they’ll need from Clark.