Building A System for Assessing Player Value, Part 2

By Dave Laidig (@daveladig)

Last week in Part One of this series, we looked at the overall player value rating and it’s underlying method, top players, its validity, and year-to-year consistency. In this part, we’ll turn to the categories of events make up the overall rating, and examine what can be gleaned from these subcategories.

Player Value Subcategories

The main goal of the player value metric is quantifying a player’s overall contribution to a team winning. But recognizing players help teams in different ways; I decided to track where the “value” was coming from. This led me to break down the overall player value into eight subcategories; (1) shot value, (2) turnovers (defense actions), (3) shot blocks (defense actions), (4) pass value, (5) turnover or loss-of-possession value, (6) movement value, (7) F-up value (conceding PKs and red cards), and (8) goalkeeper value. In addition, I have found it useful to create a sub-index of actions associated with “playmakers”; which is called the Create Index and consists of the Pass Value, Turnover/LOP Value and Movement Value added together.

Some basic data points for the subcategories are presented in the table below. The overall rating and the issues with comparing the 2017 population and 2018 population were discussed in Part 1, and included here simply for convenience.

Category	Team Avg - Home 2017	Team Avg - Away 2017	Correlation w final table - (Team, 2017)	Correlation - 2017 & 2018 (players)
Total Player Value	1.51	0.69	0.761	0.465
Defense – Turnover Value	0.48	0.48	0.219	0.74
Block Value	0.18	0.26	-0.217	0.604
Pass Value	1.18	0.98	0.423	0.749
TO/LOP Value	-1.49	-1.46	-0.37	0.903
Movement Value	0.22	0.17	0.45	0.606
F-Up Value	-0.09	-0.17	0.69	0.099
GK Value	0.01	-0.24	0.424	0.241
Create Index (Pass+TO+Move)	-0.1	-0.34	0.383	0.736

As a brief review of Part 1, the overall player rating reflects the sum contribution of all game actions from a player. And generally, the player contribution consists the chance of scoring when the player is done with the ball; minus the chance of scoring when he first received the ball.  The chance of scoring is defined as the expected xG of a possession at that moment.

It’s worth noting that these subcategories do not overlap with each other; an effort was made to avoid double-counting actions. But while these subcategories do not overlap, it’s possible for a player’s time on the ball to cover several categories with a single touch. For example, a player could intercept a pass, successfully dribble past a defender, advance the ball, and then make a long connecting pass to a teammate. This one touch would create value for Defense-Turnover, Movement (value created moving from take-on to the area where the ball was passed), and Pass Value categories.  Most commonly though, a player’s touch will involvement movement plus passing or shot value.

Shot Value

Shot value is calculated as the chance of a shot scoring (independent of the goalkeeper) minus the zone value of where the shot occurred. For this rating, the three possible end results of a shot are (1) shot on target using the head, (2) shot on target using the feet/other, and (3) a missed or blocked shot. For missed shots, the end result is equals the standard shot xG. For shots on target, the end result is the percentage of shots on target that actually score from the same zone (based on all shots over three MLS seasons). 

The shot value subcategory is the most important for a team, with the highest average value. Shot value attempts to isolate the value created solely from the act of shooting. In other words, if a player dribbles through three defenders and takes a shot, only the act of shooting shows up in the shot value.

The end-result-minus-start-value approach leads to some key differences from a typical evaluation of shot quality using standard xG. Most significantly, the location value is subtracted from the result. Removing the location value prevents double-counting the value added during the build-up. Additionally, this feature also moderates the “poacher” value. Generating high xG shots from an already valuable location is not as impressive as generating a similar chance of scoring from further away.

Another key distinction is the post-shot component. Goals and shots on target are treated equally in order to minimize goalkeeper effects. Shot accuracy is really all the shooter controls, and if we want to isolate a player’s contribution, it shouldn’t matter whether the keeper is Nick Rimando, or a backup making his first start. In an initial pilot study, this method led to more consistent values from year to year than including goals scored.

A final distinction, penalty kicks are not significant features of shot value. Most penalty kicks are scored, and there is not a lot of variability in the outcome. And the pk math for shot value follows suit. The zone value for a pk is the avg xG for a penalty. And a missed shot is the xG minus the zone value, which is zero. And a shot on target has a higher chance of scoring, but minimally so because a penalty already has a high chance at scoring. As a result, a pk goal adds about .03 xG equivalents. (Most of the value for pks is assigned to the foul leading to the pk). As a result, shot value is not likely to be inflated, or harmed, by players that take many penalty kicks for their team.

Overall, the shot values reward shot accuracy, and minimize receiving the ball close to goal. And the shot value subcategory tends to be consistent over time. For players with 10+ shots, the year to year correlation for Shot Value per 90 minutes is 0.837, which is a bit higher than the xG per 90 correlation for the same group.

2018 Shots Value leaders*, as of September 3rd:

Player	Team	Minutes	Shot Value per 90
Adama Diomande	Los Angeles FC	860	0.55
Josef Martinez	Atlanta United	2396	0.54
Tosaint Ricketts	Toronto	620	0.43
Cristian Techera	Vancouver	1182	0.43
Bradley Wright-Phillips	New York	2223	0.40
Diego Rubio	Kansas City	653	0.39
David Villa	New York City FC	1268	0.38
Zlatan Ibrahimovic	L.A. Galaxy	1706	0.38
Alberth Elis	Houston	2180	0.37
Daniel Royer	New York	1915	0.37
Samuel Armenteros	Portland	1364	0.35
Joao Plata	Salt Lake	1358	0.35
Sebastian Giovinco	Toronto	2064	0.34
Darren Mattocks	DC United	1350	0.34
Wayne Rooney	DC United	885	0.34
Raul Ruidiaz	Seattle	659	0.33
Dominic Dwyer	Orlando City	1721	0.32
Ola Kamara	L.A. Galaxy	2287	0.32
Derrick Etienne	New York	853	0.31
Luis Silva	Salt Lake	796	0.31

* Among Players with 500+ minutes

Defensive Turnover Value

A defensive turnover is a defensive action that immediately leads to an offensive possession for one’s team. The defensive turnover value is calculated as the opponent’s zone value prior to the turnover plus the team’s starting zone value immediately after the turnover.

Most defensive actions, whether labeled successful or not, do not immediately lead to an offensive action by one’s team. A “successful” tackle may slide out of bounds, giving a throw in to the team that just “lost” the ball. Or perhaps defensive actions will deflect a pass or dislodge a ball into the path of a different opponent. For various reasons, looking at raw numbers of defensive actions can be misleading.

For this rating, I only value actions that directly affect a possession. When a player takes the ball away from an opponent, they are extinguishing an opponent’s chance at scoring (thus taking their opponents expected xG for their possession down to zero). And, by definition, they are creating a chance to score for their team by starting a possession.

In some situations, the “turnover” does not follow an opponent’s possession (e.g., after someone booted the ball to relieve pressure, or the back and forth after a goal kick). If the defender did not end an opponent’s possession, the defender does not get credit for reducing an opponent’s scoring chances to zero. And for these non-possession turnovers, the calculation only includes the start value for the team. While not identical, these non-possession turnovers are similar to some definitions of recoveries.

The defensive-turnover value is fairly consistent from year to year, with a correlation of 0.740. At the team level, the correlation with the final table is not strong, at only 0.219. However, this relationship is stronger than the relationship between the raw number of defensive actions and table results. Thus, it represents a slight improvement in capturing value for a team. But it’s obvious that correctly valuing defense in one area that needs more work, and will be addressed in subsequent iterations.

2018 Defensive-Turnover Value leaders*, as of September 3rd

Player	Team	Minutes	Def-TO Value per 90
Francois Affolter	San Jose	533	0.19
Chris Mavinga	Toronto	524	0.19
Victor Cabrera	Montreal	976	0.14
Rasmus Schuller	Minnesota United	2185	0.14
Drew Moor	Toronto	512	0.13
Lamine Sane	Orlando City	985	0.12
Mohamed El-Munir	Orlando City	1760	0.12
Aly Ghazal	Vancouver	1065	0.11
Jimmy Medranda	Kansas City	782	0.11
Guram Kashia	San Jose	765	0.11
Rod Fanni	Montreal	1868	0.10
Jacori Hayes	FC Dallas	1088	0.10
Yangel Herrera	New York City FC	1072	0.10
Antonio Mlinar Delamea	New England	1233	0.10
Harold Cummings	San Jose	1512	0.10
Jason Hernandez	Toronto	562	0.09
Mohammed Adams	Chicago	969	0.09
Dejan Jakovic	Los Angeles FC	997	0.09
Samuel Piette	Montreal	2613	0.09
Christopher Durkin	DC United	1242	0.09

* Among Players with 500+ minutes

Block Value

The Block Value is calculated as the opponent’s shot xG. The idea is that if a defender blocks a shot, then they have prevented that opportunity to score. And the xG created by the shot has been nullified, and the credit should go to the defensive player.

Blocks are separated from turnovers because the value has different effects. This category was not as consistent from year to year with a correlation of 0.553. At the individual game level, block value is positively correlated with game results. However, at the season level, block value is negatively correlated with points per game (-0.217). Thus, it seems that blocks are associated with winning a game, but too many over the course of a season are associated with poorer performances.

Reconciling the role of blocks, it seems that blocks capture some of the randomness of games and is not necessarily a repeatable skill. Over a longer period of play, lots of blocks is a sign of a team that allows too many shots; a sign of a defense under duress.

Pass Value

The Pass Value is calculated as the pass recipient zone value minus the zone value for the start of the pass. Pass Value is only calculated for completed passes.

If a pass is incomplete, but the team retains possession (i.e., another offensive action follows), the value is set to zero. Looking at possession sequences, I found it too difficult to create a uniform rule for whether an incomplete pass (non-turnover) helped or hurt, and which player is responsible. An incomplete pass, with possession retained, could be a situation where a pass was deflected out of bounds or to a teammate, or a defender committed a foul. With many possibilities, I took a no-harm-no-foul approach and set these values at zero for now.

If the player has an incomplete pass and the team loses possession, it is a turnover and captured on the turnover/loss of possession (TO/LOP) scale.

From this definition, pass value does not capture any movement or advancement with the ball at one’s feet, only the value from the act of delivering the ball to a teammate. And the value for a particular pass can be negative if the start zone was had more value than the delivery zone (e.g., back passes or square passes). Even for passes that are negative, the result is always better than the negative value from a turnover.

Pass Value has a strong relationship among players from year to year (0.735), and has a modest positively correlation with the season table (0.423 in 2017).

2018 Pass Value leaders*, as of September 3rd

Player	Team	Minutes	Pass Value per 90
Nicolas Lodeiro	Seattle	1919	0.39
Romain Alessandrini	L.A. Galaxy	1464	0.37
Mauro Diaz	FC Dallas	853	0.37
Yoshimar Yotun	Orlando City	1638	0.34
Federico Higuain	Columbus	1971	0.32
Maximiliano Moralez	New York City FC	2551	0.30
Darwin Quintero	Minnesota United	1792	0.30
Diego Valeri	Portland	2336	0.30
Marc Rzatkowski	New York	1217	0.29
Romell Quioto	Houston	1975	0.29
Pedro Santos	Columbus	1814	0.28
Graham Zusi	Kansas City	2524	0.28
Diego Fagundez	New England	2275	0.28
Alejandro Romero Gamarra	New York	1976	0.26
Haris Medunjanin	Philadelphia	2102	0.26
Zoltan Stieber	DC United	1734	0.25
Magnus Wolff Eikrem	Seattle	655	0.25
Carlos Vela	Los Angeles FC	1902	0.24
Magnus Eriksson	San Jose	2240	0.23
Borek Dockal	Philadelphia	2055	0.23

* Among Players with 500+ minutes

LOP/TO Value (Loss of Possession & Turnovers)

The loss of possession or turnover (LOP/TO) value sums up the (negative) value of non-shot actions that end possessions for a team. And losing possession means a team has zero percent chance of scoring. Following the end-result minus starting-value framework; the LOP/TO value is calculated as zero minus the start value. The values represent the expected possession results that were lost by the player.

In addition, I decided that it was not fair to have a completed pass end a possession, and penalize the passer. Thus, whenever a completed pass is the last action in a possession, I have added a special LOP/TO entry for the pass recipient. Thus, the passer gets credit for a completed pass and players who would not otherwise be recognized are credited with a lost possession and the negative value.

The year to year correlation for this subcategory is very high (0.903). And while the TO/LOP values are (minimally) positively correlated to match results, they are negatively correlated to the season table. It’s possible that losing possession may not be great for a particular game, but high values on this category may reflect an aggressive offense, which could lead to team success. Or perhaps values reflect a team that can get into the final third before losing the ball. It’s worth noting that attacking players have more negative LOP/TO values (but they offset it with higher shooting values in the overall ratings).  The TO/LOP values are not much help a stand-alone scale, but can be used in conjunction with other scales and position norms to tease out playing styles and decision-making.

Movement Value

Movement value represents the value added by a player from advancing the ball at one’s feet. A common example would be the difference between where the player received a pass and the location of the next action in the possession chain. Due to the mechanics of my possession chain files, movement value is calculated as the portion of the total value that is not captured by shots, passes, or any of the other subcategories. Movement can be a positive value (taking the ball to a higher value area), or a negative value (retreating from a higher value area to a lower value area).  

The Movement scale is correlated year to year at 0.606 for players, and modestly related to table standings (correlation = 0.450).

2018 Movement Value leaders*, as of September 3rd

Player	Team	Minutes	Move Value per 90
Ignacio Piatti	Montreal	2465	0.09
Alberth Elis	Houston	2180	0.09
Johnny Russell	Kansas City	1853	0.08
Ilsinho	Philadelphia	942	0.08
Valeri Qazaishvili	San Jose	2433	0.08
Ezequiel Barco	Atlanta United	1467	0.08
Ismael Tajouri-Shradi	New York City FC	1107	0.08
Cory Burke	Philadelphia	1041	0.08
Fabrice-Jean Picault	Philadelphia	1745	0.08
Sebastian Saucedo	Salt Lake	1001	0.08
Darwin Quintero	Minnesota United	1792	0.08
Aleksandar Katai	Chicago	1984	0.08
Miguel Almiron	Atlanta United	2543	0.08
Michael Barrios	FC Dallas	2052	0.08
Joao Plata	Salt Lake	1358	0.07
Justin Meram	Orlando City	1563	0.07
Santiago Mosquera	FC Dallas	1193	0.07
Sebastian Giovinco	Toronto	2064	0.07
Darren Mattocks	DC United	1350	0.07
Romell Quioto	Houston	1975	0.07

* Among Players with 500+ minutes

F***Up Value

The F-Up Value represents game-changing mistakes such as red cards and conceding a penalty kick. The value of a red card follows Mark Taylor’s formula for the effect of a red card on goal expectancy; i.e., 1.45 * (Minutes Excluded from Game/Game minutes)^0.85. For fouls leading to a penalty kick, the defender is assigned -0.55 xG equivalents and the player that suffered the foul gets +0.20.  Thus, a PK award leads to a difference between the teams of 0.75 xG equivalents, which is close to the expected value of a penalty shot. The decision to split the penalty value at -0.55 and +0.20 is simply a decision made on the ASA slack channel based on what seemed fair (and subject to future revisions with better data).

Applying the definitions, players with positive values in this subcategory are able to draw penalties. Players with negative values concede penalties and earn red cards. The 2018 leader, as of September 3rd, is Luis Argudo (0.05 per 90). Sebastian Saucedo has the highest value for a player over 1000 minutes (0.04 per 90). At the other end of the scale, Dejan Jakovic brings up the rear with -0.18 per 90.  In other words, as calculated, Jakovic’s screw ups have cost his team the equivalent of two goals over his eleven games this year.

The F-Up values were not strongly correlated from year to year (0.166). While this scale is consistent with match results, it appears to capture some of the randomness that occurs in games and not necessarily some player trait.

GK Value

Goalkeepers are notoriously difficult to value, and nothing new was introduced here. The goalkeeper value is calculated as the ASA xGK value minus the shot result (i.e., minus 1 for a goal conceded or minus 0 for a save). Unlike shots, the GK values consider whether the ball scores because the keeper is the last actor before it hits the net. And other pros and cons applicable to xGK ratings apply here as well.  Shots that miss or hit the post are not credited to the keeper, since no goalkeeper action was required to prevent the score. This category has positive correlations with match and season results.

The challenge for this measure is that the year to year correlations are very low (0.241). As a result, although this measure seems to explain match results, player’s value is not very predictive about the next year’s performance. Further, there are home and away differences that are difficult to explain if this measure was entirely based on a keeper’s skill. It’s possible that this category captures an unmeasured defensive factor, or randomness and thus unfairly blames or rewards a goal keeper.

As a result, I’d recognize that unexpected saves are tremendously valuable, and this subcategory may inform where value is coming from in a series of match reviews.  But because of the challenge in interpreting GK value, I tend to separate goalkeepers from field players.  And will likely do so until better keeper metrics are created and included.

CREATE Index

While looking for creative playmakers, I decided to combine the most relevant subcategories to form a new “Create” Index: consisting of the sum of (1) Pass Value, (2) LOP/TO Value (which is negative), and (3) Movement Value.

This index provides quick look at how positive a player’s actions are, and whether the lost possessions are ultimately worth it. Strikers may have negative values because they receive the ball in high value locations and either shoot (not captured in this category), lose the possession (because of defense intensity near goal), or send the ball back to lower value areas to recycle and try again. Defenders tend to have lower values because they are not consistently delivering the ball into dangerous areas.

As a team measure, it has a modest correlation with final standings (0.383). But on a player level, the year to year correlation is fairly strong at 0.736.

2018 CREATE INDEX leaders*, as of September 3rd

Player	Team	Minutes	Create Index per90
Graham Zusi	Kansas City	2524	0.17
Yoshimar Yotun	Orlando City	1638	0.17
Romell Quioto	Houston	1975	0.16
Nicolas Lodeiro	Seattle	1919	0.15
Haris Medunjanin	Philadelphia	2102	0.14
Zoltan Stieber	DC United	1734	0.14
Jaylin Lindsey	Kansas City	591	0.13
Mauro Diaz	FC Dallas	853	0.13
Ismael Tajouri-Shradi	New York City FC	1107	0.12
Andrew Farrell	New England	2474	0.11
Albert Rusnak	Salt Lake	2377	0.11
Matt Besler	Kansas City	2182	0.10
Maximiliano Moralez	New York City FC	2551	0.10
Adam Lundqvist	Houston	1062	0.10
Romain Alessandrini	L.A. Galaxy	1464	0.09
Liam Ridgewell	Portland	678	0.09
Auro	Toronto	1133	0.09
Alejandro Bedoya	Philadelphia	2342	0.09
Sebastian Saucedo	Salt Lake	1001	0.09
Jack Price	Colorado	2278	0.09

* Among Players with 500+ minutes

Next Steps

Although this is an early evaluation, and the ratings will continue to be improved over time, initial signs point to its utility. These ratings tend to align with team performance, and allocate value to individual actions. The method of calculating minimizes (but doesn’t remove) the influence of teammates, allowing greater confidence that the rating reflects the player’s contribution. In addition, this feature also increases the utility of cross team comparisons.  And quantifying a player’s contribution to results also informs the allocation of salary resources. One can consider the estimated effects of positional upgrades, likelihood of a successful acquisition and the like. 

And a performance metric can help overcome observer biases.  By focusing solely on results, and actions that lead to results, valuable actions that tend to be missed can be captured.  Human memory is fallible, and it’s a challenge to watch 11 players, all moving for 90+ minutes and appropriately weight the relative value of every touch.

However, even with some utility, there is ample room for improvement of this baseline performance rating. The role of positions and tactical assignments needs further work. In addition, the current framework focuses on acts that directly affect possession status because these calculations are less speculative and easier to calculate. However, this process can also be used for other game actions (possession result with action v. possession result without action), and can be tailored to individual or club points of emphasis.  For example, future versions will consider the value of defensive fouls leading to free kicks, and defensive actions that do not interrupt an opponent’s possession.

Further, any valid performance metric can be used as the basis for a WAR-like metric (my earlier attempt is archived here).  And using these player values may also serve as the foundation for a Points Above Replacement metric.  Team goal differential is very highly correlated with league points per game.  Because this rating is reported in xG equivalents, it is a small leap to convert a goal added (or taken away) to expected points on the table.  The only real challenge is determining the replacement level that makes sense, and then applying the math. 

Finally, measuring value can serve as a dependent variable to unlock other insights into a game. For example, one might look at value created over the course of a game to determine when the best time for a substitution occurs. Or one may use value created to assess whether high-risk, high-reward strategies yield better results than conservative play.  Or one can evaluate value create to judge which player combinations are most effective. In short, if this rating mirrors what matters most – getting results – then it can be a useful tool for analysis.