Philadelphia Union’s head coach Jim Curtin created a bit of a stir (at least on the ASA Slack) when Joe Tansey tweeted a short quote from Curtin’s weekly press conference:

Curtin on squad rotation:

"You can rotate, but the data shows if you rotate four or more players, you will lose."

I'll type out the full quote later.
— Joe Tansey (@JTansey90) May 4, 2022

There was a bit more context to the quote and this is apparently not the first time that Curtin has made this type of statement and it has even been checked by The Philly Soccer Page in the past. But you better believe that when an MLS coach publicly starts talking about what “the data shows”, we at American Soccer Analysis are ready and willing to see if the data really does show it.

On its face, Curtin’s quote makes sense - if you replace half of your starters in a single match, that team is significantly weaker than they normally would be by default, no matter how tired those starters were. Yet there has been a lot of debate about using rotation to handle busy match schedules. Many have looked at squad rotation before, Isaac Schmidt from Sports Analytics Berkeley Group and Dustin Nation here at ASA have shown it doesn’t make too much of a difference or may potentially help to make more changes, while ESPN’s Ryan O’Hanlon covered it a bit for Europe.

As a first step, we just looked at simple correlations between the number of changes in a game and either performance in that game or the difference from the previous game. The 2020 Covid MLS season was excluded from all analyses. In the case of points and xPoints earned in a game, we actually saw an extremely weak linear correlation between changes and game performance. However, cutting against coach Curtin, there was a very weak correlation between team performance and increased changes in starting lineups. These two results would seem at odds, but teams coming off poor performances are likely to make more changes, so improvement from the lower baseline is more likely.

Looking at a full season level, there was a very small negative correlation between changes and total points in a season. These findings are consistent with previous work on squad rotation, yet they are just as contradictory.

To untangle this problem, we need to move to something a bit more complex than a simple linear regression, in this case a linear mixed effects model. While we won’t go into how these work, we can use one to model how a team’s expected points in a game are affected by multiple factors. In our case we used changes in the starting lineup from the previous game, how well a team performed in the previous game, the match week of the season, and whether a game is played at home or away (because MLS) while also controlling for the effects of an individual team in a season. In all, we looked at 5,742 MLS games since 2013 across 174 team seasons.

After running the model, we chose to look at three different scenarios of how a team performed in their previous game, a disaster-class game where a team has 0.5 xPoints (see SKC recently vs. Nashville), an even game where both teams have about 1.33 xPoints (see Columbus vs. Orlando), and a game where a team dominated with 2 xPoints (see NYCFC’s dismantling of the Quakes).

In each case the more changes a team made, the fewer predicted expected points they would earn. However, if the previous game was a disaster, the difference between no changes and rotating the entire squad was not significant. If a team had previously played an even game, it would take eight changes to the starting lineup to have a significant effect. Only when a team dominated the previous game did making three or more changes lead to a significant difference in team performance in the game.

So while it’s complicated, Jim Curtin’s vibes were correct, and likely aided by Philadelphia’s analytics department: there is a tipping point for when the number of changes in a starting lineup negatively affect results. That said, the thing that is going to make you lose the next game is not really the number of changes in the lineup, but whether you are playing at home or on the road. Yet again, MLS’s ridiculous home advantage is more powerful than the players on the field.

Of course there are plenty of limitations to this model. We didn’t take into account other competitions, like CCL or US Open Cup, which may produce more rotation for participating clubs. The data that went into the model spans seasons where both three and five substitutions were allowed and has not taken this into account. The model also doesn’t know about which players were rotated or why. An injury induced change isn’t exactly the same as a coach’s choice. Additionally, swapping, say, 2019 Carlos Vela for whoever his backup was that season is going to probably have a bigger effect than swapping out someone who is not the MLS Messi. Perhaps in future work player g+ could be integrated to get a better sense of these effects.

But to truly determine the effects of squad rotation, we need to do something drastic. MLS needs to start playing all games at Disney World again. Only then will we be able to truly determine the validity of the Curtin Theorem. Don Garber, we are eagerly awaiting your announcement.

Acknowledgements: We’d like to thank Jay Carter, Kieran Doyle-Davis, and Ben Bellman for their help with modeling and interpretation.