Roster Consistency Part Two: Do Consistent Lineups Lead to Better Results?

By Dustin Nation (@D_Naish)

In a previous article, I looked at the effect of roster consistency on overall team performance. There were enough interesting trends in the data that I wanted to look a little closer and try to see if there is a “right” number of changes that teams should make on a week-to-week basis.

After looking at each squad’s rotation and how it affected their performance over the past three years, it makes sense to look at how changing lineups from one week to the next effected team’s performances in that week. That is to say, given a team’s roster changes from the previous week, how likely were they to perform well?

A quick note before we begin: as discussed previously, there are many reasons that teams would rotate players and, as such, simply looking at weekly squad rotation numbers doesn't paint the entire picture.

Let’s dive right in and take a look at each year.

2015

Here you can see that in 2015, the most common number of changes that teams made from week-to-week was two, followed by three changes and one change. It makes sense that one to three changes is most common from observing the league, and as you’ll see, it is a pattern that holds true across all years evaluated in this article.

Interestingly, teams that made zero, one, or two changes in a week were more likely to have won than lost, but teams that crossed the line and made three or more changes were more likely to have lost than won. I’ll call that line the “magic number.”

2016

Making two changes was again the most common number of changes in 2016, and again one, two, and three were the by far the most frequent. Overall the distribution of changes looked very similar to 2015.

The performance graphs, on the other hand, do not.

While the likelihood of winning dropped as more changes were made, there was a lot more turmoil between the likelihood of losing and drawing. There was also not a defined grouping in zero to three changes like there was in 2015. Teams that made two changes in 2016 were actually more likely to lose than win, and teams that made three changes were more likely to win or draw than lose.

The “magic number” where teams were more likely to lose than win moved to at least four lineup changes per week.

2017

The distribution of changes in 2017 was fairly similar to previous years. Despite one overtaking two as the most common number of week-to-week changes, one, two, and three, were still the most numerous number of changes in 2017.

Unlike previous years, the results for this year had no real “magic number” where teams who had higher numbers of changes were always more likely to lose than win. In fact, strangely enough, if you look all the way down to the right, teams who made nine changes were actually 2-0-0. In reality, that’s a very small subset of the total, so it’s probably safe to ignore it and declare the “magic number” for 2017 to have raised to five changes.

Also of note, 2015 and 2017’s likelihood of winning stayed above the likelihood of drawing (at least until seven or eight changes), while 2016’s likelihood of wins and draws swapped places at four and again at 6-8. That’s because 2016 had far more draws than the two preceding years.

Here’s the breakdown:

Year	Wins	Draws	Losses
2015	262	136	262
2016	225	210	225
2017	277	172	277

Point Delta from Previous Week

In order to truly know how the number of changes a team made effected its performance, we need to look at not only the number of points the team obtained that week, but also the difference in points from the week prior. This will tell us the direct result of making those changes.

To come up with this metric, I took the points earned in a week, subtracted the previous week’s results, summed those deltas with the other deltas for that number of week-to-week squad changes, and then averaged that across the number of times teams rotated that number of players.

That may be a bit much to follow, so here’s an example: If a team made two changes to its starting lineup and won this week (earning three points) after drawing the week prior (earning one point), then the delta would be two points and would be added to the pool of results for two changes. Likewise, a team that made four changes and lost this week (earning no points) after winning the prior week would add negative three points to the pool of results for four changes.

Obviously this doesn’t give credit to a team for successive wins or punish them for successive losses, but it does give a good idea of the overall tendency of teams to improve or worsen based on the number of changes they made to their lineup.

Here’s what the average deltas looked like for all three years:

I’ve left off results for changes greater than five because the sample sizes were so low and the results so variant.

The most surprising trend here is that in all three years teams that made three, four, and five changes from week-to-week had positive results on average. One explanation for this is that when teams lose, they tend to make more changes to find the right combination. Since their previous week’s results were losses worth zero points, any improvements would increase the average.

By the same token, teams that are winning are likely to make fewer changes, which might explain why zero and one changes average negatively. Winning again won’t increase the average, but since there were more combined draws and losses than there were wins, it stands to reason that teams that made zero changes didn't pick up a consecutive win every time.

It’s very interesting that all three years have such similar shapes. This lack of variation might validate that these tendencies mentioned above might be universal. As we discussed in the previous article, 2015 and 2017 were Gold Cup years and thus had forced roster changes due to international players being away, while 2016 didn’t, yet the average differences in results for different numbers of week-to-week roster changes remained remarkably similar across them. Obviously three years is not enough data to make a definitive claim of the trend being universal, but it will be interesting to see if 2018 shapes up similarly with the forced substitutions of the World Cup.

Final Thoughts

These numbers still don’t include the time of the season in which changes are made, nor do they take into consideration the position of the players being changed in the starting lineups, but it’s definitely interesting to see the patterns that emerge in direct results from managers making changes in a given week.

Note: Numerical data for this article can be found in csv format at https://github.com/d-nation/mls-roster-consistency