Over the past decade, the NFL entered its (somewhat limited) version of Moneyball as more and more teams are using advanced data and integrate analytics into their day-to-day operations. the use of data and advanced analysis has also become common in the public sphere with tons of great online content and studies of the game (and everything around it). With so much data and great analysis out there, predicting performance over time has become common. Yet, one prediction remains relatively elusive - a successful playoffs run. The NFL playoffs format (a single game elimination tournament) makes it super hard to nail down the odds even with a lot of data and a host of good indicators of performance. The Cowboys and Bills have the best point-differential over the past 3 seasons, yet not much playoffs success to talk about. Three times since 2019, the number one ranked team in DVOA failed to win a single playoff game (2019 Saints, 2021 Cowboys and 2022 Bills).
As advanced analysis has become great at projecting season-long performance, but the question remains: what about the playoffs?
To tackle this question, I pull data from RBSDM.com on offensive and defensive EPA/play (2019-2023), and test to what extent high ranking in either metric can predict playoff success. You may wonder what do I mean by playoffs success? It's a great question (with many possible answers) but I emphasize the ability of teams to get to the Super Bowl and to a lesser extent, the conference championship games.
As numerous talking heads, players and coaches have told us, the NFL season is a marathon, not a sprint. Therefore, a good ranking early-on does not guarantee anything. But does that mean teams have to be good throughout the season? what about the second half, or maybe it is more important to finish strong and get that playoff momentum?
I address these alternatives by running several sets of logistic regression models that compare how teams' EPA/play rankings predict playoffs success based on the ranking for a season-long, half a season, and the final stretch.
A Season’s worth (of EPA)
The first set of models test the probability of getting to the Super Bowl and conference title games using teams’ end-of-season offensive and defensive EPA/play ranking over a 17 games season (18 games since 2021).
Overall, the models indicate that both metrics are important, but offensive EPA is much more powerful: Teams that are ranked near the top on the offensive side have a predicted probabilities that range between 20-30% to reach the Super Bowl. The numbers are much higher (50-60%) for getting to the conference title games. On the other hand, the analysis for defensive EPA ranking produces substantially lower odds, mostly in the single digits for either success option.
So the broader lesson is that offense is much more important for predictive purposes. I build on that and focus on testing the different probabilities of reaching the superbowl and conference title game for the top-20 teams in offensive EPA/play.
The panel on the left presents the probabilities to reach the Super Bowl for the teams in 1-20 ranking in offensive EPA/play. The right panel is the conference title game. Beginning with the SB: it is hard to get to the big game - only the top-2 teams have odds above 20%, and those in the 3 to 5 slots see their chances drop from 19%, to a little over 10% for the number-5 ranked offense.
These are not high probabilities and they reflect the challenges for even the best teams (per EPA/play) of surviving the gauntlet of the one-game, single elimilation NFL playoffs. BTW, a 27.9% of reaching the Super Bowl (the odds for the top ranked offense) are not as low as it seems. For the upcoming 2024 season, recent sportsbooks place the Chiefs as the most likely to win the Super Bowl with a +550 odds which have an implied probability of about 15.4%!!!
The projection numbers are more comforting for the conference title games. The top offense has a little over 60% chance, followed by 53% for the second-ranked offense. Rounding-up the top-5, the odds remain fairly high with 46.5% (number 3 offense), 39.6% (number 4) and 33% for the fifth-ranked offense. The higher values reflect the fact that many teams that are ranked at the top of the offense charts are also likely to secure a bye week, and thus only need to win a single (home) game to reach the conference championship (see the 2020 Packers, 2023 Niners and multiple Chiefs teams since 2019).
How much do these results reflect reality? In other words, is it true that high-rank offenses (by EPA) are actually better suited for success in the playoffs?
Overall, the projections are pretty good - over the five years of data used in this article, only in two cases, the SB participants were not in the top-5 in offensive EPA.
The projections are slightly better for the conference title games when in most years at least 2 participants are ranked in the top-5, and in some cases all four had high performing offenses.
The chart displays the odds for the top-5 offenses (based on the models) and the best and weakest matches of seasons from the data. Tthe best match is the 2020 season when both Super Bowl teams were ranked third (KC) and fifth (TB) in full season offensive EPA/play. Also, all four conference title participants were in the top-5 offenses list. The weakest match is last season (2023) when only one top-5 offense (SF) reached the conference title game and the Super Bowl.
"The NFL Season starts after thanksgiving"
How many NFL coaches have you heard repeat said mantra? Well, it seems like they actually have a point. For the half-season EPA analysis, I use the rankings in weeks 9-17 (or 10-18 for 2021-2023) and test the predictive probabilities for reaching the Super Bowl and conference title games.
The SB odds (the left panel) improve when looking at the half-season offensive EPA rankings: almost 40% to the top-ranked, and 33% for the second-ranked offense. Teams with the third and fourth best offenses in these weeks have 27.6% and 22.3% to reach the big game, the number-5 offense is at 17.9%. The relatively higher odds can be a result of teams using the early part of the season to jell and try different schemes or plays. Then, their real skill is more evident during the last 8-9 weeks. The data over that time period offers a sense of sustained offensive performance over multiple weeks.
How about the conference championship games? In this case, the projections are fairly similar to the full season data. Odds for the top-2 offenses decrease by about 4% compared to the end-of-season data. The probabilities remain almost the same as the end-of-season EPA rankings for the third to fifth-best offenses. When looking at offensive metrics over the second-half to predict the conference title games, it seems that the top teams are clustered more closely as the gap between number 2 and 3 is less than 5% comapred to almost 7% when using the season-long data.
Do better projections (based on second-half data) are a good reflection of what actually happens in the playoffs?
Once again, the chart provides the predicted odds for the top-5 offenses (models using half-season EPA data), and I add the best and weakest matched seasons.
The 2023 season makes a second appearance, this time as the best match for the half-season EPA/play metric. In this case, both Super Bowl teams are in (KC at third and SF at the 5-spot). Also, three of the confernece title participants have a top-5 offense, with the Packers (the number 2 offense) having a strong playoff run which ended with a close loss to the Niners who reached the NFC title game (please don’t ask me to delve into Pit here, it is strictly Mike Tomlin sorcery).
On the other end of the scale, 2021 is the weakest match: No top-5 offense over the second half of the season made it to the Super Bowl, and only KC (2nd) and SF (4th) reached the conference championship games. How can we explain that? It may be hard to demonstrate with data but this was a COVID season so it has to be part of the deal. Also, some of the top EPA teams that half-season suffered either surprise losses (the Chiefs second-half crumble and loss to Cincinnati) or got completely exposed (the 3rd-ranked Patriots were hammered by Josh Allen and the Bills in the Wild card round).
The "End of season momentum"
Ever heard of that? It is a common way to explain successful playoff (and SB) runs like the Giants (2012) or 2010 Packers. The logic is simple: teams that hit their stride in the final weeks of the season keep getting better, enter the playoffs with momentum (yea, that thing) and are more likely to advance. To explore the stretch-run EPA hypothesis, I use the rankings in weeks 14-17 (or 15-18 for 2021-2023) and test the predictive probabilities for the SB and conference title games.
The data does really not align with the momentum argument (more here): the SB odds plummet as no team has even 20% chance of reaching the big game, and the gap between number 1 (18.8%) and number 5 (13.1%) teams is a mere 5%.
It looks a little better for the conference championship games, but still remains the lowest of the three options I explore as the top offense has only 38.8% of reaching the conference title game, followed by 35.4% and 32.1% for the 2nd and third-best offenses. Teams in the 4 and 5 spots have less than 30% chance of reaching the conference title games.
How can we explain the gap with the full-and-half season metrics? For starters, for the final stretch analysis, each data point is based on, well, less data. It is logical that a 4 or 5 weeks’ worth of data are less indicative of team’s real ability and only represents a short run of good (and perhaps lucky) games. Those runs may not even end in a good playoffs spot, which would make it harder for the high-ranked (by EPA) team to advance in the playoffs.
Perhaps the less than stellar predictions actually better materialized in reality?
For a third time, I chart the predicted odds for the top-5 offenses and the best and weakest match seasons based on the EPA/play rankings from the last 4-5 games of each season.
The 2019 season offers the strongest match with both Super Bowl teams: SF (3rd spot) and KC (5th). That season also has three of the conference finalists as top-5 offenses during the end of season stretch-run. The weakest match is 2022 when only one Super Bowl team (Chiefs, ranked 1st) had a top-5 final stretch offense, and two of the conference finalists (Niners and Chiefs).
What is the main lesson of the analysis?
The "explosion" of advanced metrics prompted extensive use of analytics and advanced studies of the NFL. Despite so much quality research, offering many great insights, the playoffs were always a hard "nut to crack", mostly due to the one-game elimination format.
Yet, my analysis finds that some of the data does offer good projections. In particular, using the half-season worth of offensive EPA/play rankings led to a fairly high predicted probabilities to either the conference title of the SB. On the other hand, the end of season stretch-run hypothesis does not seem to help in projecting playoff success.
Teams that are good throughout the season are generally more likely to be good at the playoffs. But the end-of-season high ranking may also reflect dominance that wanes over the final weeks (for whatever reason), and that drop in performance may carry-on to the playoffs. On the other hand, it makes a little more sense that teams that are good during the second half of the season are able to sustain that skill-level and perform better in the playoffs.
So when your highly-touted team begins the season not as expected (I'm looking at you, Texans and Packers), take a deep breath and relax - it is more likely that post-thanksgiving football that is more predictive of your teams' odds of accomplishing that playoff success.
Enjoy the season!!
Comments