It's fourth down. The pressure is mounting. A big decision must be made. Do you kick it or go for it?
Before any given fourth-down play, an NFL head coach must decide between keeping the offense on the field or calling for the special teams unit. Whether the coach makes the "right" decision is more about process than outcome. The optimal call is the choice that gives the team the best chance to win with the information available at the moment the decision is made. A series of smart decisions can lead to compounding effects on the scoreboard.
Nearly every NFL team has at least one staff member crunching the numbers on these crucial situations over the course of a game. League rules prohibit the use of technology in the coach's booth, so fourth-down advisory is limited to printable documents, often abridged to a single card.
Next Gen Stats and Amazon Web Services teamed up to take the analytics on fourth down and two-point conversions to the next level. With the help of tracking technology, fans will have the opportunity of knowing the optimal decision before the play. We call it the Next Gen Stats Decision Guide powered by AWS.
How the Next Gen Stats Decision Guide works
The Next Gen Stats Decision Guide is built on a series of machine-learning models using the Amazon SageMaker platform powering live fourth-down and two-point-conversion decision analytics. For every crucial decision, Next Gen Stats breaks down the numbers in real time. To create a quantitative tool that can aid in live decision support, we first have to break down the driving factors of the decision: win probability (the impact of each decision on the game) and conversion probability (the likelihood of success with each decision).
The first component of the decision equation -- win probability -- helps inform how much the game will change in the hypothetical event of each outcome. In other words ... On fourth down, what are a team's chances of winning the game if it converts a first down, fails to convert, makes a field goal or punts? The second component is conversion probability. A team's chances of converting on fourth-and-2 are significantly higher than fourth-and-10. It's even higher if Patrick Mahomes is your quarterback.
Two separate conversion-probability models were needed to split fourth-down and two-point-conversion plays. Field goal probability, which debuted in 2019, is also factored into the value of a field goal decision.
By combining the likelihood a team converts with the impact of all of the possible outcomes on the game, we can estimate the value of each decision in terms of winning the game.
Colts go for it -- and win!
It's Week 15 of the 2020 season. Indianapolis and Houston are tied 20-20 with 3:21 remaining in the fourth quarter. The Colts face fourth-and-1 at the Texans' 36-yard line. In the thick of the playoff hunt, Indy head coach Frank Reich faces a crucial decision. Should he send out kicker Rodrigo Blankenship to attempt a 54-yard field goal? Or keep the offense on the field and go for the first down to keep the drive alive? It is worth noting that Blankenship made a 53-yard field goal earlier in the game.
Conventional wisdom might suggest to "take the points" and kick the field goal. However, by leveraging learnings from historical outcomes in similar situations, the Next Gen Stats Decision Guide can provide a more informed recommendation. Let's break down the numbers ...
To calculate the chances the Colts gain a first down, we take the live features of the situation as inputs in our fourth-down-conversion probability model. These features include: yards to go (measured using tracking data, in inches), the strength of the offense, the strength of the quarterback and the strength of the defense.
According to our tracking data, the Colts needed approximately 0.25 yards to gain a first down, a clear "fourth-and-inches" scenario. The offense also had an advantage against a Houston defense that finished the season ranked 30th in yards allowed. Altogether, the Colts had an 81 percent chance of gaining a first down in this situation. The odds of Blankenship making a 54-yard field goal? Only 42 percent.
Now we turn to win probability. To estimate the value of each decision, we combine each conversion probability with Indianapolis' win probability for each possible outcome and compare the choices. By leveraging our current win-probability model using hypothetical numbers, we can play the "What if?" game. Below is a breakdown of the estimates needed to effectively analyze the value of each of Reich's choices.
If the Colts go for it, they have a 69 percent chance of winning the game. If they kick a field goal, they have a 56 percent chance of winning the game. The difference between these values is the estimated advantage of the optimal decision. In this case, a 13 percent difference represents a clear "GO" situation for Reich and the Colts. The Next Gen Stats Decision Guide recommends going for it -- and Indianapolis did!
With less than a foot to move the chains, Reich kept the offense on the field. Given Philip Rivers' aversion to quarterback sneaks, Reich subbed backup QB Jacoby Brissett into the game. Brissett and the line picked up 2 yards on a quarterback sneak. Five snaps later, the Colts scored what ended up as the game-winning touchdown.
Precision matters
What separates the Next Gen Stats Decision Guide from previous iterations of a fourth-down calculator is that it uses live tracking data to get the precise location of the ball. When it comes to short-yardage situations, football truly is a game of inches.
As Michael Lopez, director of data and analytics for the NFL, points out in his 2019 paper "Bigger data, better questions, and a return to fourth down behavior," teams were already instinctively accounting for "inches" situations. Using Next Gen Stats tracking data, Lopez found teams that went for it in fourth-and-1 situations were 20 percent closer to the line to gain than teams that did not go for it. Not all fourth-and-1s are the same. Where the ball is spotted has a direct effect on the likelihood of a conversion.
Teams are getting smarter
In recent seasons, teams across the league have made significant strides in their embrace of numbers, using analytics as a tool to guide decision processes across the organization. Recent trends show teams are making the optimal call in fourth-down situations at a significantly higher rate than even a few seasons ago.
League-wide, teams are going for it when the numbers say "GO" at a significantly higher rate of late. When the Next Gen Stats Decision Guide recommended "go for it" by at least 2 percentage points in 2017, the team went for it just 31 percent of the time. Three seasons later, that number increased tremendously, as 53 percent of teams went for it in similar situations during the 2020 campaign.
How about when the numbers recommend punting? Coaches have been much more in line with the data here for years. In each of the last five seasons, when the advantage of punting was two-plus percent, teams made the optimal call 98-plus percent of the time.
When the numbers say go, teams are going for it more. When the numbers say punt, teams have always gotten it right.
Two-point conversions: New trend emerging
The Next Gen Stats Decision Guide doesn't apply to just fourth downs -- it also includes decision recommendations following a touchdown. Should a team kick an extra point ... or go for two? Teams are also getting wiser in point-after situations. This is perhaps most apparent when a team is trailing by 14 points, and then scores a touchdown.
In Week 5 of the 2018 season, the Eagles scored a touchdown to cut their deficit (against the Vikings) to eight early in the fourth quarter. Head coach Doug Pederson made a choice that only one other team had since 2002: The Eagles went for two. And converted! By our model, the Eagles had a 64 percent chance of converting and a 91 percent chance of making the extra point. Going for two presented the team with a +1.3 percent increase in win probability when factoring in the impact of each outcome with the probability of converting. The "go for two when down eight" theory has long been an edge that the analytics community has clamored for, and since Peterson broke the seal, the change in philosophy has gained some momentum around the league. The "go for two when down eight" decision has been made 11 times between the 2017 and 2020 seasons, with the Eagles accounting for three of those moves.
The fusion of data and instincts
Fourth-down and two-point-conversion situations are pivotal responsibilities for head coaches. Any one particular call can be misconstrued when the right process produces the wrong outcome, but the key to long-term success is making smart decision after smart decision. Over the long haul, you're rewarded.
Teams are using analytics as a complementary tool, to enhance decisions typically drawn from instincts and experience. And this tool's not a replacement -- it's a guide.
Check back on Wednesday for our next piece on the best decision-makers in the NFL.
-- Follow Next Gen Stats Analysts Mike Band and Keegan Abdoo on Twitter.