By David Hill
David Hill is an Assistant Editor at Coaching Management.
Coaching Management, 12.10, November 2004, http://www.momentummedia.com/articles/cm/cm1210/runningnumbers.htm
Early in a game during the 2003 season, the Wofford College Terriers faced fourth-and-two near their own 30-yard line. Conventional wisdom made the call a no-brainer: Punt. Yet Head Coach Mike Ayres and his staff opted to try to keep the drive going.
The decision turned out to be a good one. The Terriers got the first down and went on to win the game. As the season progressed, both the bold play calling and winning continued. Without using against-the-book moves throughout the season, Ayres firmly believes the often outsized Terriers would not have made their surprise run to the Southern Conference title and the NCAA Division I-AA semifinals.
As unconventional as Ayres’ decision appears to the tradition-minded coach —why risk giving your opponents such great field position, especially so early in the game?—it may actually have been the statistically correct call. From a pure probability and statistical stand-point, coaches are often overly conservative when it comes to game-time decisions, according to academicians who have studied the subject. Whether it’s a matter of going for it on fourth-and-short at midfield or trying more on-side kicks, coaches may be surprised to find that the road less traveled often leads to more points.
Typically, these scientific examinations compile data and create elaborate mathematical models to predict results in certain situations. The papers are full of advanced mathematical terms and complex formulas, but for all their academic language and high-level math, these explorations formalize what many coaches do naturally when weighing the possible benefits of a decision against the likelihood of a positive outcome. Still, these studies show that when all possible outcomes and ensuing ramifications are considered, coaches may be leaving points on the table by always following conventional wisdom.
No one is suggesting that statistics will supplant coaching experience when it comes to make-or-break decisions on the field. Even the most ardent advocates of high-level probability analysis—the mathematicians and economists—know the numbers won’t ever replace a coach’s experience, game planning, and knowledge of his athletes’ capabilities. Football games are played by imperfect beings who can defy statistical odds, both by completing the most unlikely touchdown pass and fumbling the most routine snap from center.
Although the inherent uncertainty of the football field makes it a far different environment than the sterile one of statistical probability, there can be benefits from looking at the odds. In some cases, examining the numbers behind game-day coaching choices may confirm the wisdom of decisions already being made. In other cases, it may lead to different decisions. In either case, it can leave coaches feeling more confident and well prepared.
Not all statistical analysis of football comes from academia. Chris Meidt, Head Coach at St. Olaf College, has been a college football coach for 10 years, including several years as Offensive Coordinator at Bethel College. He also spent two years as an information technology analyst before entering coaching and has a bachelor’s degree in math and an MBA in information sciences.
By applying his math and business knowledge to football, Meidt has found that there’s a lot to be gained from aggressive, against-the-grain play calling. Two-point conversions and surprise on-side kicks are common practice for his teams.
"One year, I think we scored 52 touchdowns and 54 points after touchdown," Meidt says. "So we were better than perfect, because we went for two a lot and probably faked a kick once a game. Obviously, we made it more than we didn’t."
Meidt views the on-side kick, particularly in unexpected situations, as a statistically justifiable call. "We recover about half of our surprise on-side kicks," he says. "If it’s expected, you’re not going to recover that many."
Instead of looking at the on-side kick as solely an act of last-minute desperation, Meidt prefers to think about the possible outcomes of an unexpected on-side kick and how they compare with the more conventional options. "What is the negative value of a failed surprise on-side kick? Well, you’re giving them great field position," he says. "But what if you kick it deep and they return it to midfield—is that the same result psychologically? I would argue that even if you don’t recover the on-side kick, you create a lot of tension on the opponent."
This way of analyzing decisions is the approach taken by mathematicians and economists who look at football problems. Their probability models on on-side kicks, two-point conversions, and going for it on fourth down at midfield show that unconventional calls are actually more likely to pay off in the long run than is normally thought.
A few coaches are at least investigating the possibilities. When Harold Sackrowitz’s paper in the academic journal Chance titled "Refining the Point(s)-After-Touchdown Scenario" was mentioned in Sports Illustrated and The New York Times, several coaches called him from the collegiate level and the NFL. Sackrowitz—who began examining football decisions from the NFL as example problems for his classes at Rutgers University, where he is a Professor in the Department of Applied and Mathematical Statistics—says he’s seen little evidence of coaches actually following his advice to be more aggressive. But he’s not surprised.
"The things I’ve worked on, like the two-point conversion, can’t possibly be learned just by being involved in football because the situations do not come up often enough," Sackrowitz explains. "How many times has a coach been in a position where, with 13 minutes left in the game he’s behind by 14 after scoring a touchdown and he’s said, ‘Okay, this time I’ll try going for two’ or ‘This time I’ll try going for one’? Those instances are so rare that you can’t learn from them. But people like me try to develop a model to compute the probabilities of all the things that can happen in order to try to decide the best course of action.
"There are all sorts of possibilities of what your opponent can do and what you can do, and nobody, no matter what they say, can get a model that exactly fits what’s going on," he continues. "But the model can approximate it. And if what comes out of the model is an obvious decision, then that’s what you should do."
Sackrowitz developed a chart designed to guide whether to go for two points after a touchdown or kick the extra point that’s more sophisticated than what is typically used on sidelines (see "One or Two" below). He’s also analyzed such decisions as whether to punt or run another offensive play on fourth down at various spots on the field and with differing amounts of time remaining.
Punting on fourth down and choosing the PAT are common areas for statistical study. William Krasker, who analyzes decisions in pro games on his Web site, footballcommentary.com, believes coaches often play it too safe. He bases this assertion on his analysis of the outcomes following certain decisions in actual games.
"Pretty much every time, regardless of the score or time remaining in the game, teams should be going for it on fourth-and-one at midfield, yet very few coaches do," Krasker says. "And most of the time, it makes sense to kick the extra point after a touchdown, but not 99 percent of the time, which is what teams tend to do."
Krasker appreciates coaches’ reluctance to buck conventional wisdom. "These problems are complicated," he says. "You’re talking about a situation in which a coach doesn’t understand how a model like this works and has no way to judge whether it makes any sense. He can be very reluctant, and I understand that.
"Plus, there’s usually not enough incentive to deviate from the conventional wisdom," he continues. "Let’s be honest: Following the results of my model on every play can increase the probability of winning the game by, maybe, five percentage points. But what does that translate into? Maybe one win every year and a half or two years on average. If coaches ignore it, it’s probably not going to have any big effect on them, so why should they stick their necks out?"
Coaches don’t always have an opportunity to let their decisions even out over the long run. Knowing that the decision to go for a first down will pay off seven times out of 10 offers little consolation the three times it doesn’t work.
"One thing I know: These academics are sitting behind desks figuring these statistics out, and they don’t get fired based on whether they make the two-point try," says Bob Johnson, Head Coach at Mission Viejo (Calif.) High School. "Coaches lose their jobs on calls like that, and I’m not sure they want to lay it on some statistical thing that some guy in Berkeley figured out. Coaches know their personnel, they know what’s going on at the time, and the fact is they can get fired over a bad decision. That’ll make you conservative real quick."
In this regard, coaches base many decisions not so much on maximizing potential gains but on avoiding maximum losses. Thus they choose the chip-shot field goal instead of going for a first down in the red zone, or they punt instead of going for it on fourth down at midfield.
To statisticians, these are bad calls because they don’t pay off in the aggregate. But coaches don’t play in the aggregate. They play one game at a time. "Let’s say it’s fourth-and-one on our 49-yard line, and it’s in the first quarter. You punt the ball and play defense," says Cecil Flowe, Head Coach at Parkview (Ga.) High School. "You don’t create a chance for the other team to have the momentum swing because of your stupidity. You punt the ball away, make the other team go the length of the field, and hope you get the ball back in good field position."
This is not to say that all coaches are completely averse to risk. There are times to take chances. The key to recognizing such times is knowing what you and your opponent are capable of and what’s needed in a given situation.
Ayres believes that his team, often outmatched athletically because of a small student body, rigorous academic demands, and a highly competitive conference, has more to gain by taking chances, including unorthodox calls early in a game. "Our philosophy is to go for it on fourth down whenever it’s reasonable," Ayres says. "Sometimes people say our expectation of ‘reasonable’ is a little high, but that’s helped us win."
Ayres hastens to add that players recognize risky play calling when they see it, so the coaching staff helps them understand what the coaches are thinking. "We let them know early on that philosophically we want to be aggressive in the kicking game, on defense, and on offense," he says. "When you explain that’s going to be the mentality, they all understand."
Facing a stronger team, even the conservative Flowe might opt to go for it on fourth down more often, especially after he’s had time to assess how his squad is playing. "After your defense plays three or four sets, you can better judge how they’re going to be able to handle the opposing team," he says. "Then you have some decisions to make when you get a fourth down on the 50-yard line. Sometimes you know, ‘We need to roll the dice right here. We need to go for it.’"
Tactical decision making has room for the math-focused and the math-phobic, the play-it-safers and the gunslingers. A math major in college, Tim McGuire, Head Coach at Indiana State University, says that instead of using true probability analysis, he relies, as many coaches do, on finding trends in opposing teams and designing game plans with that information. Breaking things down to find that in "x" situation, the other team lines up in "y" formation, and runs play "z" 90 percent of the time is the sort of mathematical analysis that can help a coach make tactical decisions.
"The math helps you prepare," says McGuire. "You can’t be haphazard in your thinking. And math creates order in your thinking."
As someone who often takes it to the extreme, Meidt says the real value of probability analysis isn’t always in the actual decisions. The benefit, he says, comes from having thought about what could happen and how likely each outcome might be before the situation arises.
"It really helps if you’ve thought through those decisions before you get to crunch time," he says. "Plus, it’s a blast, especially if it’s something you love doing, like football. It gives you a reason for what you decide. If nothing else, it gives you confidence, which is a big benefit for any coach."
SIDEBAR: One or Two?
If there’s a religious text to be found on football sidelines, it’s the point-after-touchdown chart that coaches at all levels of the game use to decide whether to kick or attempt a two-point conversion. And like most religious texts, some people live by it unquestionably, some ignore it, and some use it in their own way.
"The chart works real well," says Bob Johnson, Head Coach at Mission Viejo (Calif.) High School. "Instead of letting emotion get into it, it tells you to go for one or go for two. I stick with that almost entirely."
But some coaches question the sanctity of the chart. "Every time someone says, ‘Don’t you have the chart?’ I ask, ‘Well, who made the chart? Why should I believe in someone else’s chart?’" says Chris Meidt, Head Coach at St. Olaf College. "My view is the chart is invalid until the final five minutes of the game, and it’s valid only if you’re not going to score again. I can tell if we’re down by five that we need to run or pass to get within three, or if we’re down by nine that we need to kick to get within eight. And if I can go up by two scores, why should I just take the point? I can make my own decisions about that."
Harold Sackrowitz, Professor in the Department of Applied and Mathematical Statistics at Rutgers University, has encountered "the chart," too, especially after he published his own in an academic journal on probability and statistics. Based on the dynamic programming model of probability analysis, his version consists of a pair of grids, one for when a team that has just scored a touchdown is behind on the scoreboard and one for when it’s ahead, with columns and rows representing point differentials.
But his table also accounts for how much time is left, expressed in expected remaining possessions. Each square has either a 1 or a 2 or a blank space, which indicates that his probability model didn’t show a clear-cut advantage to either call.
After Sackrowitz’s chart was referred to in the mainstream sports media, several coaches showed him their own charts. "They all know that early in the game is not the same as late in the game," Sackrowitz says, "so they write little notes to themselves saying, ‘This is a risky situation’ or ‘This is safe.’ They all know that there’s a problem, but I guess they are not able to look at it scientifically.
"I’ve had a number of them tell me that they’re satisfied with what they’ve got," he continues. "And I, of course, think they’re wrong."
Sackrowitz’s article, including his two-point/one-point conversion chart, can be downloaded at www.public.iastate. edu/~chance99/133.sackrowitz.pdf
SIDEBAR: Case Study
When Chris Meidt, Head Coach at St. Olaf College, was Offensive Coordinator at Bethel College, he spent a lot of time analyzing game scenarios, such as when his squad should go for it on fourth down, punt, or attempt a field goal. He used game theory, which, despite the name, is most often used in business as well as military and international-relations decision making. His approach wasn’t surprising, given his academic and work background. Meidt has a bachelor’s degree in math with a business minor, and a master of business administration degree in information sciences, and spent more than two years as an information technology analyst before entering coaching.
"In game theory, you’re trying to take a decision, ask what are the possible results of that decision, and then put a value on each outcome," Meidt says. "For example, if we were to fake a punt from our own 30-yard line, we’d look at the odds of success—is it a good time to do it, are they in the right defense, that kind of thing. Then we look at the value of being successful and what kind of value we would put on not being successful. You have to put all these values together and decide what’s going to give you the optimum value for your decisions. It doesn’t mean you’re always right, but analytically speaking, you’re looking for what’s going to give you the highest total value."
This process should sound similar to coaches who dissect opponents’ game film for tendencies, strengths, and weaknesses. But Meidt goes a step further by putting that information into numbers and equations. As a head coach, he doesn’t have the time to go into as much detail as he did as offensive coordinator, but his general approach remains the same.
Of the two main sets of numbers Meidt establishes, one is the value of each possible outcome, which are largely subjective and rated on a scale of 1 to 10. "If we fake a punt and score a touchdown, we would put a 10 on that. If we don’t get it, I’d put a certain number on that, depending on field position. Or we might get a first down out of the fake but not a touchdown. There are lots of possible outcomes. I’d list all possible outcomes and weight each of them."
The other main component is the likelihood of each possibility. That is less subjective because it’s based on real outcomes in actual games. Meidt uses statistics from NFL games put into spreadsheets.
Obviously there are differences between the pro game and NCAA Division III, where St. Olaf plays, but Meidt is confident that the frequency of situations and their outcomes is transferable, particularly when adjusted for less consistent college kickers and different clock-stoppage rules. If college stats were readily available, Meidt says he’d use them, but they aren’t.
Meidt admits that probability modeling and game theory aren’t for everyone. "You have to enjoy crunching the numbers," he says. "It won’t work for people who are scared of numbers, because you’ve got to be pretty analytical."