Measuring Success

A national championship is one sport may not reflect the overall strength of a school’s athletic program. That’s why many athletic directors are placing increased emphasis on raising their school’s NACDA Cup rankings.

By David Hill

David Hill is an Assistant Editor at Athletic Management.

Athletic Management, 15.4, June/July 2003,

After the Kutztown University men’s indoor track and field squad earned its highest-ever finish in the NCAA Division II national championships this past March, its coach surprised Director of Athletics Clark Yeager. “He came back and said, ‘Tenth place—that’ll get us some Sears Cup points,’” Yeager says.

This exchange shows the increasing awareness of the national all-sports trophy after 10 years. The NACDA Directors’ Cup, formerly named the Sears Cup after its original corporate sponsor, has become as much a part of intercollegiate athletics as homecoming games and awards banquets.

To be sure, the importance of the Cup varies widely from campus to campus. But most of today’s athletic directors do keep track of the standings and have an opinion about the significance of the results.

“There are not many rankings that relate to the overall success of an athletic program,” says Arizona State University Director of Athletics Gene Smith. “We want to be a part of them, not unlike the academic units on campus, which strive to be recognized as among the best in the nation in their particular field. We want to do the same.”

“It is a benchmark,” says University at Albany Director of Athletics Lee McElroy, Chair of the National Association of Collegiate Directors of Athletics (NACDA) Directors’ Cup Committee. “Is it the appropriate benchmark? That depends, you know, on the eyes of the beholder.”

The importance of the Cup depends in part on how well a school scores in the annual rankings and how realistic it is to strive for higher finishes. It also depends on the aspirations and resources each institution attaches to its athletics program.

A handful of schools have made winning the cup a regular event: Stanford University, the University of California at Davis, Williams College, and Simon Fraser University are prime examples. But finishing first isn’t the only goal. Many athletic directors see where they finish relative to peer institutions as a benchmark of success and a higher rank than last year as something to strive for.

The College of William and Mary goes into each year thinking it can finish among the top 100 Division I programs, says Director of Athletics Terry Driscoll. The program has stood as high as ninth after the fall rankings, before finishing 42nd that year, Driscoll says.

“For us it’s a great measure because our goal is for each sport and each student-athlete to have the best experience they can have,” Driscoll says. “For most of our teams, the goal is to make the NCAA tournament or championship in their sport [which earns the school 5 to 48 points, depending on the size of the postseason bracket and the team’s seeding]. It also gives us a barometer to judge ourselves versus other institutions.”

Some athletics programs have set firm targets for themselves in the Cup’s annual competition. For example, Kutztown has made breaking into the Division II top 20 while consistently placing in the top 50 one of its goals for 2001-2005.

“In 1996, we were 25th, so the top 20 was a reasonable target,” Yeager says. “This represents the upper 10 percent of Division II. It didn’t seem realistic to be in the top 20 all the time, but being in the top 50 is, and that puts us in the upper realm of the division.”

The Cup especially benefits a school like Kutztown, Yeager says, because while it’s competitive in its conference and region, its teams are not perennial Division II title contenders. The Cup creates an arena in which Kutztown can have a national presence.

“We have goals within our conference and region,” Yeager says, “so it seems reasonable to have national goals. It shows the rest of the university that we desire a national presence and allows us to compare ourselves as a department, not just individual teams.

“Even though we’ll never compete against Davis or North Dakota [in certain sports], we do compete against them in this big-picture way.”

St. Mary’s College of California is considering making a specific Division I Directors’ Cup finish a target to include in its athletics strategic plan, Director of Athletics and Recreational Sports Carl Clapp says. In determining a numerical goal, it’s taking into account its past finishes—No. 219 in 2002—and that of comparable schools.

Clapp says Cup standings are definitely watched and discussed when NACDA releases them at the end of each sports season. “It’s really a topic of conversation among the staff,” he says.

At Grand Valley State University, Athletics Director Tim Selgo does not include a specific Cup ranking among the department’s goals, though it does seek to win its conference all-sports trophy. But its second-place finish in 2002 made the central administration more aware of the NACDA Directors’ Cup program. Because of that, Cup points are in the back of the program’s collective mind during planning and budgeting—albeit in an indirect way.

“We’re more focused on trying to help certain sports be more competitive,” Selgo says. “We don’t specifically think, ‘Gee that could help us with our NACDA Directors’ Cup standings,’ but we certainly are aware that the end result can be a better standing in the all-sports competition.”

McElroy says that Selgo’s approach espouses the true meaning of the Cup. “I tell athletic directors and presidents this all the time,” McElroy says, “‘It’s not the scoring that counts. It’s really the investment in your program, the broad-based quality of your program, and where you invest your resources. If you choose to invest your resources in one or two programs, you aren’t going to get results in the Cup. If you invest your resources across the board, in all of your program, you’re going to get some results.’”

Smith concurs. “There are very few people out there thinking, ‘I’ve got to win the Directors’ Cup,’” he says. “Instead, it’s, ‘We’ve got to create an environment where we’re putting teams in position to win a championship.’”

There is some concern that the Cup creates an unnecessary arena of competition and adds pressure to an already difficult endeavor. McElroy acknowledges that some athletic directors have financial incentives in their employment contracts based on year-end standings. At Arizona State, Smith says earning a certain number of Cup points can result in financial bonuses for coaches of some sports.

“The benefit of the Cup program is enhancing the promotion of a well-rounded program,” says Old Dominion University Athletics Director James Jarrett. “However, I believe it puts pressure on programs and coaches that can create more problems than positive public relations benefits justify.”

McElroy admits Cup rankings can create pressure, at least at certain levels of competition. But this pressure can be a good thing, he says, if it’s channeled toward creating and maintaining well-rounded athletics departments. He points out that the Cup has promoted gender equity because women’s successes count equally to men’s in the scoring system. For example, the University of South Carolina had its highest-ever finish in the Cup in 2002, thanks in part to 100 points from the women’s outdoor track and field squad’s national championship, 80 points from the indoor squad’s fourth-place finish, and 60 points from the women’s basketball team’s deep run into the NCAA tournament.

Clapp agrees that the competition can be a positive. “I don’t worry about the pressure,” he says. “In Division I athletics, you’re in a very competitive environment, and the people who are in here thrive on it. They enjoy the opportunity to put their best efforts up against other people’s best.

“When you think about what we’re doing with young people,” Clapp continues, “we’re teaching them how to respond to success, and how to respond when you fail. I think competition is a good thing.”

“The reason we’re under pressure in my view is that we may have done too good of a job of presenting the Director’s Cup as a symbol of a broad-based successful program,” McElroy says. “For example, my president has been broached by other presidents with questions about the Cup standings. Things like, ‘Why wasn’t my school in the top 25?’ and, ‘We finished ninth a year ago and we finished 15th this year. Are we less strong? Do we have less quality, less value, in our program?’ To me, these questions bring about important conversations, which are a good thing.”

McElroy suggests that the Cup represents a chance to explain what’s going on in all the sports from year to year. “All they see is that you finished at a, b, or c, and you’re either better or not as good,” McElroy says. “They don’t understand what goes into it, how you build a program, which programs are represented, why you’re more competitive one year than another, and so forth. But those are all good questions that any athletic program should have the answers to.”

Another way to handle any pressure the Cup might create is to maintain perspective. Quinnipiac University Director of Athletics and Recreation Jack McDonald says he doesn’t begrudge his counterparts at other schools who play up their Cup standings. But Quinnipiac prefers to strive for improvement in particular sports—such as its men’s and women’s basketball programs’ RPI rankings—and to focus on improving its conference standings.

“With 320 Division I programs, many of them having been around for 100 years, it’s just not realistic for Quinnipiac to finish near the top, not when we’ve been in Division I just five years,” says McDonald. “For Quinnipiac, it’s not an evaluation tool at all. There are far more institutions that don’t make the NACDA top tier than do.”

For those athletic directors who do choose to pay attention to the standings, a high ranking can be more than a benchmark. Many athletic directors have turned it into a source of motivation for their athletes and coaches and goodwill for alumni, fans, and donors.

“Athletic directors, campus presidents, and boosters are increasingly viewing this ranking as an important measure of success,” says Deborah Yow, University of Maryland Director of Athletics. Yow tells boosters and others with an interest in Terrapin athletics that if more women’s scholarship teams can be added, the school can then add scholarships to men’s sports that do not now get their full complement—thus making those squads more competitive and more likely to earn Cup points that come with making the NCAA postseason.

The University of Mary, which led the NAIA division after the 2002 fall season, makes remaining in the top 15 a goal that inspires coaches and athletes, says Athletics Director Al Bortke. Cup standings are discussed in staff meetings, and coaches make their athletes aware of the points they can earn the school. “It involves a team effort for athletes, and it spreads to the other programs,” Bortke says. “Athletes know a good effort will keep us there.”

At Grand Valley State, Cup points and rankings are brought up formally to athletes at the start of the academic year’s all-sports meeting. “Our administrators and coaches, and most importantly our student-athletes, take a lot of pride in knowing that we can compete at the national level in a lot of different sports,” Selgo says.

Yeager says many Kutztown athletes get excited about the Cup rankings because, he suspects, it’s another way they can feel part of a larger effort. “They like to hear they’re contributing to obtaining national recognition,” Yeager says. “After the summer, some of the kids come back and say, ‘How’d we do in the Sears Cup?’”

The buzz needn’t stop at the edge of campus. A good Cup performance also helps recruiting. Bortke says most recruits learn of Mary’s rankings through the media, but he tells coaches to talk it up in case they haven’t.
While some schools receive publicity only after standings come out, others add to the buzz throughout the year. “We send out press releases on it, and we make it part of the information we send to alumni and donors,” says Driscoll.

Yeager says Kutztown includes its final ranking and those of the top Division II schools nationally as part of its year-end compilation of athletics events, and in a quarterly newsletter sent to supporters. He supplies more detailed information directly to interested university administrators.

“They use it in many different university publications,” Yeager says. “If you’re sending information to your alumni, it doesn’t hurt to say ‘We were in the top 20 of all Division II schools in the country.’ Especially since schools like ours tend not to be able to say things like ‘Our business school is in the top 10.’

“We advertise it whenever we have an opportunity,” he continues. “And we do it particularly for our track and swimming teams—teams that don’t get a lot of publicity for what they do.”

If there’s a constant in the 10 years of the Directors’ Cup, it’s that the scoring system has always been changing. Chances are that will continue. The latest fine-tuning limited the number of sports that can earn points by eliminating those with particularly small championship fields. It also slightly de-emphasized individual sports. (See “Scoring Changes,” below). The rules for 2002-03 have also limited the number of sports for which schools may get credit: 10 each for men and women in NCAA Division I, seven in Division II, nine in Division III, and six in the NAIA.

While understanding the committee’s desires to cut out sports sponsored by comparatively few schools, some athletic directors say schools should get credit for having plenty of sports. “If you’re willing to spend your money on lots of sports, you ought to get credit for it,” says Yeager. “We have 21 Division II sports. Why can’t we score in all of them?”

Small-field sports were eliminated because, with so few teams competing, chances of earning Cup points are significantly greater than in more broadly played sports, and the committee thought it fair to make that distinction, says McElroy.

Another change some athletic directors would like to see is a way to account for differences in resources. Ideas include awarding points for a high championships-per-dollar ratio and breaking the divisions into sub-divisions or classes by the size of their budgets.

For example, suggests Driscoll, take two schools with 18 sports each, but one spends a third more money. The school that spends less but makes as many championship fields would be outperforming the other. “There should be some consideration for how much they do with the resources they have,” Driscoll says.

McDonald wishes there were a relative-strength system for rating entire athletic departments’ schedules, something analogous to the RPI standings in basketball and Sagarin ratings in football. “You pick up Thursday’s USA Today and the men’s basketball teams are ranked by their RPI, 1 to 328,” he says. “It’d be nice to pick up Friday’s paper and see the athletic departments ranked 1 to 328. I wouldn’t mind seeing NACDA and USA Today print that.”

Many athletic directors would like to work academics into the scoring. This is especially important as the NCAA, NAIA, many faculty members, and others call for re-emphasizing academics in athletics. “As you see the NCAA moving to recognize institutions for academic excellence as part of their academic reform package, I think you will see the Director’s Cup respond to the environment in the athletic enterprise,” McElroy says. “The way the NCAA is headed right now, I could see academics at some point in the future being a component in the NACDA Cup.”

Bortke says attempting to tie academics to Directors’ Cup points would generate pressure on all sides, but he’d still be for it. “I would certainly go along with that,” he says. “We’re extremely proud of the amount of scholar-athletes we have.”

Not everyone is sold on tying academics to athletics in this way, though. Selgo says including academics in the points system would be productive, but the problem is measuring classroom performance. Schools have different missions, and a 3.0 GPA at one institution isn’t equal to a 3.0 GPA at another. “I think there’s nothing wrong with an athletics-only competition,” Selgo says. “That’s why we have athletics, after all, for the competition.”

Some have suggested awarding points for having Academic All-Americans. But the thresholds for these honors vary, says McDonald, who nonetheless endorses the principle of an academic tie-in.

“We’re still a couple of years away from finding the right instrument or tool, but academic success should be a measure of athletic success,” McDonald says. “Coaches promise an education to all their recruits. We have to be accountable.”

Smith agrees it’s hard to measure academics, yet also believes it should be tied in anyway. “We do not have any way to evaluate departments’ overall academic performances,” he says. “We talk about graduation rates, and that’s good, but are there other things? It could be GPA and a different graduation rate than the NCAA graduation rate, maybe the four-year graduation rate rather than the six-year graduation rate.”

Smith suggests points could be awarded for having a collective athletes’ GPA higher than that of the institution as a whole. Questions of equity would be raised, however, because some schools are seen as harder than others, and certain fields are harder than others at the same school. “That’s why it’s never been added,” Smith says. “But I think eventually some of the objections have to be overlooked because we’re not just about the competition. We’re about academics as well.”

Even with these suggestions, athletic directors aren’t exactly begging NACDA and McElroy’s committee for changes. “One of the goals of the system is to keep it simple,” says Driscoll, “and if it got more complicated, there’d be more room for people to take issue with the scoring than they do now.”

And the Directors’ Cup is not the only all-around ranking out there. Yow is proud that Sports Illustrated for Women put Maryland in its list of the best universities for female athletes, and notes that some rankings already take a more broad measure than the Cup. “The U.S. News and World Report annual ranking is in some ways more significant because it takes into consideration important factors such as academic success of the student-athletes and equity,” Yow says. “I believe that the U.S. News annual ranking will become increasingly important to athletic departments and their institutions over time.”

“The Cup is a great symbolic award,” says Smith, “but from a daily basis, or from a long-range planning point of view, athletic directors don’t sit down and say, ‘I’m going to shift my priorities to win the Directors’ Cup.’ There are so many other things that you’re judged by.”

And, as Yeager observes, perhaps the most important factor can never be judged by a system of points, rankings, or ratings. “The real value of sports at colleges like ours is the experience kids have, and those experiences can’t be measured in terms of national success,” he says. “The value is actually in going to practice every day and working with teammates to do the best you can. There can only be one champion, and that’s not an absolute measure of quality.”

Sidebar: Scoring Changes
The most recent changes in the NACDA Directors’ Cup scoring system recognize two facts: It’s easier to score well in sports played by relatively few schools, and just a couple of outstanding athletes in certain sports can earn schools lots of points.

Previously, points were awarded in all sports that had NCAA- or NAIA-sponsored championships, but starting for 2002-03 there must be at least 12 teams in the championship field. This eliminated rifle, men’s and women’s water polo, and men’s volleyball in the combined NCAA all-division championships. In Division I, it eliminated women’s ice hockey; in Division II, field hockey, men’s and women’s lacrosse and women’s rowing; and in Division III, women’s golf and rowing and women’s and men’s ice hockey. No NAIA sports were eliminated from the scoring.

At the same time, however, the minimum number of points awarded for making a championship field went from 20 to 25 in sports with team brackets. In sports scored by the results of individual athletes, such as wrestling, track and field and gymnastics, schools will get at least five points for making a championship field even if the field is larger than the previous limit of 64 teams. (In individual sports, points are awarded according to schools’ cumulative team scores.) Also, schools will receive points only for the higher of their team’s indoor or outdoor track and field finishes, instead of both, as was done in 2002 in Division I.

“We wanted to make sure that there was a balance and fairness in the scoring system between the individual sports and the team sports,” says Lee McElroy, Chair of the NACDA Directors’ Cup Committee and Director of Athletics at the University at Albany. “In some of the individual or Olympic sports, one or two individuals can make such a significant contribution that it can skew the numbers. A great sprinter or a jumper can win two or three events and put you in a pretty good position to win in track and field. Winning a championship in football or basketball is a lot different. So, after the tweaking you will see more balance between the team sports and individual sports.”