U.S. News Ranks Stanford’s EMBA High

The first thing you’ll notice about U.S. News & World Report’s 2010 rating of the best Executive MBA programs is that the magazine ranks Stanford University’s Graduate School of Business 13th. Problem is, Stanford does not have and never has had an Executive MBA program. So how is it possible that U.S. News has Stanford in this specialty ranking?

Chauk it up to how U.S. News creates these specialty rankings. They are based solely on a survey of business school deans and MBA program directors–not even the EMBA directors–who know next to nothing about the Executive MBA landscape. They’re largely ranking programs based on their sense of the schools’ reputations. Stanford obviously has a great reputation as one of the world’s best business schools. So some of the deans jotted down the school when asked to name those with the best EMBA programs.

There’s a common saying you’ll often hear in accounting classes: “Garbage In, Garbage Out.” Sadly, it applies here. Even worse, it’s hardly U.S. News’ first boo-boo. In past surveys, the magazine has listed Princeton, which has no business school at all, and Harvard Business School, which also does not have an EMBA program, in this list.

It’s a major and telling flaw recently highlighted by author Malcolm Gladwell in his scathing New Yorker critique of U.S. News’ rankings. Writes Gladwell about the “Best Colleges” version of the survey: “Every year, the magazine sends a survey to the country’s university and college presidents, provosts, and admissions deans asking them to grade all the schools in their category on a scale of one to five. Those at national universities, for example, are asked to rank all 261 other national universities.” The magazine’s rankings editor, Robert Morse told Gladwell that the typical respondent assigns grades to roughly half of the schools in his or her category.

“But it’s far from clear how any one individual could have insight into that many institutions,” concludes Gladwell. “Sound judgments of educational quality have to be based on specific, hard-to-observe features. But reputational ratings are simply inferences from broad, readily observable features of an institution’s identity, such as its history, its prominence in the media, or the elegance of its architecture. They are prejudices.”

As an example to prove his point, Gladwell cites an analysis of another ranking by U.S. News, its rating of the Best Hospitals which also rely heavily on reputation scores generated by professional peers. “Why, after all, should a gastroenterologist at the Ochsner Medical Center, in New Orleans, have any specific insight into the performance of the gastroenterology department at Mass General, in Boston, or even, for that matter, have anything more than an anecdotal impression of the gastroenterology department down the road at some hospital in Baton Rouge?…

“When U.S. News asks a university president to perform the impossible task of assessing the relative merits of dozens of institutions he knows nothing about, he relies on the only source of detailed information at his disposal that assesses the relative merits of dozens of institutions he knows nothing about: U.S. News. A school like Penn State, then, can do little to improve its position. To go higher than 47th, it needs a better reputation score, and to get a better reputation score it needs to be higher than 47th. The U.S. News ratings are a self-fulfilling prophecy.”

Well, yes and no. Thankfully, there are other rankings of EMBA programs so U.S. News isn’t the only game in town. Gladwell misses another equally important flaw related to surveys filled out by people who have no direct knowledge of what goes on at other schools. If allowed, they are likely to rank their own schools highly, even above schools with far better reputations, faculty and students. Consider U.S. News’ rank of 14th for Marquette University in Milwaukee, Wisconsin, or its rank of 17th for Xavier University’s Executive MBA program in Cincinnati, Ohio. We have nothing against those schools, but need to point out that it’s highly improbable that any credible ranking of EMBA programs would put Marquette and Xavier ahead of Virginia’s Darden School (ranked 18th by U.S. News), Cornell (ranked 20th), or Georgetown (ranked 23rd). It’s just not believable.

That said, this ranking is in the public domain. People turn to it on a fairly regular basis, and given U.S. News’ own reputation as a quality publication, the ranking is viewed with some measure of authority. And while it’s embarrassing for U.S. News to highly rank an EMBA program that does not even exist, the survey at least provides the opinions of the deans and MBA program directors–whatever that may be worth and however dishonest some of them may be.

One other caveat to consider: Many of these ranked schools have several EMBA programs, often in partnership with other schools in Europe and Asia. Given the limits of U.S. News’ survey, only schools are rated here–not the different Executive MBA programs any school may offer in its portfolio of programs. And if you go to U.S. News’ website to pull up these rankings, you’ll also find another rather egregious error. The data accompanying its EMBA ranking is not for the Executive MBA programs but rather for the two-year, full-time MBA programs at the schools ranked.

U.S. News, of course, isn’t the only rankings player to survey B-school officials for their views on Executive MBA programs. BusinessWeek also does a similiar survey, but at least BW insures that only EMBA directors are polled. So we show the U.S. News ranking in comparison to the BW survey of EMBA directors.

2010 U.S. NEWS RANKING OF EXECUTIVE MBA PROGRAMS AND HOW IT COMPARES TO BUSINESSWEEK’S POLL OF EMBA DIRECTORS.