An Economist Ranking That Lacks Credibility

METHODOLOGY

The Economist’s ranking is based on findings from two questionnaires, which were distributed during the spring. One was completed by schools, which included quantitative data such as the work and managerial experience of students, gender balance, faculty Ph.D. attainment, salary increases, and overseas projects. The other was a satisfaction survey that targeted students who had graduated within the past three years. Here, respondents rated their schools in areas such as the quality of classmates, faculty, facilities, curriculum, and the alumni network.

In its methodology, The Economist gives an equal 50 percent weight to the career development and the personal development and educational experience categories. Four subcategories – student quality, student diversity, faculty quality, and program quality – are each worth a 12.5 percent weight. Conversely, salary data, such as earnings at graduation and within a year or two of graduation – carries a 27.5 percent weight. At the same time, career progression (goal attainment and promotions) and networking comprised 15 and 7.5 percent of the rank, respectively. The Economist also integrated 2013 results into the formula, giving it a 30 percent weight against 2015 data (70 percent).

While most of these measures appear relatively solid, it’s rather peculiar that the results lack much credibility. Our guess is that students who fill out the survey know it will be used for a ranking so their answers are not entirely truthful. Therefore, the results are more indicative of cheerleading than honest perspective. As a result, schools with brands that have something more to prove may fare better in a student survey than schools that are more solidly established and confident of their superior quality.

As wacky as the results were, an observer can only imagine how wackier they would have been if The Economist hadn’t thrown its previous survey into the mix with the 30 percent weight. That’s a clue that the latest results were all over the place. There’s no other reason to add a previously published ranking into the latest results except to attempt to better stabilize the findings–which as it turns out weren’t all that stable even then. It’s also important to note that if highly ranked schools decline to cooperate with a ranking from a publication as prestigious as The Economist, it’s a sure sign they don’t respect the methodology or the outcomes.

Overall, The Economist received 7,000 student responses to this year’s survey, down from 8,400 responses in 2013. In addition, The Economist seemingly played fast-and-loose with the definition of an EMBA program. In its methodology, the publication pointed out that they “allowed the schools themselves to classify their programmes,” provided they were “part-time and enrolled students with significantly more work experience than those on their full-time MBAs.”

DON’T MISS: THE TOP 10 EXECUTIVE MBA STORIES OF 2014

How the New Economist EMBA Ranking Stacks Up

Sources: The Economist, P&Q and the Financial Times