Rainbows and Kaleidoscopes, Oh My!

Colleges are missing out on a lot of students who have potential for creativity, leadership, and wisdom, but who don’t get admitted (at least to the schools they want) because they appear to be weaker than other students. Weaker, that is, on the key consideration for selective schools, namely their academic record.

That’s the argument made by Robert Sternberg in his recent book College Admissions for the 21st Century. Sternberg, a professor of psychology who taught at Yale and served as dean of arts and sciences at Tufts University, writes, “It is time for colleges to use twenty-first century science to identify, select, and nurture tomorrow’s potential leaders for the good of the individuals who apply, the institutions to which they apply, and ultimately, the world.”

That’s quite a claim. If only we could get colleges and universities to abandon their old-fashioned notions about which students to admit and how to teach them, the world would be a much better place.

I’m highly skeptical. Sternberg has, I believe, greatly overrated the ability of admissions officers to identify future leaders and creators—calling it “science” is preposterous—and of college professors to nurture those traits in young people. His book is of a piece with books by educators who envision universities as the drivers of social and economic progress, such as Holden Thorp’s co-authored Engines of Innovation. Higher education is important, but let’s not expect more than it can really deliver.

According to Sternberg, higher education would do more to fulfill its role if it revised the admissions process, which is too heavily weighted in favor of the traditional measures of academic ability. Unlike some people who denounce standardized tests like the SAT, Sternberg merely says that we ought to look beyond them, probing for talents that the SAT (and high school transcripts) may miss—leadership, creativity, practicality, and wisdom.

As he sees things, a lot of the nation’s (and presumably the world’s) problems could be avoided or lessened if our leaders were better educated—if their schools hadn’t put so much emphasis on cognitive skills and subject-matter comprehension, but had worked more on developing their practicality and wisdom. For example, Sternberg observes that although George W. Bush enjoyed a stellar education at Andover, Yale, and Harvard Business School, “the greatest financial crisis since the Great Depression began on his watch.”

The implication is that the financial crisis could have been avoided if only Bush had been differently educated. I’m not a fan of the former president, but that’s poppycock. The financial crisis had its roots in foolish policies by Congress and the Federal Reserve. Even if Bush had been schooled in Austrian economic theory and had been sensitive to the way in which government policies can cause grave economic distortions, he could not have prevented the crisis.

That’s just one of Sternberg’s examples, but the others are equally weak. Ken Lay was well educated in economics, yet he ran Enron into the ground. Dennis Kozlowski, a graduate of Seton Hall, looted Tyco Industries for his personal gain. Sternberg wants us to believe that those and other bad results could have been avoided through better student selection and education that stressed ethical leadership.  I doubt very much that admonitions in college courses to act ethically would have caused Lay and Kozlowski to take different paths.

Let’s get back to the book’s title and examine Sternberg’s ideas on changing college admissions. While at Yale, he developed what he called the Rainbow Project, which he describes as “an attempt to show that one could improve college admissions by using assessments that measure creative and practical, in addition to analytical, skills.” He devised tests meant to evaluate a student’s creativity, such as having the student write a short story based on an unusual title (e.g., “The Octopus’s Sneakers”) and coming up with captions for cartoons.

And to assess practical skills, Sternberg has students do things like coming up with the best route to get from one point on a map to another and choose the best option for solving problems that might confront the typical adolescent.

Based on the results from nearly 800 students at a range of colleges, augmenting SAT scores with “Rainbow” assessments increased the ability of admissions officers to predict college success compared with only using the SAT. That is, Sternberg says, colleges can get a better student body if they assess student creativity and practicality rather than just basing their decisions on SAT scores and high school records. Looking at this from another angle, some applicants who don’t seem to be quite as good as others based solely on academics turn out to be better students because they have latent creativity and practicality.

Assume that Sternberg’s finding is true. Assume that college officials can reliably detect practicality and creativity in applicants and that they use such assessments to admit some students they probably wouldn’t have admitted otherwise. That also means rejecting an equal number of applicants they would otherwise have accepted. Students in the former group get to enroll in a college they probably thought “a stretch,” but simultaneously students in the latter have to settle for a “backup” school.

Therefore, all the use of “Rainbow” assessment accomplishes is a slight redistribution of students. Sternberg is so excited about using the “Rainbow” that he doesn’t seem to recognize that even if he’s right about the ability to detect hidden talent, one college’s presumed gain is another’s loss.

Or maybe he does recognize that, but didn’t want to try explaining why it matters so much if a student attends College A rather than College B. Despite our mania over college rankings, the truth is that there’s no necessary relationship between the supposed prestige of a school and how much a student learns.  Nor is it true that America’s leaders and innovators all come from elite colleges.

After leaving Yale, Sternberg went to Tufts, where he was able to get outside funding for another admissions innovation called Project Kaleidoscope. That entailed a set of additional essays Tufts applicants could choose to do. The essays were designed to “find out about a student’s wisdom.” Students could write on a variety of topics like “The End of MTV” or how they’d advertise a new product. 

Obviously, such questions don’t have right or wrong answers, so how did the admissions people decide which ones showed the most wisdom? Sternberg anticipates that question and says “through well-developed scoring rubrics…we can achieve a high degree of consensus among raters.” Fine, but that hardly proves that these raters are really able to discern the glimmerings of wisdom (or its absence) in the essays of 18-year-olds.

Once again, Sternberg likes the results. For the first class with an academic year behind them (2011), the analysis shows no significant difference between students who were admitted on the strength of their “Kaleidoscope” scores and those who had comparable academic credentials but were judged weaker on “Kaleidoscope.” That result permits Sternberg to pronounce it at least an apparent success since Tufts enrolled more students who seemingly have the traits of leadership, creativity, and wisdom the school wants.

All of this dovetails with an effort by the school to “brand” itself as a school that trains our future leaders.  It is too early to expect evidence that a higher than usual percentage of Tufts graduates will become leaders or innovators, but this strikes me as a case of wishful thinking.

So, is Sternberg’s approach the way for colleges and universities to do admissions in the new century? Tufts is a private school with enough money to afford the added cost of having “raters” read thousands of essays (which might be genuine, or might be written by someone like this) in an effort to plumb for hidden talents. Should most or all other schools follow suit?

Sternberg thinks he is on to a big idea, not merely a new wrinkle at a boutique school. If we don’t drop the old methods of evaluating students, he says we’ll reduce our “chances of creating a more equitable society.”

Adding rainbows and kaleidoscopes at a few small schools probably won’t do any harm, but to employ such procedures on a national scale would be a big waste of resources. All that could accomplish is to shuffle a few students around, as I explained earlier. It won’t create a more equitable society any more than our mania for racial preferences does.

There is an old story about a rooster who has an over-inflated ego because he thinks that his crowing causes the sun to rise. Similarly, there are many higher education leaders who seem to think that the nation’s success or failure depends on what they do. How colleges decide which students to admit doesn’t matter nearly as much as Sternberg and other advocates of manipulating the student body think.