Completely Serious: Let’s Drop the GRE

Imagine you were interviewing for a job.* The interviewer shuffles through your application package one last time and then looks across the desk to say, “Your references are all top notch, you’ve got the experience we’re looking for, and so I’d really love to offer you a position, but before I can, I need you to memorize the following facts and pay someone $160 to verify that you have. Ready? Here they are: 1) 2 is the smallest prime number and the only one that’s even; 2) x% of y is the same as y% of x.” Confused, but game, you ask, “Is that all?” and are told, “Almost. I also need you to 3) learn how to make a factor tree and 4) memorize the decimal and percent equivalents of the simple fractions from 1/2 to 1/10.”

Sounds crazy, right? Yet that is precisely what graduate admissions committees tell thousands of applicants each year. Granted, they don’t put it in those exact words. Instead, they’ll note the surprisingly robust applicant pool** and the difficult time they had arriving at a decision, and they’ll wish the rejected all the best in their further endeavors, but what they’re avoiding saying outright is this: your application was cut in the early stages of our review because your GRE score was about 100 points below our school’s acceptable range.

Most schools don’t realize, of course, that the difference between a GRE score of 1000 and one of 1100*** boils down to acquiring some passing familiarity with those four simple facts. I only know it because I’ve worked (on the side) in the test prep industry for about eight years now, and in that time I’ve raised a lot of 1000’s to 1100’s, and while there are many roads to a 100 point increase, the road most traveled is paved with some version of these four. Some students travel the road alone, navigating it with $30 books bought at Barnes and Noble, others pay up to $120 an hour for someone like me to guide them down it.**** Those without the cash to spend on prep courses and materials can sometimes find their way down it, too, but that way usually requires a lot more work.

Obviously, most grad programs don’t care if their students are dab hands at calculating percentages without calculators, and why would they? But they care about GRE scores, even though they say they don’t. Your GRE score is always one of the first items on the application. Schools make decisions based on a set Mendoza line that varies from school to school, even though they say they don’t.

So why do they? Likely it’s due to a vague institutional memory of the time when U.S. News and World Report factored the average GRE score of the admitted students into the arcane proprietary formula they use to make their rankings. They haven’t been using GRE scores for some time now, though it might be hard to remember because undergraduate, business school and law school rankings all still do factor in test scores.

Luckily, the people who make the test have given us the perfect excuse for dropping the test sooner rather than later.  Next year, the GRE is changing. Beginning in Fall 2011, the testmakers are 1) dropping antonyms and analogies from the verbal section in favor of more reading comprehension, 2) scaling back geometry in the math section in favor of more data analysis (charts and graphs) questions, and 3) restructuring the score so that math and verbal are combined into one score pegged to a 130-170 range.  These changes might seem just like a minor superficial calibration, but they actually matter quite a lot.  They make an already suspect test even worse. Allow me explain.

Notice that there were no verbal tips in my 100 point facts. That’s because the verbal section–as it’s currently administered–is a lot harder to quickly prep a student to take. As a glance at the official scoring scale will show, the scores on the verbal section are lower than those in the math across the board. The top math score (800 on a scale of 200 to 800) puts you in the 92nd percentile of test takers. To be in the 92nd percentile in verbal, you only need a 650 (on the same scale of 200 to 800), and at a 740 verbal you’re in the 99th.

There are a couple of quick point boosters in the verbal section to be sure. Students usually don’t know that the analogies the testmaker is looking for must be definitional, for one. PERNICIOUS:LIE will never be found in a question on the test nor in the correct answer to a question, because lies are not always pernicious and not all pernicious things are lies. X is sometimes Y, often Y, occasionally Y, usually found with Y, and so on are all incorrect by the GRE’s metric–a metric they never explicitly state in the directions to the test.***** Likewise, students don’t know that when the testmaker uses the word “infer” they’re using it in the strictest sense, not to mean “something that’s reasonable and likely given the provided information” but rather “something that absolutely must be true given the provided information.”******

As it stands now, these verbal point boosters aren’t nearly as useful as the math ones because the verbal section includes one further challenge that can’t be easily coached away: vocabulary. Knowing how analogies work according to the testmaker is a useful thing, but even then you do still need to know the definitions of the twelve words used in the question and answers. I can tell you what the top 100 words used over the course of the GRE are, but there are still enough obscure words in the English language that memorizing those 100 is of but small help.

Now some of you might protest that knowing the definitions of any particular set of less-frequently encountered words is as irrelevant to graduate work as knowing how to calculate percentages in your head. You might also continue by pointing out that words like mercurial, aver, inveterate, eschew, restive, inchoate, or stentorian are cool and all, but knowledge or ignorance of them is likely not the best indicator of future success in graduate school. And you’d be right. A robust vocabulary is neither sufficient nor necessary for success in graduate school. But, having one tends to correlate with reading widely and attentively, and being well read is a very good indicator of graduate school success. By dropping analogies and antonyms from the test, you lose that (admittedly imperfect) correlation with an actually useful marker.

The testmakers would likely contend that skill at reading comprehension is far more important to graduate work, and they’d be right. Problem is, their version of reading comprehension actively discourages the sort of reading that you do in grad school. Take this sample paragraph from the official website of the testmaker:

Picture-taking is a technique both for annexing the
objective world and for expressing the singular self.
Photographs depict objective realities that already exist,
though only the camera can disclose them. And they
depict an individual photographer’s temperament, dis-
covering itself through the camera’s cropping of reality.
That is, photography has two antithetical ideals: in the
first, photography is about the world, and the photogra-
pher is a mere observer who counts for little; but in the
second, photography is the instrument of intrepid,
questing subjectivity and the photographer is all.

If you were to read something like this in graduate school, one would hope you’d take a moment to consider what the author means by objective and subjective and whether the author’s definitions line up with the other literature you’ve read.  You might take issue with the use of the word “mere” to describe observers and “intrepid” and “questing” for the artist or at least consider what sorts of unconscious biases the author might be revealing through using them.  Perhaps you’d even begin preparing counterarguments for the class discussion, because surely nothing good could build off of such a simplistic dichotomy.

On the test, those thoughts would actively interfere with your ability to score points.  In other words, thinking like a grad student would make it harder for you to get into grad school.  As your expensive GRE tutor will tell you, it’s important to completely ignore the question of whether the author is correct or not, to isolate the passages from any context, really, and never to consider whether the author is being fair or biased.  Sadly, it’s usually the best students, the ones I’d most want to have in a grad class, that have the hardest time turning off the inquisitive, critical, and discerning parts of their brains in order to answer the sorts of questions the GRE asks.  Come Fall 2011, they’re going to have a rougher time of it.

Losing geometry in favor of “data analysis” might also seem like a good change to the test.  Few really need to know the characteristics of isosceles right triangles in grad school, but surely the ability to understand and manipulate data is important, right?  Certainly.  But the way the testmakers tend to test your ability at data analysis is by asking simple questions in bizarre ways.  They don’t care if you know in what sorts of situations a median measure is superior to an average, for instance, or whether a certain sample size is large enough to make claims about or not.  Instead, they’re just going to try to trip you up by writing the scale of one chart in centisomethings and another in decisomethings and hoping that you don’t notice.

And finally, if nothing else I’ve said above has convinced you that this test measures neither jack nor squat, the change to the scoring system ought to.  It’s not that there’s something inherently superior about a test that scores you between 400-1600 in ten point increments over one that operates on a 130-170 scale of one point increments.  Rather, it’s that both scales are designed primarily to obscure what they’re actually reporting.  Is a 150 a good score or a bad one?  And how much better is it than a 145?  And how does it compare to a 760 on the old test?   Why switch from a system with 141 distinct scores to one with 41? Beats me, but that’s the point.  It sounds more sciencey and legit when the system is opaque and off-putting, so they go out of their way to make it harder to understand.   Wouldn’t someone who cared about being understood go out of their way to use a familiar scale–say, 1-100?

If you’re wondering why they’re making all these changes in the first place, it’s probably because the GRE and the GMAT (the test used by business schools) are no longer administered by the same company.  ETS (the company that puts out the GRE) is trying to horn in on the GMAT’s market by making their test more palatable to business school admissions boards.******* I say, let the business schools have it.

P.S.  If anyone’s wondering it there’s some sour grapes behind this unprompted screed, there’s not.  Indeed, I’m one of those people who’s naturally good at tests.  My scores rocked and with them I got into the best program around.  And, as I said, I make a fair amount teaching people how to game the test.  It’s not fear or disappointment that has bred my contempt, but familiarity.  Since I know that some people who actually have some input in admissions read this blog, I figure it can’t hurt to bring the subject up, especially since we’re pretty much done with the admission and recruiting season for this year.

P.P.S. Also, if you’re worried that I might now be in trouble with my corporate overlords, don’t be.  No one should take my comments above as a signal that you shouldn’t take a prep class for the GRE if you’re considering grad school.  Indeed, knowing how easily gameable it really is should send you Kaplan- or Princeton Review-ward that much quicker.  Are their courses really worth $1100?  Absolutely.  Consider the difference in your job prospects graduating from a top 100 school vs a top 25, or a top 25 vs a top 10.  Consider the different financial aid packages an extra hundred or so points might put you in line for.  Grad school is a freaking expensive gamble. The cost of a prep class, a tutor, or some books is a drop in the bucket.

*In this economic climate that might take a bit more imaginative oomph than normally, but play along.
**Year in, year out, the pool is always surprising.  Either we need committee members with longer memories or the pool must grow exponentially better each application cycle.
***And in the lower range, the difference between, say, a 600 and an 850, because the scale is not linear.
****Not what I personally charge, but rather my price plus the markup tacked on by the test prep company that employs me.
*****Instead, they bury this distinction in the secondary materials that accompany your enrollment ticket and in the preparation section of their website.
******If after reading that last sentence you patted yourself on the back and said, “duh, of course infer doesn’t mean ‘what’s likely’!” keep in mind that the most common way the testmaker phrases inference-based questions is by asking “Which of the following does the author most LIKELY believe…”
*******How exactly this appeals to business schools I don’t know, but that is the motive.

Comments on this entry are closed.

  • Matt B.

    For my fellow international readers here's the definition of GRE courtesy of Wikipedia: "The Graduate Record Examination or GRE is a commercially-run standardized test that is an admission requirement for many graduate schools, in the United States,and in other English-speaking countries."
    (not that I didn't infer as much from your post…)

    Having heard of that test for the first time today I agree that it should be abolished mostly for two reasons:

    1. It's run by a private company. Shouldn't something so important be in the hands of the federal or at least the State governement?

    2. Why is there a test at all? Shouldn't the applicant already have proven that they have all the necessary skills for their chosen area of study at that stage?

  • pilgrimchick

    I hated the GRE. I would think that by the time you opt to enter graduate school, you've chosen a field of study of some kind. If that field of study is entirely unrelated to either math or language, what's the point?

  • Whyte Fairy

    amen! Also, something I learned: English Universities don't ask for your GRE scores. Also, they (mostly) don't have application fees (unless you're Oxford. But if you're Oxford, you could probably ask for left ears to be sent in with applications and people would do it.)

  • Got Medieval

    Sorry, Matt. And the rest of you. Sometimes I forget the Internet can reach the far flung reaches of not-America.

    But even so, it's kind of weird how you not-Americans can still read American. None of us can read European.

  • Another Damned Medievalist

    I only applied to grad schools that said they didn't need the subject test (mine was about 8 questions on pre-modern Europe — the rest was all US and post-1648 Europe) or math scores. Because my GRE scores? 700 verbal, 740 analytical, 550 math.

    Um … yeah. Last math class in 1978. GRE in 1986.

  • Janice

    Like ADM, I didn't apply to any school that asked for the subject test which I refused to take on the grounds that American history was a subject I'd successfully avoided my entire university career for a good reason. As you've explained, even the general tests in verbal and analytic skills (areas which we value most in the humanities and social sciences) aren't in line with what graduate school actually demands!

    The GRE is a poor predictor of grad school success but an easy tool for admissions committees to use to make the cut on large applicant pools. That's the only reason why it endures.

    We don't use the GRE or anything like it in our M.A. program. We do have the luxury of being a smaller university where applications may hit the dozen or two dozen mark in a busy year. But even so, I'd encourage universities to ditch the test and create more useful applications based on their own tracking of successful and unsuccessful student skills and achievements prior to admission.

  • theswain

    Amen, brother! I took the GRE the first time in 1986 (ADM, are we the same age?)–I don't remember my scores, but I do remember thinking it wasn't a big deal and the only problem was the length…6 hours in a room in an uncomfy chair and no amenities. But I was young then.

    I took the GRE the second time a dozen years later. Now it was on computer and took half the time! I did quite well on the analytical and vocabulary but abysmal on the Math since it had been 20+ years since my last math class and in real life one really only needs the basics of addition, subtraction, multiplication, and division–few of those questions on the GRE.

    Fortunately my Master's program didn't care about my math score or at that time about the GRE score, though other places I applied obviously did care.

    But that brings me to the area tests, in English that I took twice. The first time, there was 1 medieval question, 4 Shakespeare and nearly all the rest was modern theory questions. I didn't read modern theory, and while I've read more now, I'd still do badly on the test. The second was better but not by that much: there was one question on Beowulf, one on Chaucer, 6 on Shakespeare, 2 on Dickens, a few other literature ones that I didn't know, but again still heavily weighted to the modern theory end.

    Now I have no problems with theory. But the skills and knowledge that I have built that make me a pretty good medievalist are simply not the skills needed for the GRE…the test has nothing to do or predict about me or my success in the field. Abolish it. It's ridiculous.

  • a stitch in time

    Wow. Now I'm even happier that I live in good old Germany, where you might have other problems with getting into the top tier jobs (not that this would apply to archaeologists anyway), but there's no stupid additional test to prove nothing really significant for your studies but will determine a lot. Hey, they started changing the final exams for the phd years ago already, putting much more emphasis on the work you did as phd than on re-learning and then re-telling the stuff that you had contact with in your studying time, but years ago!
    Oh, and what happens to non-native speakers on the GRE test? Are they just out of luck if they don't grasp the verbal things engineered for native speakers?

    (Word verification: depro. How fitting.)

  • lilburne

    OTOH one needs to have some way of differentiating one from another. When I was selecting recent graduates for job recruitment one would get 20-30 applications and resumes a day. All of them were the same especially if they had come from some agency. Same school and college experience, all the boxes ticked, but with the best will in the world one isn't going to interview 20 a day. So you needed to toss a coin or find some other differentiating marker. In our case it was would we want to spend 3 hours with this person. So the ones that got interviewed where those that said they had learned how to juggle or ride a unicycle, or had done something interesting or unusual.

  • Patricia of Trakai

    Arrggghhh … don't get me started on the Physics GRE …

    (Posted as someone who got 780 verbal and 700 math on the regular GRE…)

  • Judy

    I was always a great test-taker and I took 'em in the pre-Kaplan days. I wish now that I had seen the 300% difference in my SAT scores and my best friend's scores (vs. the .001 difference in our gpa) as a business opportunity, as Kaplan did. (Mind you, my best friend of those days, who was deemed by the SAT "not college material" went to law school and presumably makes 3000% of what I make…)
    I remember doing well on the GRE but by that time it did not seem important, so I have no record of it. This blog suggests I remember right.

  • John

    What measures would you prefer? Grades are absurdly inflated, and recommendations are both inflated and influenced by how much the professor likes the student. The practical impact of dropping the GRE would be to increase the importance of the reputation of your undergraduate school and whether your undergraduate adviser has lots of friends in graduate departments. A really high GRE score is the only chance a kid from a podunk school with an unconnected adviser has of getting into a good graduate program. Eliminate the GRE and he or she will have no chance at all.

    • Ben

      John, you make a noble point, but my admittedly anecdotal evidence tells me that while there remain professors in the acadamy who write hyperbolic letters, admissions committees are far more piqued by those letters which frankly report an applicants strengths. Grade inflation is a problem, I agree, but with regard to the GRE acting as an academically underprivileged applicant’s last best hope for admission: a high GRE score may save them from ther initial cut, but when they make it to the committee’s final round, barring some affirmative action for the little guy, they still won’t last long against applicants from “better” schools who’ve had the resources and opportunities that they didn’t. Oftentimes it’s the tertiary benefits of the better school, not the name itself, that makes an applicant a better fit, or at least appear so.

      What’s the solution when faced with so many applications and a need for a single, standardized metric with which to compare them? I think, rather, the question is wrong before we even look at solutions. Instead of finding a way of coping with the over-saturation of applicant pools we should be working on reducing the applicant pool by creating a culture that doesn’t deem someone as societally illegitimate without a degree of higher education that may neither benefit them in their future career nor fill them with intrinsic reward. It is the volume of “grad school is my backup plan” applicants who drive admissions committees to create arbitrary cut-offs for standardized test scores.

  • Got Medieval

    Essays all the way. If you don't have enough staff to wade through the applications without an almost completely meaningless and arbitrary number to guide you, then you don't have the staff to have a grad program.

  • not quite medieval

    The GRE general test is trivial, even without memorizing the fractions that you list (I worked them out on the spot and had time left over). I liked the adaptive computer test because it saved me from the the imbecile-level questions, and left me thinking that anyone who can't get all three scores above 750 probably doesn't belong in graduate school. There are, of course, exceptions in both directions, and a serious weakness of the GRE is that it offers no way to catch them.

    I am more bothered by the attitudes of graduate faculty to the GRE content. Even relatively common words that we are tested on before admission are met with complaints of obscurantism if actually used. If you use the high-school mathematics, they balk that it's all too technical. Why take a test in order to study under professors who would apparently fail it?

  • John

    I'm curious; what sort of essays? How long? Would you want them standardized and centrally administered, or would each school craft its own?

  • jb

    The GRE is a waste of time, a waste of money, and there is no evidence of its validity as a measure of apititude for graduate level work. You would think at least the social scientists would be all over the lack of validity testing.

    What should be used as a measure of aptitude and skill is work that is relevant to the specific program. Undergraduate grades should be taken into account, of course; someone getting all Cs in the relevant program probably is not prepared for graduate work. And once given a certain level in grades, the next step is to compare actual performance. For a physics program, that may be a mathematics test or certain problem sets. For experimental sciences, maybe it is the design of a certain experiment. For history, it should be a piece of work that demonstrates reasoning from historical evidence, such as a research or historiography essay.

    I agree that there is a concern that students from "Podunk U." will have a harder time getting noticed without stupidly high GRE scores; like Got Medieval, I did very well on the (now oldie) GRE, especially those logic puzzles. And I have always wondered whether that was what got my application from a middle-ranking Canadian University considered (I went to

    But my ability to do logic puzzles still was not a good measure of my aptitude to do historical research, and why should I (with my logic-puzzle aptitude) be accepted to a program over someone with a better aptitude for what they actually wish to study but for whom logic puzzles are as clear as mud?

    The solution to discrimination against students by undergrad institution is just don't do it. We don't accept discrimination by gender, class or race, we shouldn't accept discrimination by program. Those of us who have graded in different places know that an A- in the Ivy League is no better than an A- at most state schools (and possibly worse); you would hope that the tenured faculty looking at graduate admissions would have at least as much perspective as their TAs.

    Oh, and as for mixing verbal and quantitative? That's just messed. I happen to be doing a history project which involves a lot of quantitative work, but it still has no geometry and the vast majority of humanities people don't even need to know how to count to do excellent research. Just like how many scientists don't have to be able to put together an elegant sentence to still do good work (trust me, I've edited the work of a top scientist – brilliant woman, don't let her near your creative writing round-robin). I had something in the 60th percentile in the old quantitative section, and this pathetically low score for a physicist or mathemetician or even a biologist was rightfully ignored by my history program. (I do enough demography that I regret my lack of statistics, but I have never needed to know how to work out the volume of a cylinder.)

  • jb

    On another side entirely: the GRE may be an economic and logistical barrier to international students and disabled students. While those in Canada and the US, at least those of us in big cities, can take the GRE at many different times, in other countries it may only happen a few times a year and involve considerable travel. I have no idea what ETS is charging overseas, but it wasn't cheap even in Canada.

    And there is no accomodation for disabled students. It's recognised by universities all over that some students with learning disabilities simply need a bit more time to organise their thoughts or tools such as calculators to do simple arithmetic. These accomodations are no more of an advantage to them than offering a note-taker to a deaf student. But ETS doesn't believe in learning disabilities, and if even the most talented learning disabled student wants to go to grad school in the US, they have to just pay up their money for a really invalid GRE and then hope that the admissions committees will ignore this expense piece of tolite paper.

    I know people from overseas (NZ, Australia) who went to grad school in Britain because taking the GRE was simply too much bother; I also know a learning disabled student who didn't apply to any American programs because of the GRE. If the USA wants to really bring the best in the world to its universities, it needs to drop the GRE.

    (Of course, the Brits also seriously need to clean up their application and funding systems – but that's another issue).

  • Orly

    I'm with theswain – the subject test was a horror for any medievalist lit person. I can recite Chaucer, do any number of close reading analysis on Gawain and the Green Knight , and even be able to identify time periods for various versions of Middle English (yes, even 10 years after graduating from college).
    However, ask me to identify modern theories by similarly obscure passages (i.e. put Lacan, Derrida, Kristeva, et. all in the same question) and I flounder – no matter how many times I tried to memorize my Blackwell Anthology of Criticism and Theory (I have tried). I only ended up scoring 630 on the subject test, and am now worried that Yale and similars will toss out my application while laughing at my presumption. I will have to take the horrible thing again in November (the regular GRE is over and done with, with a nifty 720V, which is what counts for English, thank god).
    What does cocktail-party knowledge of literature have to do with succeeding in an English PhD??

  • Alistair

    Graduate schools claim to keep the GRE scores, and also know who graduates and how many years it took (though they do not know who ends up in a happy life). Are these data readily available for us to analyze for correlation? Perhaps in some HIPAA-friendly anonymised form?

    At my first two graduate schools, admission was determined primarily via correspondence and conversation, and application was often the result of face-to-face invitation. The common opinion was that high grades could be achieved by pandering, or at least without the independent scholarly motivation required of graduate students. On the other hand, pressure from administration and government ensured that low grades really did indicate dullards. Outright failure was more respectable than a low pass.

    Scholarships, unfortunately, were controlled by the number-hungry administration on the basis of grades alone. The officers appointed to process the applicants had to be only minimally educated: competent to convert grading systems and sort numbers into order, but lacking any learned opinions that might bias the results. Such is the world of inappropriate quantification.

  • Alistair

    I started looking for data relating GRE scores to graduate school success measures. I found that not only are subsets of data available, but analyses and meta-analyses are even more readily available. And there IS evidence that GRE predicts graduate school success. Not what I expected given how often I hear that there is no evidence whatsoever.

  • Pingback: Felix sit annus novus, etc. (The Top 5 Posts of 2010) — Got Medieval

  • Priceeqn

    I agree with much of the above. I got the minimum general GRE scores to get considered by any graduate programs in mathematical biology or biology, and even with 3 papers, 2 of them first-author, accepted and in press (the 3 one finally came out a couple of weeks ago, very stoked about that!), some schools refused to interview me. However, my Master’s advisor, wise woman that she is, suggested that I either focus on publishing yet more papers, or waste lots and lots of time to raise my Quant. scores 50 points and possibly score some more interviews. I’m lucky that a clever theoretical ecologist thought I sounded cool and now I’m in a top program in my field. However, because of my GREs it took me 2 years to get into any PhD programs. Meanwhile, I keep running into these kids with super high GREs who haven’t published a thing and might and don’t much about our field. If they get lucky, they might publish something where they’re second or third author, in the allotted 5 years we have to finish. It baffles me how someone could think a badly constructed IQ test would have some correlation with how well someone will do in scientific research.

    • Priceeqn

      It appears jet lag has struck. Sorry for the couple errors!

  • Pingback: Tweets that mention Completely Serious: Let’s Drop the GRE — Got Medieval --

Bad Behavior has blocked 1167 access attempts in the last 7 days.