The SAT is a scam. It has been around for 50 years. It has never measured anything. And it continues to measure nothing. And the whole game is that everybody who does well on it, is so delighted by their good fortune that they don't want to attack it. And they are the people in charge. Because of course, the way you get to be in charge is by having high test scores. So it's this terrific kind of rolling scam that every so often, somebody sort of looks and says--well, you know, does it measure intelligence? No. Does it predict college grades? No. Does it tell you how much you learned in high school? No. Does it predict life happiness or life success in any measure? No. It's measuring nothing. It is a test of very basic math and very basic reading skill. Nothing that a high school kid should be taking.

Is it an IQ test?

No.

Why isn't it an IQ test?

It's not an IQ test because it doesn't measure IQ. It is used that way. And it was developed from the army IQ test. But even the College Board will refuse to say that this is an intelligence test. And I'd love to see them say it. I'd love to see them say anything because then you can attack it. But there's this kind of mushy response that when you work your way through it, there's sort of nothing left. Well, it has a slight predictive validity to freshman year grades in college. We spend a 100 million dollars a year for that? You know, your grades in high school predict college grades better than this and we didn't have to spend anything.

They say the SAT provides a common yardstick for comparing grades at different schools.

Right. That's where all of the anti test people, I think, are wrong. And where the testing folks are right. You do need a common yardstick. You do need some way to judge an A at this school or this teacher versus an A at this school or this teacher. But there are lots of common yardsticks. Again, you could use blood type. You could use height. Anything is a common yardstick. What you have to say is, fine, it's common. But it is useful? And there are lots of tests that are more useful than the SAT that are also common.

Like?

Well, for instance, the Advanced Placement Tests. They are rigorous. They're difficult. There are lots of them. You can say, my interest is history. And so I'm going to take a history Advanced Placement test. And I'm going to get--I want kids to be rigorous. I want curricula to be rigor to be rigorous. But I want them to be not one size fits all and not mindless. Like, let's have kids studying hard but let's have them studying something useful, hard.

Let's talk about what you do.

We're here to raise your scores. And depending on what class you walk into--if you're a 600 level math student and we want to get you a 700, then this course has nothing to do with math. It is--you know the math. It has to do with being a good test taker. If you're a 300 level math student and you really don't know how to add fractions, then we're a math course. It really depends on where you walk in. But the fact is, that for most of our students the problem is more the SAT itself than it is their math skill, their English skill or their ability to do college level work. Not every student.

When you walk into a Princeton class, do they tell you that there are certain tricks?

And if you're bad at test taking--you always think that's sort of, you know--a parent says, my kid just can't take tests. And in the back of your mind is, well what you're saying is, your kid is an idiot. But it's really true--some people can't take tests, because they don't speak that language.

The tricks we teach are common sense to a good test taker. It's what you do. We're just saying, this is what kids who do well on tests do. They might not know they're doing it. But they're doing it.

Does it really just boil down to test taking strategies?

A kid's coming in to get higher scores. All right? We're not there to teach Shakespeare because the test doesn't measure Shakespeare. We're not there to teach trigonometry because it's not on the SAT. We're there to teach you to score better on this exam because this exam is going to determine where you go to college and how much financial aid you get. It's also going to determine how you feel about yourself in some large measure. You still remember your SAT scores. And everybody else does too. Everybody's forgotten everything about themselves, everything else about high school. They remember their SAT scores.

What were yours?

I remember. They were about 1500.

Princeton Review's model is Joe Bloggs, who is Joe Bloggs?

Joe Bloggs is the average American student. He's not stupid and he's not smart. He's average. Right? He scores a 500 in math and a 500 in English. And what you've got to do when you take the SAT is realize--people will tell you it's in order of difficulty. What they really mean or what's really true is that, they gave each question to a couple thousand kids. Right? Some of the questions--everybody got them right. How did they know if a question's easy or tough? Everybody got it right, it must be easy. Everybody got it wrong, it must be tough. Joe Bloggs got the easy ones right and got the tough ones wrong. So there you are on an easy question and you know everyone in the country got it right. Whatever you want to do, whatever you gut answer is, do that. Don't second guess yourself. Just do it. Because it'll be right. When you're on a tough question though, whatever you want to do--whatever Joe Bloggs would do--don't do it. Because it for sure is a wrong answer. If it were right, how come this question's at the end of the test? Why is, why did everybody manage to get this question wrong and somehow it came to you in a flash? It didn't come to you in a flash. The wrong answer came to you in a flash. It's a trick answer. The--give me the first word that comes to your mind. Cat.

Dog.

Hello.

Bye-bye.

Up.

Down.

Peanut butter.

Jam.

Table.

Cloth.

Hmm. Grass.

Green.

Okay. There's certain words that come to everybody's mind. There's certain things everybody does. That's Joe Bloggs. It's knowing that my gut answer is the same as the gut answer of everybody else in this room. And on an easy question, go with the gut answer because it works. And on a tough question, eliminate it because for sure it's wrong.

But it sounds so cynical--forgive the expression.

No. It's common sense. It's the way you live your life, your whole life. Honestly. Like, all the time in real life, you know, you handle the question in context. You're at a parking meter and you've just parked. And your wife says do you have a quarter? Right? The only thing that runs through you mind is, well, do I have a quarter? You know, and you look. Now, you're in the middle of the park and it's dead of night. And someone big is heading towards you. And he sort of leans and he says, hey, do you have a quarter? And the first thought in your mind is not, well, do I have a quarter? And start looking. It's, well, do I have problem here? It's common sense.

On a tough question--you're in the middle of the park and it's dead of night. You know, and you just go to-- well, why are they putting that question? Why am I putting this answer?

When we were in one of the classes, the instructor sat there and said, you can eliminate this Joe Bloggs answer. Now you're left with three. Don't you think if we were to put that much instruction into real instruction, the kid could get the right answer?

If you were put that much time into writing a good test, then we would spend the time teaching it. If this were an essay test measuring how well you understood current events, we would teach current events. Like, don't shoot us. Like, we'll go where ever they want us to go. If you write a test that has little trap answers all over the place, then we're going to teach kids how to get around the traps. Because that's what we're paid to do. Our job isn't to teach you Shakespeare because that's not going to help you get into college.

How is your business? Is it growing? Is it booming? Would you safely characterize America's view of the SAT as being obsessed with it?

There are a couple different things going on. Number one, college admissions is much more difficult, much more selective, than it was 20 years ago. So these tests are more important. Number two, college is a lot more expensive. And more kids are going to state schools. They're not going to spend a lot of time reading your essays. It's your scores and your grades. And so the test is even more important at a large state institution than it is at a small school. And more kids are going to large state schools. Number three, people still have an emotional attachment to the SAT. It's still your first kind of mono-a-mono experience. Like you against the whole country. But back then they thought, this is an IQ test. I can't do anything about it. So you sort of had a little nervousness about it. But it was no big deal. It was like, it was like a blood test. You know? It was something you had to go in and do. It made, made you a little nervous but it is what it is. Now you realize that this is a very coachable test. And so it's no longer just a little bit of nervousness. Now it's like three months of preparation.

ETS says it's not a coachable test...

ETS has refused easily 50 times to do an independent study that we both get involved in, because it would be so easy, right? We say, here's a list of our kids. Here's exactly when they took the course from and to. Go put it in a computer. ETS, you supply all the numbers. And let's look at the prior and the post and we'll know. Right? We know that as a country, kids who start where my kids start--which is around 1100 ah, better than the average--ah, go up about 20 points. And these kids go up this many points. And the difference is us. Right? It's an easy study. And they don't want to do it. The only time they do a study is by themselves, not asking for any help, not showing anybody their methodology or showing anybody the raw data.

Why won't they participate?

Let's say that they acknowledged, after 50 years of lying, that the SAT was very coachable. Number one, they've got to account for 50 years of lying. And number two, they've got to deal with, with the implications of successful coaching to race and gender bias issues. Okay. Rich kids all coach. Poor kids don't. There's a widening gap between rich kids and poor kids. We can no longer say it's because of the schools or because of the unfairness of society. Now we have to admit that we're part of the problem. There's a gap between men and women that's substantial--that they can just sort of laugh off. And you'll watch as you interview them and they do a dance. Why do men out perform women on the SAT? The SAT's supposed to predict college grades. Women do better in high school and they do better in college. What's the problem here? Ah, the more you use, the more you start accepting that the SAT's coachable, the more problems you have with it.

Do you think that it's fair to have so many kids actually paying to bring their scores up and other kids who can't afford to?

First of all, I'll tell you--I mean, there's an appropriate amount of studying to do for the SAT. Like kids in our course are spending hundreds of dollars and a lot of time. We also have the best selling books and software. And those are like $20. But somewhere in between the books or the software and the course, that's a reasonable amount of money to spend for a pretty important test. There are also kids spending a year and 20 or 30 thousand dollars prepping for the SAT. So the problem sort of magnifies as a different strata of society--no coaching, a little coaching, more coaching, a ridiculous amount of coaching. Scores just get pulled apart. And then you set social policy on people's scores and you've got a big problem.

Second--I have a big problem with these laws. These are politicians getting involved in decisions that they have no business being involved in. Being an admissions officer is like being a casting director. It's about choosing a class that's going to work well together. You want men and women. You want blacks and whites. You want a diverse class. You want people who are great athletes to be on your teams. People who are great journalists to be on your papers. And people who are great actors to be in your acting department. You're building a team. To say to them, look, all you can do is use this score, this measure of performance--that's all that's important--is making their job into a joke. These are very serious professionals. And you're saying, look you can be replaced by--hell, not a computer--like a calculator. Right? Just punch it in and, and you get a number. And I think that's ridiculous.

Your competitor says the test is flawed but it's the best we have and we should keep it. Do you think we should keep it?

We should get rid of the SAT as fast as we can. Look, there are bigger problems in society. This is not the biggest problem we have. But it's so easy to get rid of it. Right? Just pull the plug.

Can we return to the correlation between the SAT and socioeconomic status. The higher the income the higher the score. Is this fair that this is what it's measuring?

Well, it is very fair that rich people have nicer clothes than poor people. If you believe in capitalism, you got to go with that. The problem with the SAT is that it pretends to be a meritocracy. Right? Again, if it's in the back of a magazine and rich people do better than poor people, who cares? But this is a test where everybody's saying, look, we're just being an incredibly fair society here. Everybody takes this test. And the better kids go to the better schools. And it's just bullshit. You know, the better kids hire me.

Tell me about it.

I couldn't agree more with their findings. The SAT is a bad test. It is biased. It measures nothing. And we should get rid of it. We do need a common yardstick. We do need a good test of some sort or several tests. The Princeton Review can prep kids for just about anything. We have kids working with us for the medical boards on a test of, like, anatomy. It's a real serious content test. And we teach real serious content. I think we're pretty good teachers. If you give us a really good test, I think we can be really good teachers for that test. The SAT will never be a really good test. It is a scam.

You say it doesn't measure intelligence. What does it measure?

The SAT is said to predict freshman year grades in college, a little. And it does. It measures it a little. Almost anything you do, including family income, will measure freshman year grades a little. But the point is that it doesn't measure intelligence. It doesn't measure anything that's worth a 100 million dollars a year prepping for it.

But the man who started ETS, started it with a noble purpose.

Yes. To prove that white northern Europeans were smarter than everybody else. That was his men were smarter than everybody else. That was his goal.

Every problem starts out as a solution to a problem. And the SAT is no exception. The world before the SAT--it was kids from Andover and Exeter got into these top colleges. And the rest of us didn't.This was a leveling. But it was a leveling among upper middle class white men. Because that was the population who took it. And as the rest of society started taking it, the SAT became more and more irrelevant. And the tragedy about the SAT is not its invention. It's as the world changed, the SAT didn't.

You look even ten years ago--before the Internet, you know, before any number of things--the SAT looks exactly the same. You look 50 years ago and the SAT is still the same. This test is a dinosaur.

They're saying that the early bird gets the worm. If it's a coachable test, then let's start coaching yearsahead.

I got to tell you--we strongly discourage students from working with us before the end of sophomore year. Like strongly. If you call up the Princeton Review office and say, I'm a freshman and I want to start working for the SAT. We will turn you away, because we think that you shouldn't spend your entire high school life thinking about the SAT. That there is a limit to the neurosis that even we can tolerate.

But the problem with these tests--and the state exams as well as the SAT--is that as you ratchet up the stakes, this becomes more and more important. Can you really blame people for taking it very seriously? And you can't look at the people who are spending a ridiculous amount of money and time obsessing about the SAT. You have to look to the test and say, why are setting the stakes like this? Why are making a test of something that has nothing to do with high school?

Can you walk us through a question from the SAT?

P, Q, R and S are four towns. P is farther north than Q and R. S is farther south than P. Q is farther south than R. Which town is the farthest south? P, Q, R, S or it cannot be determined. Like, is it, would you consider that an IQ question?

A logic question.

Is it a logic question? I think it's....

A trick question?

The first time you see it, you might be confused or you might have seen something enough like it or you might be good with maps. Once someone has explained this question to you, you will never get it wrong again. Right? I mean, there's a simple way to do it.

What kind of a question does it strike you as?

A, very coachable. B, not really math, certainly nothing you learn in high school and nothing that will be useful in college. Plausibly a little bit of logic, I'll accept that. But I think more just a little trick. I mean, what they're trying to see is that just because two things are south of you, you can't really determine if one is further south than the other just because it's Q instead of R. It's the kind of question that they do a lot. There are four towns. A, B, C, D.

Let me start.

It's the kind of thing they do a lot. There are three towns--A, B and C. The distance from A to B is four. The distance from B to C is three. What's the distance from A to C? And of course if you're kind of linear, you would say, well it's seven. It's the two together. But of course, they don't have to be in order. So it could be one. Does it measure what you learned in high school? No. Is that useful in college? No. It's a little trick thing. You'll get it wrong once. Someone will explain it. And then you'll be ready for it.

Why develop a question like that if it's only measuring a test-taking technique?

Because the point of writing an SAT question is not measuring what you learn in high school or how well you'll do in college. It's separating out kids. There are kids who will get that right. And they're generally the kids who have been in math courses where they play with this kind of stuff. Which is to say, upper income. And there are kids who will get it wrong because they don't play with this stuff.

So the question is very good at separating kids. And that's why they have it here.

Let's do some more.

Money is to bank, as food is to basket, park is to city, cash is to store, book is library and article is to magazine. So what you're supposed to do is say, I put money in a bank. And I guess the, ah--or money is kept in a bank in the same way that a book is kept in a library. Maybe it's a vocab question. Do you know the word bank or the word money. I can't believe that it's a logic question. And I can't believe the kid who gets that right is going to be better in college than the kid who gets it wrong. Ah, especially since there are some wrong answers that are pretty attractive here. You know, you sort of think of banks and money. And there's an answer with cash. And you sort of--you know, you're stressed. This is important. You're in a rush. You're in the middle of the SAT, which is a pretty important test. It's nine in the morning on a Saturday. You're probably hung over. And a lot of kids who get that wrong--it's not because they don't speak English. And it's not because they won't do well in college. Our experience is, a kid who doesn't do well on the SAT--it's not because he gets the toughest questions wrong. It's because they make lots of careless mistakes on easy questions. They get sucked into trap answers a lot. Because they don't have their footing. They don't understand the question. It's not that they can't do it.

So a lot of the course isn't focusing on the toughest questions. It's focusing on making sure you don't get that question wrong.

How did you come up with the Joe Bloggs technique?

I met a fellow I started the country named Adam Robinson. Who was working with this girl, and every time she would get it down to two, and every time she would get it wrong. And she was going nuts. And finally he said, get it down to two. Guess something. And then answer the other thing. Because whatever you do is wrong. And then he realized that's actually not random. A bad test taker isn't bad randomly. He's bad because he believes that there's an easy answer there. He moves towards the attractive answer on a difficult question.

Let's find some questions.

Okay. I got a question that I pulled, that I, that I sort of think is useful. Did they do the doughnut question in class? It's sort of a quintessential SAT question.

What about the doughnut question?

You got a doughnut here. And the question reads, in the figure above, what's the greatest number of non overlapping regions into which the shaded region--the doughnut--can be cut with two straight lines? In other words, how many pieces can you cut the doughnut into with two straight lines?

This is the last question on the test. It's Saturday morning. You're very stressed. You're very tired. And this is really important. So what Joe Bloggs does is, he'll just cross lines. Right? The easy answer there is four. I can make four pieces pretty easily. What are the odds that on the toughest question on the SAT that you've done enough work? Right? They're zero. No way. And again, a good test taker's sitting there. And he answers four and goes, god, there must be tougher than this. And he's right. So you cancel four. And of course, since you're able to get four, you cancel three and two also. That's not the greatest number you're able to cut it into. You're able to cut it into at least four. The answer's got to be five or six. And then you sit back for a second and you say, what else would Joe Bloggs do? He'd say, well, they want the greatest number possible. So maybe it's six. Maybe it's the greatest number. So it's not that either. The answer's got to be five.

On the one hand you might way, well that's kind of a goofy way to take the test. That's all testmanship. On the other hand you might say, what is this question telling you about a kids math skill or his ability to do college level work? This is just a good question. And goofy questions deserve goofy preparation.

And another one...

There are three roads from Plattsville to Ocean Heights. And 4 roads from Ocean Heights to Bay Cove. If Martina drives from Plattsville to Bay Cove and back, passes through Ocean Heights in both directions and does travel any road twice, how many different routes for the trip are possible? 72, 36, 24, 18 and 12?

Like, what is this telling you about your son. Like, is it telling you he's stupid that he got it wrong? Is it telling you he shouldn't go to college or he should? What is it telling you? And I would claim it tells you almost nothing. A great question for Games Magazine but a lousy question for Harvard.

home | discussion | who got in? | interviews | the race issue | sat & test prep | history of the sat
the screening process | test score gap | getting in to berkeley | bibliography | links | tapes & transcripts | press | synopsis

FRONTLINE | pbs online | wgbh