Skip to content

New Evidence of Racial Bias on SAT (Participation)

April 12, 2012

New Evidence of Racial Bias on SAT

June 21, 2010 – 3:00am

By Scott Jaschik

A new study may revive arguments that the average test scores of black students trail those of white students not just because of economic disadvantages, but because some parts of the test result in differential scores by race for students of equal academic prowess.

The finding — already being questioned by the College Board — could be extremely significant as many colleges that continue to rely on the SAT may be less comfortable doing so amid allegations that it is biased against black test-takers.

“The confirmation of unfair test results throws into question the validity of the test and, consequently, all decisions based on its results. All admissions decisions based exclusively or predominantly on SAT performance — and therefore access to higher education institutions and subsequent job placement and professional success — appear to be biased against the African American minority group and could be exposed to legal challenge,” says the study, which has just appeared in Harvard Educational Review (abstract available here).

The existence of racial patterns on SAT scores is hardly new. The average score on the reading part of the SAT was 429 for black students last year — 99 points behind the average for white students. And while white students’ scores were flat, the average score for black students fell by one. Statistics like these are debated every year when SAT data are released, and when similar breakdowns are offered on other standardized tests.

The standard explanation offered by defenders of the tests is that the large gaps reflect the inequities in American society — since black students are less likely than white students to attend well-financed, generously-staffed elementary and secondary schools, their scores lag.

In other words, the College Board says that American society is unfair, but the SAT is fair. And while many educators question that fairness of using a test on which wealthier students do consistently better than less wealthy students, research findings that directly isolate race as a factor in the fairness of individual SAT questions have, of late, been few.

The new paper in fact is based on a study that set out to replicate one of the last major studies to do so — a paper published in the Harvard Educational Review in 2003, strongly attacked by the College Board — and the new paper confirms those results (but using more recent SAT exams). The new paper is by Maria Santelices, assistant professor of education at the Catholic University of Chile, and Mark Wilson, professor of education at the University of California at Berkeley. The earlier study was by Roy Freedle of the Educational Testing Service.

The focus of both studies is on questions that show “differential item functioning,” known by its acronym DIF. A DIF question is one on which students “matched by proficiency” and other factors have variable scores, predictably by race, on selected questions. A DIF question has notable differences between black and white (or, in theory, other subsets of students) whose educational background and skill set suggest that they should get similar scores. The 2003 study and this year’s found no DIF issues in the mathematics section.

But what Freedle found in 2003 has now been confirmed independently by the new study: that some kinds of verbal questions have a DIF for black and white students. On some of the easier verbal questions, the two studies found that a DIF favored white students. On some of the most difficult verbal questions, the DIF favored black students. Freedle’s theory about why this would be the case was that easier questions are likely reflected in the cultural expressions that are used commonly in the dominant (white) society, so white students have an edge based not on education or study skills or aptitude, but because they are most likely growing up around white people. The more difficult words are more likely to be learned, not just absorbed.

While the studies found gains for both black and white students on parts of the SAT, the white advantage is larger such that the studies suggest scores for black students are being held down by the way the test is scored and that a shift to favor the more difficult questions would benefit black test-takers.

The new study is based on data for students who enrolled at the University of California system across several administrations of the SAT — with versions used subsequent to Freedle’s article. (The new research is the result of a study the authors undertook at the request of University of California officials, and they note in the paper that despite the request for information from the University of California, it took two years for the College Board to provide the data needed.) While the new study found the same DIF that Freedle did, an attempt to find a DIF for Latino students failed to show one.

But, the authors write, that doesn’t minimize the significance of their findings that back the study from 2003 that the College Board has said wasn’t accurate. “Although our findings limit the phenomenon observed to the verbal test and the African American subgroup, these findings are important because they show that the SAT, a high-stakes test with significant consequences for the educational opportunities available to young people in the United States, favors one ethnic group over another,” write Santelices and Wilson.

“Neither the specifics of the method used to study differential item functioning nor the date of the test analyzed invalidate Freedle’s claims that the SAT treats African American minorities unfairly.”

Kathleen Fineout Steinberg, a spokeswoman for the College Board, said that just as the organization disagreed with the 2003 study, so it does with the new research. She questioned whether the California sample could be seen as broad enough to draw conclusions on, and said that some of the tests examined had less of a DIF than others, raising questions about the assumptions made. She called the Harvard Educational Review study an example of “presenting inconsistent findings as conclusive fact.”

She said every test question used on the SAT is subjected to rigorous analysis (before use) to weed out any that would not be fair to all test takers. “We believe that our test is fair,” she said. “It is rigorously researched, probably the most rigorously researched standardized test in the world.”

As to the persistence of score differences, Steinberg said that this is not because of the test. “There certainly are subgroup differences in scores,” she said. “We recognize that and acknowledge it. It’s a reflection of educational inequity. It’s something we are concerned with.” She also said that the College Board welcomes research on the SAT, but viewed the Freedle study as having been “discredited,” and said that nothing in the new study changed that view.

The College Board’s tough stance on Freedle’s research is not new — and was recounted by Jay Matthews in an article in 2003 in The Atlantic Monthly. (Matthews broke the news about the new research, in his Washington Post blog.)

Robert Schaeffer, public education director of the National Center for Fair and Open Testing, a long-time critic of the SAT, called the new research “a bombshell,” and said that the study “presents a profound challenge to institutions which still rely heavily on the SAT to determine undergraduate admissions or scholarship awards.”

Schaeffer said that he agreed with the authors of the new study that use of the SAT could face legal challenges, given that this study now backs the finding that some of its questions may be harmful to the scores of black test-takers. While the College Board says colleges aren’t supposed to rely too much on the SAT, and most colleges that require the SAT say that they use it only as one factor among many, Schaeffer and others have doubted those claims.

“A shrewd litigator could use this study and the process of discovery to find out a lot more about how colleges use the test and, at a minimum, embarrass them,” he said.

More broadly, he said that with more colleges considering ending SAT requirements, this new study is “another strong argument” for doing so. “It’s going to add to the momentum.”

Marist College will be announcing this week that it is ending its SAT requirement, joining many others that have done so.
Read more:
Inside Higher Ed


From → Participation

  1. Alex Vo permalink

    This article does bring to light some topics that are important in education these days, however; there are also some parts of this argument that I do not personally agree with. For example, the section that discusses how whites are favored based on the types of words used in the questions. The author claims that white students have the advantage because they were more likely to be raised around that sort of language. This is not a problem with the test, but a problem with our society. I agree with the makers of the SAT in that the reasons of the disparity in scores is the social and economic inequalities of our nation. The reason that some black students didn’t grow up around difficult words are because that their parents did not have the finances to go to prestigious schools themselves. This is a horrible reality, but there is no reason to punish the test itself.

  2. Kate McNevin permalink

    I agree with Alex. I think the biggest flaw in this argument is that it addresses issues in our society, rather than issues in our test. Our society is so separate; it’s based on race, socioeconomic status, and culture. How are people supposed to write a test that is fair to everyone when life isn’t fair? It’s not fair that they were born into a less wealthy environment or that their parents didn’t have good jobs. Some people start out way behind the starting line, and that is the reason for discrepancies in test scores.

    Maybe the argument that the SAT should not be used in college admissions is valid, but I think they’re coming at it from the wrong angle. Instead of attacking the test or taking legal action against it, blame society. There is no “fair” way to write a standardized test for people who live in completely difference environments. This is a valid argument for throwing the test out because it would give college admissions the chance to change someone’s life for the better. Instead of seeing a number, they would see a person who might be driven and motivated, but just needs to catch a break from the schools.

  3. Kyla Chappell permalink

    I believe many schools should, and will, join Marist College in ending the use the SAT as an admissions requirement in the near future. I feel as though the ETS is hiding behind the little research they have that the test is not unfair. Even though there has been ample evidence to show that it clearly is. This is why I think, instead of trying to change the SAT, because obviously ETS is not planning to do much improving, colleges should drop this consideration in admissions. Especially since the test has shown little reflection of college grades and graduation rates, it seems as though it is not that necessary.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: