Saturday, August 21, 2010

Reliable National Mathematics Achievement Test Nonexistent

David Klein's article What do the NAEP math scores really measure? (to appear in the Notices of the American Mathematical Society, Dec. 2010) puts into serious question the use of NAEP as an external audit of the developing Common Core assessments.

Until such time as a reliable national mathematics achievement test comes into existence, the plethora of education research articles that base their findings on NAEP math scores should be considered with reservations. More reliable, for the time being, are state administered K-12 mathematics assessments directly tied to the content of credible state standards, as in the case of California.

Posted via email from concernedabouteducation's posterous

Monday, August 2, 2010

Historical Disconnect - Cut Scores and NAEP Achievement

The School Administrator June 2008 Number 6, Vol. 65

Cut Scores, NAEP achievement Levels and Their Discontents

The attempts of political bodies to bludgeon public schools with arbitrary performance standards
[selected excerpts]
In 1988, Congress created the National Assessment Governing Board and charged it with establishing standards. NAEP [then] became prescriptive, reporting not only what people did know but also laying claim to what they should know. The attempt to establish achievement levels in terms of the proportion of students at the basic, proficient and advanced levels failed.

The governing board hired a team of three well-known evaluators and psychometricians to evaluate the process — Daniel Stufflebeam of Western Michigan University, Richard Jaeger of the University of North Carolina at Greensboro and Michael Scriven of NOVA Southeastern University. The team delivered its final report on Aug. 23, 1991. This process does not work, the team averred, saying: “[T]he technical difficulties are extremely serious … these standards and the results obtained from them should under no circumstances be used as a baseline or benchmark … the procedures used in the exercise should under no circumstances be used as a model.”

NAGB, led by Chester E. Finn Jr., summarily fired the team, or at least tried to. Because the researchers already had delivered the final report, the contract required payment.

The inappropriate use of these levels continues today. The achievement levels have been rejected by the Government Accountability Office, the National Academy of Sciences, the National Academy of Education, the Center for Research in Evaluation, Student Standards and Testing and the Brookings Institution, as well as by individual psychometricians.

I have repeatedly observed that the NAEP results do not mesh with those from international comparisons. In the 1995 Trends in International Mathematics and Science Study, or TIMSS, assessment, American 4th graders finished third among 26 participating nations in science, but the NAEP science results from the same year stated that only 31 percent of them were proficient or better.

At the press conference announcing the [U.S. Chamber of Commerce/Center for American Progress jointly developed report “Leaders and Laggards” in February 2007], an incensed John Podesta, president and CEO of the Center for American Progress, declared: “It is unconscionable to me that there is not a single state in the country where a majority of 4th and 8th graders are proficient in math and reading.” He based his claim on the 2005 NAEP assessments.

Podesta could have saved himself some embarrassment had he read the recent study by Gary Phillips, formerly the acting commissioner of statistics at the National Center for Education Statistics. Phillips, now at the American Institutes for Research, had asked: “If students in other nations sat for NAEP assessments in reading, mathematics and science, how many of them would be proficient?”

Because we have scores for American students on NAEP and TIMSS and scores for students in other countries on TIMSS, it is possible to estimate the performance of other nations if their students took NAEP assessments.

How many of the 45 countries in TIMSS have a majority of their students proficient in reading? Zero, said Phillips. Sweden, the highest scoring nation, would show about one-third of its students proficient while the United States had 31 percent. In science, only two nations would have a majority of their students labeled proficient or better while six countries would cross that threshold in mathematics.

NAEP reports issued prior to the Bush administration noted that the commissioner of education statistics had declared the NAEP achievement levels usable only in a “developmental” way. That is, only until someone developed something better. But no one was or is working to develop anything better. When I wrote an op-ed piece for The Washington Post (“A Test Everyone Will Fail,” May 20, 2007), an indictment of the achievement levels, I got feedback that officials at the National Assessment Governing Board were quite satisfied with the levels as they are. That can only mean NAGB approves of the achievement levels used as sledgehammers to bludgeon public schools. They serve no other function.


Posted via email from concernedabouteducation's posterous

Common Core's Standards Still Don't Make the Grade!!

Why Massachusetts and California Must Retain Control Over Their Academic Destinies
Pioneer Institute White Paper by Sandra Stotsky and Ze’ev Wurman, July 30, 2010
The case for national standards rests in part on the need to remedy the inconsistent and inferior quality of many state standards and tests in order to equalize academic expectations for all students. The argument also addresses the urgent need to increase academic achievement for all students. In mathematics and science in particular, the United States needs much higher levels of achievement than its students currently demonstrate for it to remain competitive in a global economy.

Posted via email from concernedabouteducation's posterous