kitchen table math, the sequel: NAEP
Showing posts with label NAEP. Show all posts
Showing posts with label NAEP. Show all posts

Friday, June 28, 2013

Gone fishing (and did NCLB work?)

In Princeton, NJ, waiting for our room to be ready.

While I wait I'm looking at NAEP scores, and wondering.

Are we seeing evidence that No Child Left Behind worked?

  • Compared to the first assessment in 1971 for reading and in 1973 for mathematics, scores were higher in 2012 for 9- and 13-year-olds and not significantly different for 17-year-olds.
  • In both reading and mathematics at all three ages, Black students made larger gains from the early 1970s than White students.
  • Hispanic students made larger gains from the 1970s than White students in reading at all three ages and in mathematics at ages 13 and 17.
  • Female students have consistently outscored male students in reading at all three ages, but the gender gap narrowed from 1971 to 2012 at age 9.
  • At ages 9 and 13, the scores of male and female students were not significantly different in mathematics, but the gender gap in mathematics for 17-year-olds narrowed in comparison to 1973.

About NAEP
NAEP Summary

Monday, December 3, 2012

advanced writing in the 12th grade as assessed by NAEP

According to NAEP, in 2011 only 3% of high school seniors were able to write an essay as good as the one below, which is considered "Advanced."

% scoring "Advanced":
4% of white students
5% of Asian students
0% of black students
1% of Hispanic students.

5% of students whose parents had college degrees scored in the Advanced range.

5% scoring Advanced would be fine if "Advanced" meant Advanced. But it doesn't. "Advanced" on NAEP means 3% of high school seniors are able to write a coherent statement on the subject of:
  • Story or personal narrative about real/imagined difficult choice
  • Essay about technology important to student
  • Letter persuading council to build/not build convenience store
All 3 of these prompts call for opinion and the marshaling of evidence strictly (or nearly so) from the student's personal experience, and that is not at all what college writing is about. Nor is it the kind of writing one does in business or the professions.

From The Nation’s Report Card: Writing 2011:
Sample Task: Writing to explain
When writing to explain, the task of the writer is to bring together relevant information and to present this information with focus and clarity so that the topic becomes understandable to a reader. The sequence of ideas, and how ideas are arranged, must cohere and contribute to the communicative purpose.

One of the writing tasks from the twelfth-grade assessment asked students to write about a type of technology that they use in their lives and why they value that technology. The Value of Technology task began with a short video about young people’s use of technology. This video included animation and statistics about technology use. The written part of the task then specified an audience for students to address in explaining the value of a particular technology. Responses were rated using a scoring guide ranging from “Little or no skill” to “Effective.”

The sample student response shown below was rated as “Effective” in responding to the task about the value of technology. After an opening paragraph that defines video games and introduces the ideas to be developed throughout, the writer constructs the explanation primarily through use of personal experience. This approach skillfully communicates the value of video games through the use of detailed descriptions of specific games and what the writer has learned from them. Ideas are fully developed, and the rich use of explanatory details establishes a distinct voice speaking intelligently from experience. This response demonstrates skills associated with performance at the Advanced level. Twelfth-grade students at this level are able to craft responses that strategically accomplish the communicative purpose.

Student response - Grade 12 - Advanced
Videogames are a primary source of entertainment for people of all ages. Videogames are discs or cartridges that hold data; once a disc or cartridege is inserted into a gaming console, the data is read and displayed on the screen along with prompts that allow the game to be played. Games have many genres ranging from fighting to educational and can be used for than just mere entertainment. I personally have been experiencing what videogames have to offer for over five years now. Gaming is not just something that people do for fun, people can play videogames for many reasons. Videogames are an important factor in many peoples lives including mine and are a valuable type of technology.

I have been playing videogames from a very young age. Mario was the first game I was ever introduced to and it was not through playing; through sheer coincidence my mother realized that the theme music to Mario put me to sleep as a baby. Once I was old enough to hold a controller I began playing the game. Ever since that moment I have been playing videogames. Games are multi-purposed; to some it is merely a form of entertainment, but to others it could be their job. Some people argue that games are a waste of time and that they are not product. I beg to differ; games are important to me because not only do they give me something to do to pass time but they are also educational. A prime example of this is a game I was introduced to by my cousins, Runescape. When I was about thirteen I had went to see my cousins up state and I saw them playing this browser game called Runescape (a browser game is a game that can be played within an internet browser without the need to download or upload information from a disc or cartridge). Me being the person I am, I was curious as to what it was so I began to ask questions. By the end of the day I learned two things about that game, two things that to some gamers, were their favorite word. It was a MMORPG (massive multiplayer online role playing game) that was free; in essence it was a free gamethat I didn’t have to download and I could do basically whatever I wanted that was allowed in the game. Within the game you could do any of the various skills offered, quests, and even fight against other players from around the world with you’re avatar. Once I got home, I of course signed up and began to play. Throughout the few years I played that game I realized it was set in Medieval times and I learned many things about that age. I learned the process it takes to turn ore into metal, what smelting is, how leather is crafted into clothing, how clay is used, and some of the politics of Medieval civilizations throughout the quests of the game. Although I would spend hours on this game and it seemed like I was doing nothing, I infact was actually learning.

Another game my cousins introduced to me was Age of Mythology. The game was a PC game(which means it had to be bought and it contained disc which had to upload the game onto you’re computer or device and then the game could be played) and I had played it at my cousins and eventually went on to buy it. If mythology was a subject in school, this game could be the teacher. This game focuses around Greek, Egyptian and Norse mythology. You follow the antagonists (which you name) through all three civilizations chasing an evil minotaur that is attempting to end the world. You begin in a fictional Greek city and eventually move throughout the world. This game teaches any of it’s players not only how armies from all three civilazations worked but those civilazations major Gods, minor Gods, demigods and mythological creatures. Stories based on mythology or fact are also told and experienced throughout the game; such as the Trojan Horse and Ragnorak. I have never picked up a book based on mythology or ancient Gods but because of this game I have an extensive knowledge of the mythology of those three cultures. Games are important in society; they give people a hobby and peace of mind. They can also be used for educational purposes. Toddlers no longer read books to learn how to read, write, and spell, they are given toys and games to play. Games hold a high position in society and can be beneficial to those who use them if they wish to use them in that way.
This student, who is certainly a competent "personal" writer, has advanced a thesis: video games are valuable for more than entertainment.

In support of his thesis (I'm assuming the writer is male), he tells us that he learned "the process it takes to turn ore into metal, what smelting is, how leather is crafted into clothing, how clay is used, and some of the politics of Medieval civilizations" from a video game. This knowledge he acquired over a number of years and many hours of play.

In the next paragraph he tells us that although he has "never picked up a book based on mythology or ancient Gods" he nonetheless possesses "an extensive knowledge of the mythology of [Greek, Egyptian, and Norse] cultures" thanks to another video game. He provides no further detail as to what this knowledge consists of, or how long it took him to acquire it.

Essentially, the evidence this writer offers in support of his thesis boils down to: I remember stuff I saw in my video games.

The essay concludes with the assertion that "toddlers no longer read books to learn how to read, write, and spell." The writer offers no evidence to support this claim and seems not to know the meaning of the word "toddler." Toddlers have never read books, now or in the past, because toddlers are too young to read. They can't play video games, either, for that matter.

For my money, this essay is pretty much the exact opposite of what an Advanced high school senior should be able to produce in timed writing.

Very worrisome.

Thursday, February 23, 2012

I'm confused

What Do the NAEP Tests Really Measure?by David Klein | January 2011
The answer is green, right?

Or is there some aspect of this item I don't understand?

Maybe there's a right triangle hiding in there somewhere. Speaking as a veteran of SAT prep.

Thursday, July 7, 2011

summer reruns

I just came across this old post by Ken DeRosa!

Apropos in this summer of SAT math prep.

Maybe PWN will tell us what level of difficulty this problem would be rated on the SAT. I'm thinking 3 or possibly 4, and it would be a 4 only because a lot of students haven't taken algebra 2.

Saturday, July 17, 2010

"awful to be 18 and not have choices"

I was already a devoted fan of David Steiner before he was appointed New York state's Commissioner of Education. I'm rapidly becoming a devoted fan of Meryl Tisch, too.

In today's Wall Street Journal:
Acknowledging that a New York state high-school diploma doesn't mean a student is necessarily ready for college, the chancellor of the Board of Regents said she envisions the state providing two types of diplomas in the near future: one that is marked "college-ready," and one that is not.

There is no formal proposal yet to do so, but Merryl Tisch, the chancellor, said the move would be a natural extension of a broad effort by the Regents to toughen up academic standards. Those efforts are fueled by increasing evidence that even while New York students have been showing marked progress on state tests, their performance on national tests has stagnated.

"It's awful to be 18 and not have choices," Ms. Tisch said.

Nearly a quarter of students in all New York state two- and four-year colleges need to take remedial course work, according to John King, the state education department's deputy commissioner. Students taking remedial courses in their first year of college are less likely to graduate, he said.

According to research by the Department of Education, students who scored below an 80 on their math Regents exam have a much higher likelihood of being placed in remedial college courses. Students only need a 65 to pass the math Regents test in New York.

Regent Eyes New Diploma
By BARBARA MARTINEZ
JULY 17, 2010

Wednesday, November 19, 2008

Comments sought by Obama Administration

If you go over here you will be at an electronic form that allows you to submit suggestions for President-Elect Obama's education agenda.

Catherine Johnson has come up with three very succinct and cogent goals for education which she has submitted to her local school board (Irvington). You may wish to submit these as a suggestion--modifying as you see fit, of course. (Catherine has given permission to use these as a template for anyone in communicating with various authorities.)

I have changed Catherine's goals slightly, by substituting "U.S." for "Irvington". I also added a parenthetical about NAEP, which you may wish not to include. It is my particular beef. If you do include it, note that I use the word "test" rather than "assess". I urge you to do the same.


Here are the goals:

Goal number 1: Increase steadily the number of U.S. students whose academic achievement is on par with that of their peers in Europe and Asia.

Goal number 2: Increase steadily the number of U.S. students prepared to succeed in college work, with "Evidence of Attainment" being a steady increase in SAT scores. (NAEP scores should be disregarded until such time as the NAEP is revised to actually test age-appropriate content that students should be expected to master.)

Goal number 3: Reduce systematically the need for parents to reteach content and hire tutors.


Much thanks and appreciation to Catherine for coming up with these.

Sunday, December 16, 2007

still can't do fractions?

Tom Loveless is back:

The 2007 NAEP test results showed small but statistically significant gains in both math and reading. Mathematics scores at fourth and eighth grade continued the steady progress registered since the main NAEP test was first administered in 1990. Both grade levels notched 2 point gains in scale scores. Table 1-1 reports the magnitude of the math gains in scale score points and years of learning. Figure 1-1 illustrates the upward trajectory of the scores. The gains indicate that fourth and eighth graders in 2007 knew more than two additional years of mathematics compared to fourth and eighth graders in 1990. On the face of it, this is an amazing accomplishment. Previous Brown Center Reports have raised questions about such gains. The primary question concerns the content of the NAEP math tests. Students are clearly making progress, but at learning what kind of mathematics? Suffice it to say that students are making tremendous progress on the mathematics that NAEP assesses, in particular, problem solving with whole numbers, elementary data analysis and statistics, basic geometry, and recognizing patterns. NAEP pays scant attention to computation skills, knowledge and use of fractions, decimals, and percents, or algebra beyond the rudimentary topics that are found in the first chapter of a good algebra text. In sum, we know that students are getting better at some aspects of math. But we do not know how American students are doing on other critical topics, including topics that mathematicians and others believe lay the foundation for the study of advanced mathematics. Thus, the years of learning gain must be taken with a grain of salt.

The 2007 Brown Center Report on American Education:
How Well Are American Students Learning

plus ça change (scroll down)

Wednesday, October 31, 2007

compare and contrast

good heavens

Speaking of Advanced and Proficient and whatnot, remember this? (pdf file)

Placement Test for Singapore Primary Mathematics 6A
This test covers only new material taught in Primary Mathematics 6A

A motorist traveled from Town A to Town B. After traveling 1/3 of the distance for the journey at an average speed of 45 km/h, he continued to travel another 480 km to reach Town B. If his average speed for the entire journey was 54 km/h, what was his average speed for the last 2/3 of the distance?


A car and a truck were traveling to Town Q at constant average speeds. The car overtook the truck when they were 420 km from Town Q. The car arrived at Town Q at 6:30 p.m. while the van was still 120 km away from Town Q. The van arrived at Town Q at 8:30 p.m. What was the average speed of the car?

compare and contrast (Singapore vs US)
slave parents in Singapore

department of corrections

The post on "basic for high school seniors" is wrong -- thanks to Mark R. for catching this.

(fyi: There's a special trick to using the "Submit" feature at the top right hand corner of the page, which is that you have to actually hit the "Submit" key for the Submit feature to work. Who knew?)

Assuming I am now reading the NAEP page correctly (and if I were you I would not make that assumption), the fraction item is "Below Basic" for fourth graders.

No more quickie data-mining activities for me.

.....................

oh, and the function question isn't Advanced.

It falls under "proficient."

This is mortifying.

Tuesday, October 30, 2007

what is basic for a high school senior?




update - wrong, wrong, wrong

I'm sorry..... this is completely wrong.

I didn't hit the "Submit" button, so the page stayed on its default 4th grade position.

This problem is Below Basic for 12th graders.

Must go back now and check whether the function problem is listed in the category I assumed.

12th grade NAEP results - everyone vs. high-SES

All students:

Below Basic: 37.2

Basic: 48.2

Proficient: 13.0

Advanced: 1.6


Students with high-SES (highest income quartile):

Thirty percent of high-SES students understood mathematics at the Proficient and Advanced levels by 12th grade.

Here's how the scores break down by income quartile. (BB: below basic; B: basic; P: proficient; A: advanced)

Lowest quarter
BB 62.5
B 34.3
P 3.0
A 0.1

Middle-low quarter
46.0
46.2
7.3
0.5

Middle-high quarter
32.4
55.2
11.6
0.8

Highest quarter
17.0
53.0
25.8
4.2


sample questions, all levels

source:
Interpreting 12th-Graders’ NAEP-Scaled Mathematics Performance Using High School Predictors and Postsecondary Outcomes From the National Education Longitudinal Study of 1988 (NELS:88)


room at the top
12th grade NAEP results

room at the top

I've been out practicing some amateur data mining.

Fun!

Fordham has a write-up of the new NAEP report. (pdf file) Mike Petrilli talks about it here:

[A]nalysts followed a representative group of students who graduated from middle school in 1988 and, as high-school seniors, took a math test the results of which could be equated to the NAEP's scale. (Students were followed until 2000, when they were about 26 years old.)

Four scores were possible on the test: Below Basic, Basic, Proficient, and Advanced. Kids scoring Proficient or Advanced in 12th grade almost universally went on to attend and graduate college. That's a correlation, not a cause, but on the other hand it does tell me that my kid will be attending college with kids who know some math.

You can find an incomprehensible description of the levels is here. (incomprehensible to me, that is)

Sample questions from Below Basic through Advanced are here.

I took time to do a couple of the Advanced questions, and I'm not looking too bad I don't think -- entirely thanks to Saxon Math, Algebra 1 & 2 (the first two books of his high school trilogy, "Advanced Mathematics" being the third).

I'm proud to report that, thanks to Saxon, I was able to solve this problem:

If ƒ(x) = x2 + x and g(x) = 2x + 7, what is an expression for ƒ(g(x)) ?


Before working my way through Saxon, I had never seen or heard of a function; nor had I seen function notation.

I taught it to myself using Saxon, the lesson on function notation in Paul Foerster's Algebra 1, and the lesson at Purplemath.

I still find the notation itself a bit confusing, but that's because I need more practice.

more anon


update: wrong, again

This item falls under "proficient," not "advanced."

I'll have to see whether I can do the advanced items.

At this point, I think two Saxon books probably take me to the top of Proficient, and possibly into Advanced.

But I'll have to check.

Thursday, September 27, 2007

Mixed messages

I was pondering the disconnect between two very recent "news items".

On the one hand, everyone is gushing about the improvements in math scores on the 2007 National Assessment of Educational Progress. I'm reading about it everywhere, it seems. On the other hand, the U.S. is sitting out the Advanced TIMSS which is designed to show how our advanced students compare to those in other countries. Strangely enough, I'm NOT reading about that everywhere.

It seems we've managed to raise the floor (every so slightly) while letting the ceiling come crashing down. I guess that puts "good enough" somewhere in the middle.

Apparently, the goal is mediocrity. Based on those parameters, I'd say we're right on track.

Cross-posted at Mindless Math Mutterings.

Linda Seebach has a blog!!!

Yay!!!

The Eclectic Linda

I wonder if I can finally get her to join ktm? (She couldn't do so until she retires from her newspaper work.)

But look at black students separately, and Minnesota is dull yellow average on all four tests. Likewise, it is average for Hispanic students on all four tests. For white students, it is above average statistically on three of the four, but by only a few points. The state owes its high ranking primarily to the fact that it draws a larger proportion of its students from groups that on average score higher.

The preening of officials notwithstanding, the weather probably deserves more credit for state outcomes than they do.

I love Linda.