DENNIE PALMER WOLF
Dennie Palmer Wolf: Well um I run something called Project PACE at Harvard um which is a collection of projects that really looks to the questions of whether or not we’re a society that values the contributions of young people. (interruption) It’s called—It’s Projects in Active Cultural Engagement. Um, the other thing that should probably be on the screen is that um—because I’m in transition actually so I’m the hiat professor of education at Clark University.
I think first of all I want to make a kind of general comment. Um, I think at this particular time um given standards, given high-stakes testing often at the district but certainly at the state level um it’s very important I think for teachers, the classroom level, um to understand that the way they’re being asked by policy makers to assess is in what I would call an achievement frame, that is, “Has this child achieved whatever particular mark this is?” Um, particularly for students, for instance second language students, um who are going to have a course of development which may not be exactly the same as is laid out. It’s very important for teachers to have either what you might think of as a growth frame or as a developmental frame. Um, let me just il—illustrate that in one way. Um, most of the state tests that we have are, say 4th, 8th, and 10th, um and the question in those state tests is “Is this 4th grader at the standard set for 4th grade?” Most of the systems of testing compare this year’s 4th graders to last years 4th graders and look at the achievement levels of those two groups of kids. What we fundamentally need is actually a system that would look at where is a child in 3rd grade, where is she in 4th grade, where is she in 5th grade and is that growth trajectory um, as steep in the sense of making a lot of progress over time as we want it to be? Um, we really need to be able to think about children who are in 7th grade but reading at a 2nd grade level. Children at 2nd grade who’ve had many language advantages and who are consequently reading at the 7th grade level. But given the systems we have, we can’t do that and it’s really in the case of um second language students, um students may have some other kind of disability that it really brings to light the limitations of that kind of a system. Um, but it’s actually a limitation which in a sense second language children raise up. But it’s a—it’s a limitation for everybody because we don’t think in terms of growth. So that’s the kind of general framework. Um, I think the other thing that I would want teachers to understand is that the most powerful role for assessment is diagnostic not judgment oriented. The purpose of a classroom assessment, an incidental one or a more sort of serious one, or a semester and assessment really ought to be to figure out on the one hand “What does this child have?” On the other hand, “What does this child still need to have?” Um, many of the assessments we do fall very short on the “What does this child have?” They’re essentially negatively balanced so you get things like um “Does not understand paragraphing, doesn’t have conventional English spelling um, doesn’t have subject/verb agreement” or something like that. But the question is, “What does the child have?” So for instance, um in working with young teachers who have in their classrooms um children who are still in the process of acquiring English, I often see children’s papers come back and every instance where there’s not subject/verb agreement is marked as an error. But the interesting thing is if you look at those, they are the same error throughout which is very often that child transferring to English a grammatical structure or way of saying things that exists in their first language. And it’s not that there are 46 errors, it’s that there’s a different rule that’s in use. And for classroom teachers to be able to look at student work and to think about it in the light of saying, “Oh, I see, this child has a rule. It’s not the rule that is characteristic of Standard English, but it is a rule.” And that’s evidence of a mind at work, a mind trying to understand what the relationship of one language is to another language. Um, but—but that’s the important thing to understand. I think the second thing I—I wish that um teachers understood um is that assessment—we usually think of it in a testing model as um sort of out—out there. It’s something we apply to children. Um, but we fail, I think, to understand that that episode of testing can have real consequences for kids and I don’t just mean in the retention social promotion kinds of ways which are certainly heavy handed circumstances—uh consequences. But um in a sense that every time we sit down to assess a child informally, you know—you know, running record or via a test which you, you know, hand back, it’s a—it’s an opportunity to intervene not just to judge. So I’ve always remembered um seeing a—a middle school uh student’s paper. Um, a child who had recently arrived from California um from Mexico um where the writing although it was not smooth or conventional, there was a great deal of it, the child was going after subject, the child was responding to a piece of literature. And in addition to marking the paper in the sort of standard ways, the teacher had written at the bottom, you know, something like, you know, “See how much you enjoyed this science fiction story. Here are three other books that I think you’ll enjoy.” And that—that’s an intervention. It’s to say, yes, there are plenty of issues here with the ongoing process of accusition, but I can look through those issues also to see something that is present and it’s my responsibility to respond to a child not just in a deficit model, but again it’s this issue of acknowledging what is present and speaking to that child um as a reader, as an imaginer, as um a thinker. Not—not just this—the child who’s acquiring the second language. So I think those are—the answer to specific techniques or rules to go back. But they are in a sense a kind of classroom cultural frame around assessment. Um, but I think it’s absolutely critical. Um, and again, I think in this very high stakes assessment atmosphere um we can’t depend upon district testing or state testing or, you know, should we end up with a voluntary national test—national testing, to take that stance. It’s not designed to do that, it would be too labor intensive to do that, on and on. So classroom assessment is the only place where those kinds of assessments can take place. And, you know, luckily it’s the kind of assessment that goes on every day. So it’s really a remarkable opportunity on the—on the one hand to construct a very honest conversation with kids and with parents about what still needs to be done, but also to acknowledge what is and what’s potential. Um, so I’m sure you want to follow up on that.
Well, I mean this is a complicated issue, which sits, you know, squarely in the moral and ethical laps of classroom teachers. Um, I think it may be romantic and ultimately destructive um to break from what are the very real consequential frames that exist. Um, we would be doing children no favor to say to them all those evil tests um will have a—a separate system in here which is more respectful for them. Those tests will have affects for kids. Kids must be in classrooms where they both are learning enough um about the subject matter but also where they’re being helped to take such tests. I mean it breaks my heart in certain respects to think about classroom time being spent on that, but those are gatekeeper tests. Um, I think what I would recommend to teachers is to say—to help children acquire a kind of an assessment literacy. That is to have a very candid classroom conversation about the fact that many human interactions carry with them assessments. You meet someone on the street, you read aloud, any of those things will—asks—often provokes somebody else to say, “Oh, a good reader” or “Stumbled” or something. Um, but the—there are a range of different kinds of assessments and—just like the disarrays in literature, what kids need to learn is how to respond to each um and how to have the language and the tools of each. So that—I would say to teachers, “Yes, give them the tools of um writing a constructed response question.” Help them to understand what it means when the question says um, “How would you compare the hero of this story to the hero of that story. What does it mean you should actually do? And let’s practice it together.” Not in a kind of rigid way, but in a way that gives the students an understanding of what’s being asked for there, an understanding of the things that they must do, and an understanding of the kinds of things that they’re at liberty to choose how to do. On the other hand, you can as a classroom teacher say, “OK, we’re now going to write the first drafts of your report on China. Um, in this draft, what I’m really interested in is you showing me that you have a point of view, that you have a claim you want to make about ancient China and its culture. That you can read sources and use them. And in another draft, we’ll think about how you put that together as a powerful argument, and in another draft we’ll think about essentially editing it. But these are each different kinds of assessment situations with a particular purpose and I want you as my student to be able to understand and to respond to the particular purpose of this assessment.” Um, saying to kids, “In this classroom, because we know one another, um when I give you a question and you don’t understand the question or the question—you’d like to challenge the question, this is a place where you can do that.” Let’s talk about whether that’s a wise strategy in situations for people who don’t know you. So um I wouldn’t say that I would recommend for teachers to break with the frame of accountability, but rather for themselves and for kids to construct what you might think of as a repartra and understanding the social cues um with the cues about when each of those frames is being asked for and how to respond in—in each.
Well, I mean I think one thing that’s very important for classroom teachers to understand is that most of the high stakes testing that they and their children will encounter is curriculum independent. Those district tests or those states tests or the Stanford 9 or the Teranova, is based on a notion of what many 4th grade classrooms study and so what in a sort of model could you ask a 4th grader. You know, but that’s a model. Our classroom in science may have concentrated on geology because um we have access to a—a field where we can get different samples, the teacher has geology as a deep field of knowledge, but the questions on the test may spread across geology, ecology, biology. So it’s not necessarily matched to what you teach. Um, the powerful aspect of those state tests, if used well, is that they suggest to teachers that there’s a body of knowledge that all children deserve access too. Classroom assessment, however, is at liberty to actually deal with the curriculum that was taught. So in that case what I want in a classroom assessment is something that is tightly related to what I know has been taught um and which as an assessment asks my students to bring together the knowledge that they’ve developed all through our geology work, how to test the sample, how to find a sample, how to gather evidence, how to weigh, how to measure, how to draw conclusions from data. And then it’s my responsibility to design that assessment as the kind of culminating event um which kids are in a position to draw in their past knowledge. They’re in a position to go back to their notes, go back to their labs. Um, they’re in a position to ask one another questions um and that’s what I would call curriculum embedded. It’s matched, it draws together all the parts, and it draws on a kind of social community of learners. And you can—you can study for it um and you’re meant to study for it. Um, and those—those kinds of assessments are in fact the very difficult to design, very challenging to design. And I think unfortunately um in my first enthusiasm for performance assessment, for curriculum embedded assessment, I think we broadcast the message that through every chapter, every unit that you do, out to have a curriculum embedded, gorgeous, chaste assessment that was matched to the curriculum and that you’re only an upstanding and descent teacher if you could do that. Those kinds of assessments are very hard to design and if children encountered them three, four times a year, and if teachers joined together to develop them, to look at models, to critique one another’s work um then the quality of those curriculum embedded assignments could rise. I think uh, as I was saying, by sort of broadcasting a message that everything needs to be assessed in that way we built to big a burden and so each of these “curriculum embedded assignments” in a long string of say, you know, the many weeks the teacher teaches um no one of them has been good or rarely has it been good because there hasn’t been the time to develop it, the time to think it through. I mean, you know, in many respects there’s a reason why short answer and standardized items have come to dominate um in some—although they’re challenging to design well to a large scale of performance assessment embedded in a curriculum is even harder to design. Um, and so I think, you know, if teachers took back the message that three, four or them during a year was adequate, um that would be enough to begin to shift the assessment culture. Um, those kinds of assessments, though, are very important um because outside other adult standardized tests, a single service exam, for instance, most of the assessments that students will encounter are much closer to curriculum embedded assessments. They are moments in which people are asked to be responsible for a body of knowledge um that they’ve had the opportunity to work on or to know. So, you know, when you’re interviewed for the next level up position in a company, the question is, “Have you mastered the body of knowledge of which was for your previous position? Have you been able to expand upon it, um apply it, use it in interesting ways so that you’re entitled to move to the next one?” But um, I think we forget that.
Well, one of the things I think we’ve learned um—originally myself, other people, who thought it’s very important to break the strangle hold of standardized testing um held out the hope that more open-ended performance-based assessment based on classroom content that was actually taught would allow um a greater range of children to demonstrate their knowledge. You know, as in all things human, you learn. And I think some of the things we’ve learned is that those kinds of performance assessments are often very language dependent. Not only, you know, they have longer questions, they’re often multiple part questions um to the extent that they’re actually based on curriculum, they may involve the special language of uh balance scale, weight, ounces, so that they’re also vocabulary intense. And um they are also ones which assume that an individual student can carry out um a large number of steps or interconnected tasks um independently without support. Um, and there are two aspects of that. One, you really have to have very deep mastery to be able to do that. And for children who are still learning a language in which that geology may have been taught, their hold on the information concepts, the strategies, may until they achieve that fluency be less. You know, and here is very deeply tied—um this issue that much of the moving of children from various second language learning settings I think is all to often based on what I would call their playground or hallway level of fluency. That is, “Can you be asked a question? Can you answer it? Is the answer relevant to the question?” But it’s not based on children’s ability to operate in that same way with respect to academic content. And hall room and classroom—uh hallway and playground language is um very different. It’s much more sort of ritualistic. You can get by with saying, “Fine, thank you” or, you know, “Give me that.” Um, and the language of tests in particular is not at all that—that way. It’s not supported by social cues, by physical cues, by being able to watch the expression on somebody’s face. So it really, not only the academic language, but um the isolation of the questions from any surrounding context means that some of second language learners, most highly skills like taking in a range of cues in order to gauge whether you’ve got the right answer, are not relevant. Can’t be used, can’t be brought to bear. And so those—the more sophisticated performance assessments are—the more language dependent they are, the more dependent they are on a deep level of understanding of academic content. You know, the more challenging they are for anybody, but there are particular challenges from—for second language children. Um, so I think we need to think about, for instance, to go back to this question of classroom assessment, in designing classroom assessments, um a teacher or a group of teachers really have to sit down and look at the question from the perspective of a second language learner. Um, several years ago we were working with a group of people in um Delaware as they developed their state tests and um the people working on them there had the conviction that you ought to pilot items or similar items with children who are acquiring English as a second language, testing as a second language, academic content as a second language um just as you ought to teach—uh, just as you ought to pilot those same items with children with certain kinds of disabilities, dyslexia, those kinds of things. And that the simplifications that you made in those questions or those assessment designs in order to give a wider range of children access to being able to demonstrate what they know, would serve not only that particular population, but would serve to make the assessment questions or designs or the layout of items much clearer for everybody. So that I think, first of all, test developers but um teachers need to think about—need to sit down, for instance, with a second language learner, try that type of an assessment, see where the sticking points are, see the kinds of adaptations you can make without dumbing down the content that will make it clear. Which will make the language simpler, which will make it more transparent, which will lay out much more clearly what are the subtasks here. And in that respect, to see your second language learners as a resource, not um—not as um a group of kids that we have to do sort of a whole round of adaptations for. Their insights can be helpful for everybody.