Karen Draney

KAREN DRANEY

My name’s Karen Draney. The last name’s D R A N E Y. Um and I work at the University of California Berkley in the school of education, the graduate school of education there and uh title I’m gonna guess Dr. Karen Draney.

OK, well what we mean by developmental approach to assessment is um looking at obviously kids development across time. Um and and their development in the subject area that they are working on. And the idea here is that when you’re assessing, you’re not just wanting to know well what facts have they learned? But you’re looking to see what kinds of levels of understanding, what kinds of levels of competency, you’re looking for higher skill levels, higher levels of competency, deeper levels of understanding, how well they tie things together um rather than just you know do they know the answers to simple questions or do they know a lot of the things on the list.

Well we’re actually one of the teachers that we’ve been working with um recently who is a teacher who is new to using the assessment system that we’ve been working with um what was talking about how she’s changed her thinking about student assessment and she said that when she assesses students now she is looking for right thinking she is no longer just looking for right answers. She says sometimes the work one has to do and the kind of assessment she’s doing is harder but because it’s so important to her now to look for right thinking rather than just right answers she’s learning to do that kind of harder work to make sure that’s what the kids are learning.

Certainly, but it’s almost always, it’s almost always possible when one is taking a standardized exam to figure out what the right answer is or at least to narrow down the choices for right answers using right thinking.

Well what we have people look for whenever they are designing an assessment system or whenever they’re deciding what kind of assessment they’re going to do when we work with curriculum developers and when we work with teachers we ask them to think about what are the really important things that you want the kids to be developing over the course of the curriculum or over the course of the instruction? You know if you if you think about you know maybe the five most important over arching principles or or or ideas or we call them variables and and they can be all kinds of things. Like in science we’ve got designing and conducting investigations as one of the things that people want their kids to come out of eighth grade science knowing how to do. Um and so we ask people to look for those kinds of over arching ideas, the most important ideas that someone should come out of the class knowing or being able to do.

The the kind of evidence you’re looking for when you’re looking for cognitive performance um can be all different kinds of evidence. And maybe in fact that’s that’s one of the important things is to look for the different kinds of evidence of what the kids can do. I mean there’s nothing we we certainly don’t say one should never give a multiple choice test. Um and we certainly don’t say one should never use performance assessment or one should never use you know essay writing or whatever um but it’s usually a good idea to start looking at evidence from different kinds of contexts like that. Um for example in the um issues evidence in new curriculum that we’ve been working with ah designing this assessment system the kids are asked to do all kinds of things. The kids um take some short answer tests um now and then at the end of the at the beginnings and ends of the units. Um the kids do, stand up and talk about issues in town meetings, the kids write letters to the editor, the kids do the standard lab activities where they fill out lab reports and do the data analysis. They’re they’re doing all kinds of different things. Again it kind of goes back to what it is you want to see the kids learning and what are the kinds of contexts in which that learning would be apparent.

Well well when we talk about development um we that’s something that can be looked at pretty simply in in some ways. Um for example there are there are are theories that are used I’m flounder, flounder, flounder(BIG interruption). Well one of the things that I that I think is a good example of development and looking at development ah is imbedded in the scouring guides that we use that the neubrix that we use in some of the curriculum that we are developing. And so for example one of them uses a thing that’s called the solo taxonomy and it comes from it comes from ah a theory that was developed by some researchers called Biggs and Callis, um but they basically just say that kids start out without much of a structure and then they then get one or two pieces so they call it helistructural and then they get several pieces but they haven’t really woven them together they can just put several, put several pieces on the table and that’s multistructural. And then they can start looking at the relationships between those pieces and that’s relational and then they can even go beyond the the the requirements, the task and and that so so so some of the scoring guides from the issues evidence from the new curriculum are kind of based on that idea. Um the kids get a zero if their answer’s just completely not on topic and a 1 if it’s incorrect but at least it’s on topic, and a 2 if it’s partially correct but it’s really missing something key, and a 3 if it’s complete and correct, and a 4 if they’ve gone beyond the requirements of the task in some way. And there are all kinds of ways that you can just think about, I mean that’s certainly not the only way that you can develop a rubrick to kind of look at what developmental levels kid kids are at. I mean sometimes people sit down with the kids’ work and they say well OK, well here’s the best work and here’s work that doesn’t seem to be quite so good and here’s work that really seems to me needs some help and you can kind of work backwards from that and figure out what is the difference between those things. Um and development doesn’t have to be, you know, terribly complicated theory. And someone as brilliant as Piage has to has to has has to pick up. It can be just looking at what kids do over the course of the year. Looking at what the the the better kids versus the kids who need a little help in your classes are doing, that kind of thing.

Um we’ve developed and assessment system that goes, really, well, ok. We’ve developed an assessment system that we’re starting to use with a number of different curricula. But the first curriculum that we kind of used is a beta test for it issues evidence in new curriculum. Um it’s put out by the Science Education Understanding Program at the Lawrence Hall of Science in California. And their focus is um science and society. So science can and and decision making in society(end of side of tape)
Varying, develop variables, uh um like designing, conducting, investigations, because this is science. And also variables like evidence and tradeoffs. So being able to identify what is evidence and how to make trade offs when you’ve got evidence kind of for two different sides of an issue. Um and what we did is we worked with them on to develop scoring guides based on those variables to look across the curriculum and find places where it was natural or logical to assess student um performance on one of those variables. So we made kind of a blueprint. (interruptions) Yes, so um finding logical places in the curriculum where one can um asses one or more of those variables and kind of rain that out like a blueprint saying alright you know here’s the first part of the school year, here’s the first part of the curriculum, here are the things that can be assessed in activities, say five or seven or twelve. Um and these are the scoring guides that you would use to assess those things and then we gathered a whole bunch of data from the student work um and had the teachers go through and identify what we call exemplars which are ways of making of these scoring guides, which are a little abstract, more concrete so you know here’s a perfect example of what we think is a one. Here is a perfect example of what we think is a two in this activity and so on. So we’ve got a collection of exemplars. Um and um when we’re developing mapping technology which lets teachers enter the scores they’ve given to the assessments across the course of the year, and um you know based on a little computation that the computer does for you draws and actual picture of what the student’s development looks like over the course of the year. You know it may be a student would start at ahh one between one-twoish kind of work on average and by the end of the year would be doing three fourish kind of wok on average. This way you could see a map, a trajectory of what the student had done. And so our our our our, there’s a lot of components of assessment that we’ve developed and you know they have different purposes that kind of fit together. We we actually call the resulting system um the BEAR Assessment System, BEAR for Berkley Evaluation and Assessment Research. And um we’ve had a number of other science curricula um curriculum developers in particular um approach us and want to see if they can integrate it into their curriculum. So…

Yeah, I think the most important thing about the kind of assessment that’s happening in this curriculum is it’s an on going thing. It’s not tests at the end of the chapter. It’s looking at how the students are doing in this activity, looking at how they are doing next week, thinking how they are doing a couple weeks after that, and really working with the students to help them see what they need to do to do better. A lot of the teachers in fact most of the teachers would encourage um to share the scoring guides with the students. So this is what we want you to do to get a 1 a 2 a 3 a 4. And we’ve had the students say that sometimes for the first time in their lives they actually know what the teachers want and we’ve had the teachers say um that the students come back when they get a 2 and and want to get a 3 they want to do it over and get it right where we know if a student gets an 80 on a test, well that’s ok, that’s an 80. But a 2, well that’s not complete and correct and I know what complete and correct means so I can do that. I mean that’s been really exciting for us. So…

Well, we think that there’s there’s a couple of key things that are important to student performance. First of all the scoring guides I think are really good for the students to see because it says things like like say say we are looking at evidence and trade offs and if we’re looking if we’re looking at getting a 3 we need to state noth sides of the issue and we need to state the evidence, and it needs to be evidence not opinion um we need to identify the evidence that that comes down on both sides of the issue. We need to make our decision and we need to say why we’ve made that decision and frame frame it in terms of the evidence we’ve used to make that decision. And that’s something that explains pretty clearly in some ways to the student what they’re suppose to be doing. Um there are things like exemplars, I think I think exemplars are are probably more important be a tool for the teachers although I certainly see they might be useful for the students um but for the teachers you know this is all a pretty new thing and um scoring guides aren’t something that people are use to using, scoring guides and rubricks most of the time. So it really helps they say, to understand exactly what is meant by this whole thing, to look and see what other teachers thought was performance worthy of a 1 a 2 a 3 and so on. Um another thing that teachers often do um is a process we call moderations. So you’ve got say 3, 4 science teachers at a school or in a district and they’ve all they are all using the curriculum, they are all using the same scoring guide that’s they want to make sure that all of them are kind of getting the same standards and they want those you know standards to kind of be locally consistent. And so what they’ll do is they’ll bring along some of their student’s work and they’ll all score all of the work from the different teachers and then they’ll discuss why they gave scores that they gave, especially when there’s some kind of disagreement. So that they can decide you know communally, ok what do we mean by complete and correct or you know what do we mean by incomplete or whatever, um as a group. And it it turned out to be a really um kind of fun professional development classes to watch. You just sit back, you watch the teachers they’re they’re talk goes right away to what can we do in our classroom to make this activity work better um it it’s fun and and it’s not acritficial, it doesn’t seem to be in any way.

That is probably the the the biggest issue with this kind of scoring and assessment is that it is more time in some ways. Um it’s certainly not going through a multiple choice test real quick and and and especially when you’ve got a lot of students it can be problematic. Um I know that there are different strategies that some of our different teachers have used. I know that some of them do not all of the assessments, they just kind of do some of them as quick checks, they kind of go through and say, ok you know let’s pull out five students, yeah they all seem to be getting it, they’re probably ready to go on or they probably need a little more work there seems to be some misconceptions that are coming up pretty often. Um so that’s one thing that you that you can only assume that you can assess only at certain points and once you can do kind of quicker checks for your own um your own purposes. Um another thing that some of the teachers do is that they give the students um scoring guides and you have the students score their own work or score each others work um and that sounds like it might be a thing that students would give each other 4’s all the time but it turns out that they’re sometimes their own harshes critics. That that if you let em sit and discuss, especially if you do moderation sometimes with the students and you have them discuss what paper should be given a 4 or a 3 or a 2 or whatever that they really get into why this should be a 2 and not a 3 it’s it’s pretty, we we’ve watched and video taped uh sessions of students doing this and they really think hard about it.

Well I think I’m gonna use the words of one of the teachers that we’ve spoken to um to tell you why it’s worth the investment and that is this this teacher that we’ve spoken to who says because now I’m seeing how students think. And not just that they have the right answer. Um and then to give you the other answer to why it’s worth it to watch the kids and kids really feeling like they know what it is that’s being asked of them that they’re learning something that they’ve never learned before um that they’re getting really excited by it. Um that they’re wanting to do better instead of just going oh yeah well 85% that’s ok.

Um I think that for teachers there are um a a few important issues about assessment that they should understand. Um the first is teachers need to be able to um design good test questions or good assessment activities or know how to choose good assessment activities or good test questions. Um you know that’s something that they’re gonna have to do and so knowing a few things about writing questions and about making sure that they kids can understand the questions clearly um is an important thing. Um I think that that understanding that you’re classroom testing and your classroom assessment should be related to what you want the kids to walk away with. Um that assessment isn’t just something you kind of tack on at the end but it’s how the kids learn about how they are doing. It’s how the the teachers learn, how how anybody knows whether or not this is working. Um(interruption)Well , I think I think I’m not sure,

Well I think thinking about assessment in the beginning of instruction and not just at the end of the chapter is an important thing. Um I think that even if you are going to give an end of chapter end of unit test of some sort you need to be thinking all along what’s important here, what do I really want the kids to get um what am I really focusing on, what have they done the most work on what have they paid the most attention to. And I think that when you are doing assessment that should be in there. Um you should you should make sure that the things you focused on and that you emphasized and the things that you thought were important are the things that are showing up in your assessment.

Well I think we’ve probably talked about most of my favorites actually. (interruption) If I were a teacher I think I think that if I were to do an assessment soapbox I’d be talking to other people than teachers. (interruption) What to say, goodness of all the things I think. (interruption). Well I I think I think that the concern a lot of times is some of the issues that come up with standardized testing and how standardized testing can sometimes drive instruction in a way that I think is probably healthy. Um there’s certainly nothing wrong with standardized assess or standardized testing, standardized assessment, but I think that it’s purpose isn’t one that works well in the classroom. You know you ah standardized assessment really is kind of trying to see how students are doing across a very large you know swap of society or or you are looking at how the same set of the same set of you know cut of students is doing year after year but the stuff that’s on a multiple choice test for example is not everything that a kid needs to know when they leave school and so the current focus where we’ve got to stop and make sure our kids do well on the standardized test I certainly understand the need for the kids to do well on the standardized test and how that affects the teachers, but on the other hand it seems a pity when so many teachers are doing so many interesting and creative things um that they feel the need to stop and and introduce some fairly unimaginative drill just so that their standardized test scores will be high enough. And this is not something that I’m blaming on the teachers, I just think it’s kind of an unhealthy system.