Norman Webb

NORMAN WEBB

Norman Webb a senior research scientist at the University of Wisconsin Madison. I work at the Wisconsin center for education research.

With the role of assessment in a in a reform I do think that there’s been some uh change over the last uh thirty years and that I particularly think of reform periods as going every fifteen years and where you have uh the uh new math era back in the sixties and the seventies and their assessment didn’t play very much uh a role. They were starting, states were starting to get involved with assessment and doing uh state assessment but uh it was not uh as large uh effort as it is uh uh current time. And then in the ma—uh in the middle they had about uh back to the basic uh era in the early 1980’s uh with the um uh uh with the Reagan (?) and and the nation at risk uh was when they states got much more interested in in assessment uh particularly at the lower end uh where you had the minimum competency testing and and so the strategy become more to to uh to uh look at (clear throat) what is the what is the knowledge that students need to have who are at the very minimum level of uh understanding uh as they go through the school. And so that uh that’s when a lot of the states began their state assessment program is when you started to get graduation requirements and then you have the standards movement that came toward the end of the 1980’s and 1990’s which we’re in right now and uh assessment has become more now looking at the the full range of knowledge and and relating to particular standards and so assessment is not so much as the driver but it’s one of the important components.

Well the way I think about the relationship between the standards and and assessment I see those uh really tide in also with the curriculum because I think you have to have the curriculum and standards and assessment all working together and so that I I think that the standards are expectations, they they express what people want uh what people should know or what people think other people should know. And I think that the assessment then is a very important part of that because without having any ways of measuring that understanding then uh people will not attend as much to what what is in the standards and so I think that the assessment and also the whole accountability effort is is very very important to that and and without accountability effort I think of not only the measures the instruments but also the incentives that uh is built in for people to accomplish the goals and so forth. And one of the things uh uh is this in special education and LEP uh uh language (clear throat) uh uh where where there’s language or whether learners of of English as a second language um I think one of things that why I mentioned the accountability is I think it is very important for them to be counted. And so that uh I I think the uh uh with regards to assessment even though there’s language issues and language problems that that they might face, as soon a possible I think they need to get into the mainstream with regards to the assessment particularly in an assessment standard system.

Well I think the uh uh if if they’re not accounted if they’re not included in the numbers then it’s too easy for them to freak out to be uh left out and and and that there is not as is said about the accountability systems there’s not as strong incentives for teachers, school, principles, districts, large districts to apply the resources to get them to uh to the level that they should be. That they will take their resources, a large district will take their resources and apply them other places where where if they’re going to be held accountable on on test scores. And so I think that (clear throat) it’s a it’s a fine balance that you have to walk because uh clearly uh um students that uh have uh are just learning English and to be given uh uh test all in English and and and it’s difficult and so that would in in invalid, it won’t measure their full knowledge. But in term of trying to get them included in that um um testing system and getting them included in the high stakes and and also the high standards uh I think it’s very very important. In Wisconsin what we do as a state is that we include all students in the denominator, that is that uh of all the students that uh uh districts should be held accountable for and even if uh the uh students are not tested it’s still included in the denominator and so that gives the the schools some incentive to try to get those students up to a level so they can be tested.

Well I think where we where we have been is is that there’s a large number of students that have been excluded from from the accountability system. Uh we use a word, uh we’re walking we’re working with um um Milwaukee public schools and so we did an analysis of uh of students who were in special education and uh found a lot of them were excluded from from the assessment and we call that MIA’s or missing from from assessment. And so that uh (clear throat) uh I think that what we’d like to do is to is to get those stu—students in some form of assessment and particularly in relationship to the standards. This is why I think it’s so important to have a standard based system uh where you are really saying that this is what we expect students and this is our understanding of what students need to know in order to go on to higher education, go in the work force, be just a uh ordinary citizen. And I think that the closer we can get to having students uh achieve those standards which means that you should be able to measure in some way and so that I think that uh I think we’re learning more about accommodations. I think we’re learning more about uh how we can uh look at assessment instruments that that are comparable uh but uh maybe are uh done in different ways that that where we can include more students where they are with regards to their learning and understanding and so we can report out their progress in in in meeting the standards.

Standard for for with regards to um um curriculum standards in and and standards for um a learning and and what is um uh what was before and what was now. I think I think the uh the standards the the NCTM standards, the National Council Teachers in Mathematics standards came out first in 1989, I was apart of the writing group and lead the working group that really looked at the evaluation by way those are called NCTM curriculum anti-evaluation standards. A lot of time people forget the evaluation. Um the uh before that I thought that there’s a lot of work or a lot of uh of material that are included in the standards that that uh were basic knowledge that people knew about. Uh for example for for a number of years, twenty years they’d been trying to get more problem solving in to the curriculum and NCTM has had a number of policy documents that they’ve printed that talk about we need students to learn how to solve problems in mathematics. And and um and that that we something that even in the new math era that people talked about. Uh but but what was happening in the new math era that they really focused on the upper level of the of the student population. Some people say the top five- percent or maybe ten percent. The standards movement is uh very uh distinctly talks about these are standards for all students that these are expectations for all students and they um and and it talked about reasoning it talked about communication it talked about um it talked about problem solving and and it sort of laid out the curriculum, it sort of laid out what was for K4 what was for 5-8 and all uh uh and and what was for 9-10 or 9-12. And so what I think that the standards has done is add focus and it and it really has allowed uh um states to model those standards and to start to to plan and to manage the curriculum in their states in in a common direction and so I think the standards movement has been very positive and I think there is even evidence that that the student outcomes have have improved and this notion of students for all or mathematics for all students I think is is a a very very important concept and I think that that was something that that was not said as much in the 80’s or in the um in the 70’s as I hear in the 90’s, and so I think that’s related to that having the standards now.

I think what we’re learning about um how students learn mathematics uh that that uh again in the last ten fifteen years that we’ve done considerable amount of research that particularly the cognitive guided instruction, you probably had other people talk about that um is where where you focus on students understanding of mathematics that the instruction and the teacher needs to have ways in order to uh gain from students and understand from students what their understanding of the mathematics is and then you need to build on that understanding, their understanding of of um of in the direction to increase their knowledge and and so they eventually come to the level where the understand mathematics as expressed in standards and and even beyond. And so I think that uh this this notion of of understanding what students understand is is a very very critical part of of of this uh concept.

As so that the the the notion of communication in the classroom is one way that that you can get at the students understanding is have the students talk, and then have the students write about what they understand about mathematics. Also have them model using manipulative and so that I think that uh incorporating more dialogue uh within the uh mathematics classroom can be very helpful. Uh at the same time it takes a particular skill and training of teachers in order to do that. I I’ve done some evaluation of curriculum projects where where um communication was a very important part of that. Have students work in small groups uh have students report out to explain their thinking, have students uh report on projects and so forth. Uh but what in terms of having students work in small groups, in theory it can be very effective, but in practice uh it it takes a skill of a teacher to orchestrate that and uh and particularly where I saw that uh uh if if students could could talk with each other about a problem and and verbalize their their particular perspective on it, sometimes the mathematics was wrong and so that the teacher needs to some how be able to first of all observe that particularly if there’s multiple groups with in the classroom, be able to have techniques that that she can intervene with that group not necessarily telling them the answer but but getting them so that they converge more on on on the solution. I think what this implies is that the teachers need to be, first of all they need to be very knowledgeable about the mathematics and and so that um I think that’s something that teachers continually have to work on because the mathematics is changing somewhat. They also have to have a very uh uh strong knowledge of Pedagogy (?) and techniques that they can in order illicit students thinking and also in order to to orchestrate these these groups. I think they also have to have a very strong notion of the curriculum and the structure curriculum in terms of where it needs to build. And so that uh where a teacher can isolate what that students knowledge is what the strategies the student is able to do, where that student places in the whole field of understanding the mathematics. And I think this applies too to special education. I think it applies too to English as a second language students that that there ought to be some mechanism that can be use to place the these students on that developmental uh progress, or developmental map of learning mathematics.

In terms of the uh kinds of knowledge that teachers ought to have and and and what they ought to be exposed to in in a a pre-service education. I think it’s pre-service education and in-service education because I think that uh teachers continually update be updated with the uh advancing technologies and assessment. I think teachers need to have a basic understanding of uh the attributes of assessment. And that uh and just a basic understanding of the uh underlying notions of of uh of assessment and I’m thinking particularly on is that assessment is an inference, it’s it’s you can never measure everything exactly, and so you have to infer from from from assessment. Assessment is is uh trying to get data on students in order to further their understanding and and your understanding of what they what they know about mathematics or science or what ever. And so that they also need to know that uh about validity and reliability which are uh two I think uh critical aspects that is that when ever you do an assessment when ever you gather information about a student you ought to be questioning your self about the validity of that. Is it really measuring what you’re trying to measure? Because in many cases that that uh uh a student will produce a a result and that it will not really get at the knowledge that you you think you’re trying to get at, whether it be computation, whether it be measurement or some other uh mathematical uh skill. I think you also have to look at reliability. And reliability in a in a measurement means that they uh have to uh uh that a measurement has to be consistent. That is uh if you give a a instrument and a question and students answers it one way this moment and then you give them the same question next they answer it in another way they it’s not reliable. I think another important attribute in the standards based system is alignment. And I and I think that that’s where the assessment fits in with the standards and also fits in with the curriculum and also with uh where you’d like students to to move. So I think those are all important attributes. I think that uh they also ought to have an understanding of of assessment, uh just what assessment is and that every assessment involves probably about five parts. One is that it involves a question of some sort. That is that were somebody is giving people uh uh given a student a question and that could take a variety of forms. Another uh uh part of that is that it involves a response. That is you have to get information from the student in response to that question and they you have to do some sort of uh interpretation. That is where the teacher has to interpret what those that response is and then you have to do some sort of a a uh analysis and so put it into perspective and this is where I think you have to put it in uh perspective in terms of the curriculum and the structure of the curriculum and finally there’s a reporting aspect of that. And so that uh uh assessment does no good if you don’t do anything with it and so that uh in terms of writing up reports I think that teachers need to know about that. And so I uh uh there’s also uh I think uh teachers need to understand about formal assessment uh and and those are where you have to know what a stanine (?) is what a percentile is, uh what a what’s the difference between a norm-reference test verses a criterion reference test the first is a performance assessment. These are all I think important uh uh bits of information that that that’s teachers need to understand and we find that uh that that uh in working with teachers that they have a hard time interpreting data uh as it’s reported from standardized tests. And so I think that that would be another really important attribute they have. I think with regards to in-service and it’s going a little bit beyond uh the question is is that we are in a very uh fast changing society with regards to data and so that we have much more data that we ever had before and I think that that teachers need to understand more about how to use data and and what data is available and so I think this is something where it’s impossible to put it all with in the pre-service curriculum but it’s something that teachers need to continually update on in terms of how do you analyze data, the data that’s available and the the different parts of the data.

Okay multiple uh the the importance of multiple measures uh uh is there’s a couple of ways of of looking at that. One is that multiple measures as I said uh uh assessment is an inference and and so that you there’s always error that is built in any assessment and so to have multiple measures helps to alleviate some of that error and so that uh if you could get the same result two occasions on two occasions on three occasions or with different kinds of instruments then you the teacher can say that the information’s starting to converge and I can really draw more uh inference from this and I give it more credibility. I think multiple measures are also on a larger scale are important because one of the things that uh that we have to guard against is for the curriculum to converge on the assessment instrument. If you have a very high stakes assessment instrument and we have some examples of that in the country now is that the teachers begin to teacher towards the test and and that’s the only thing they teach which uh which means that uh because a test is only a small sample of what is important knowledge uh that that uh then uh does does a disservice. So by having multiple measures and and um and and multiple measures with equal weight or important weight causes teachers then not to so much teach for one test but to broaden their instruction so it’s a safe guard. So I think those are the two reasons that uh multiple measures are important.

Multiple measures would be using a standardized test which uh normally standardized norm-reference test, uh like a teri----(?) or or a stat-9 (?). And then another test would be a performance assessment uh which would require students to uh uh maybe do a project or require students to do an extended uh problem uh where they would have to do writing they might have to gather data then report out their thinking. That would be an example of of multiple measures.

Good question. Uh in terms of how teachers can use reliability and validity concepts in their own teaching I think is is uh is an important question. Um I think first of all that you don’t want to expect teachers to to apply uh the like uh like a reliability formula uh could a Richardson twenty and have them compute that or be able to go through the computation. Uh but but I think it’s important for them to understand the concept the underlying concept of that even in a simple quiz uh that that where you’re looking at uh how students are responding to four or five questions is there consistency in terms of how they respond to that question or or is there where where a student might respond to the first question differently even though there differently from the from the last question even though they’re the same uh uh the same um um uh type. Uh I think that for teachers to to look at their how students are responding is is a very very important part. One of the things that they could do uh is if they have uh like uh an hour test or end of course test or even uh uh uh six week test one of the things they can do is they can look at uh the first half or of the test verses the last half of the test and that adds a that’s a measure of of reliability. Of course it depends on the structure of the test. The validity notion is is if you’re looking for students understanding of say um of number uh does this really measure number and and have I included all possibilities or are students able to get the right answer for the wrong reason. Uh there’s there’s the um uh an example in the NCTM standards where they show a a uh hexagon uh that uh is a irregular hexagon uh shape. And they put the the numbers of all the uh six sides that are that are on that hexagon. Two three five six seven and they ask the students to uh what’s the perimeter of that hexagon and they say well if they get it right then they mean the that they um are understanding perimeter how ever the most logical thing for students to do to that is just add all those numbers together and so a students could get that answer and not have an understanding of perimeter. And so I think that that would be an example of of the validity that is the test the students get the right answer for the wrong reason and so how do you get that how will a teacher understand. First they have to have some notion to look for this question of validity and then they have to have some techniques of of doing that and that is uh that’s why again multiple measures are important. What if we give them uh uh uh another problem that doesn’t give all measures of all the sides ask them about perimeter or ask them to uh to define what they mean by perimeter have you know two or three ways of doing it.

In terms of having the the uh standard uh attributes of any assessment uh the five standard attributes of any assessment and how teachers can can um can use that and why it’s important to be more standardized in terms of thinking. I think that um uh after a teacher has has uh practiced this kind of uh uh assessment for for um a long time it does become sort of intuitive and I think that’s that’s good. But I I think one of the things to to to think about on this assessment and and particularly the idea where you go from question to response to uh to interpretation. One is for them to start to think about what is the response. What are the different ways that I’m having students respond and so that that I think that this get in particularly when we’re doing with diverse students and students that that uh maybe some students can respond better in writing and students can respond or think better verbally. And so for a student for a teacher to have this notion of response and that the response can be of different kinds and different types I think is very very important. I think it’s also very important to include this notion of interpretation. And that that teachers have to interpret that, there has to be some thing, some way of interpreting it and that and that it’s always an investigation in a certain sense of what they are doing. What does this test score mean? What are these – what does this uh um uh if I give them a ninety-eight on a on a test and and and another student has a ninety-six, what’s the difference in that and and how much should interpretation should I give in that. How much error is built into that assessment? And so that then gives them the notion that they have to look at that and then I think it’s also important for teachers to understand this this they have to assign meaning to that and meaning means that they need to put it context. They need to put it in what does this student know now what do I know about the students now what about the prior experiences that the student has, where do I want that student to be or where does that student need to be, how does this all fit in and and where can I build upon that.

The assessment role in the classroom and how it should inform instruction I think uh I think that is important it’s an important uh attribute or it’s an important um idea. I think I think the uh in terms of assessment I see assessment in that sense a tool and and it’s a it’s one of many tools that teachers have available to them and so that I think that uh that that in terms of informing teachers as to what they should be doing within the classroom and particularly where they’re dealing with a number of students might be thirty students might be forty students might me twenty students. Because all those are going to be in different places and different understanding and so that assessment gives a tool uh that the teacher can gather data about students and about their understanding of mathematics and them use that to manage and to guide their instruction what they need to do. One of the biggest mistakes I think that a lot of happens within uh uh the instruction is that teachers keep on re-teaching what students already know and so they go over and over and over again and one reason why they do that is they haven’t used assessment effectively enough to really understand what students know about that and so that they can be much more efficient I think with regards to their instruction. I think some replication or some repetition is important but uh but not as much as I think goes on. And I think also that that uh where you’re identifying students with severe learning issues that that I think that that’s very important to measure progress that that that you have to see progress that at some how you’re making progress with that particular student and so assessment helps in that because then you that that that you know a students has made this progress from this ---(?) gives you more encouragement to continue, otherwise you might without having any information on this student you might give up, you might uh say why should we.

Having done a lot of interviews that’s always my last question too and so I like I like that last question sort of the catch all you know what have I forgotten as an interviewer and so that uh I really like that question. I don’t like to respond to that question but I like that question so uh the the um the soap box or what what do I think or you know the sort of concluding remark about assessment I I think uh assessment is is uh a very very important of the educational system and I think that where we are right now that that it’s uh that it actually has a uh prominent role. Where we going to be ten years from now is another question. I think that assessment is going to be a very important part I think we are going to do more with computer uh uh uh adapted uh uh testing and computer assisted testing that I think it’s going to make it much easier and and much more efficient in term of trying to get at students knowledge. And so that I’m encouraged that that the the assessment field is going to be changing and I think along with that we have to have our teachers and and uh and our educational system to change.

With regards to what teachers ought to know about large-scale assessment I think that the more they know about large scale assessment I think is is critical. One of the things is is this notion of standardize test. I think people by enlarge use that wrong because what standardize means is means it’s administered the same way and and that doesn’t necessarily have to mean that it’s a multiple choice test and that you could have a standardized performance assessment and so that I think this notion of standardized test we need use that uh uh appropriately and that uh I I think that that teachers need to um clearly as as as responsibility for their craft is to know about how these standardized multiple choice norm-reference tests are are used and how that differs from say a standardized norm-referenced or or standardized criterion- reference uh test and that uh and that the greater understanding that they have and and that the greater understanding they have on terms of of what that means I think is is uh important like one of the things is is great equivalent is something that is is is used uh I think people don’t understand how great equivalent uh is developed and that a lot of times it’s only an estimate and so that uh to use that uh a lot of times people use that erroneously and so that um I I think that teachers need to know more about how information is reported, you know what uh what great equivalence is, percentiles, stay-nines(?), uh uh NCE’s, which is normal curve equivalent, and and what the differences between those uh uh different statistics are uh I think it’s the more they know about that they better.