I’m the associate director of the Center for Assessment. Center for Assessment – it’s located in Portsmouth, New Hampshire.
Accountability systems, are, are, really a, a, uh, incurrent version, primarily at the state level. And the states are concerned with having common uh, content standards in particular and they are working towards common performance standards. And, then they often have common instruments for assessment. Some states are also working towards comprehensive assessment and accountability systems, which allow for diversity at the local level within some overall framework. But in general, the uh, I think that the, um, movement will be towards greater commonality at least in content um, and uh, expectations.
Accountability systems are coming in as part of the standards based movement and common standards, I think will uh, as part of their core philosophy, will increase commonality across communities, in content standards and often in expectations of what students, uh, are expected to learn. Uh, and then in the state systems and accountability systems, they will, those are often translated into very specific policies about uh, what may be uh expected at a grade level or a grade span in a content area and student performance where those students are expected to be able to know and and do. Some states are working on comprehensive assessment systems that allow for local discretion in how those meet, those standards may be approached or how evidence may be um, demonstrated for it. So they aren’t looking for a particular way of students learning or demonstrating their learning, but they are looking for evidence that students, um can meet the the state sponsored curriculum or or standards framework.
I think that there’s uh, uh, a commonality in, in , state, state standards are, are important for all students, I, I think including second language learners, um, special ed students, and, and gifted and talented—full range of students—because the state standards are supposed to be identifying those knowledge and skill aspects that are, um, essential for all students, to be able to do in order to be productive in their schooling years as well as afterwards. I think that one of the issues is how can that be demonstrated and that how soon are those standards expected, but uh, most states I think are saying that second language learners should have the, core educational foundation that will allow them as many opportunities and choices as any other student.
Assessment is a key area for teachers to understand. One of the things that I think teachers should decide about is, what is the purpose of the assessment. There are assessments that um, that are done for the state, for state purposes, there are probably some that are done for districts and large districts, some that in High schools will be done for department reasons, uh, in fact there will be things that the teacher will want to assess for his or her own purposes, and then there are some for the individual students or groups of students. So I think deciding on the purpose is really important. Those purposes might be to find out where those students are, what they’ve learned. That’s probably one of the most common purposes. But there are other purposes as well. Teachers may want information from assessments in order to improve their program, their curriculum. What I do this year, have the affect that I wanted in terms of helping students. I need to be able to look at information so assessment in the broad sense, so I can improve what I do for the next year. Sometimes uh districts or uh schools want to make decisions about uh programs. Should I, should we adopt this reading program? Should we change our math program? And they ought to be looking at assessment data for that as well. So there are full range of uh purposes that teachers would want to consider when they think about their assessments.
It’s um, I think there are 38 states now that have large scale external assessments. And um, all of them have had to come up with a policy regarding second language learners. A federal law requires that, that second language learners be included um, but it leaves some discretion to the state. So, most state policies will say that second language learners must be included if they have been in an English language program for two years or or in some cases one year. Um, the teachers need to be aware of, of, of, in the, in those testing situations what the administration policies are. Um, who is included and then if the student is included what are the possible or permissible accommodations and modifications and then how it affects both score reporting and score usage. Accommodations would be, um, is or a, a, a language version of the test other than English. If so, is it available for all the subject areas. Some states will have a second language, or uh a non English version in math because math should, uh, in it’s purist form many people argue not be uh dependant on English uh, but they’ll have a read—the state standard will be to read in English and so they won’t have a um, uh non-English version. In some cases, uh in writing though, they may say it is very permissible for the student to be able to demonstrate their competence in writing in their native language and not in English. So, uh, those, those policies vary from state to state and teachers need to be aware of, of what they are so that they can both understand the purpose of the state assessment and make sure that their students are, are best served in, in matching those purposes of the state with their own.
One of the really key questions going on now is, the relationship between state assessments and local assessments. I, I think that, they actually serve pretty different purposes. State assessments, um, are usually done to provide policy makers at the state level and the public information about how schools are doing in general… how education in the state is going. They are not very good at providing sensitive information about individual students. Um, and I think that most good teachers can generate much more um, valid and extensive information about students from within their own classroom, uh classroom settings. So what is it that teachers need to um, think about in terms of, of information from the state assessments? Um I think usually it checks—it serves as a check on, on their own perceptions, I may think that I’m doing well in some areas or not doing well, it’s, it’s helpful to to have a large scale assessment check on that. Uh, because states have the constitutional responsibility for providing an adequate and good education in every state in the union, um, looking at the state frameworks is actually a very important thing. And, and local teachers should take that responsibility pretty seriously. It’s not, I, I don’t think, it’s up to teachers to say ‘I can emphasize math and de-emphasize science’ when the s—when the state frameworks say that we need to pay attention to both for example.
A teachers have uh, have uh wonderful responsibility. They’re charged with helping every student learn. And I think that their responsibility for every student means wherever that student is and we all bring different abilities and capacities and, and um, experiences to the classroom as learners. And, and teachers have this great opportunity for trying to, uh, work and help every student um develop along those lines. Second language learners have some uh, both, uh, strengths and challenges. And I think teachers need to be aware of those as they are with, with uh every other student and try to work with them. Uh, one of the, the great things about second language learners is that there is a body of evidence and uh research that helps in form teachers so, so that there’s um a lot of training uh available to help teachers who might have students who uh are learning English as a second language, um or another language, have another native language uh to help them in their instruction.
The um, professional literature and federal um guidelines and in many state-state guidelines emphasize that there must be multiple sources of evidence, assessment evidence in making decisions about students. There are really two reasons for that. One is, um, we know that very few assessments, I can’t think of any, um, provide a full look at what a student could do in an important area – mathematics, writing, or reading comprehension. So in order to be able to sample the full range of, of content standards, or the construct as it’s called, we need to have multiple, um, sources of evidence. The second one is that, um, one assessment doesn’t provide a reliable enough indication of what a student might be able to do. Uh, and we all have good days and bad days and uh, so the variation of student performance might be, uh, actually fairly large and so, statistical folks have emphasize the need to have multiple meas-measure-measurements of, or, uh times when these things are assessed in order to have a more reliable or stable, uh, measure. I think that teachers inherently know that. As we get to know our students, we-we have in fact day to day contact with them, and we uh know when the performances are not what we would expect. We’re, we’re pretty good at reliability. Uh, I think that sometimes we, we need a little more challenge on the validity part. We need to expand the opportunities we give our students to demonstrate their, their competence or what they can do. Um, so that we really are sure that we have an accurate picture. Um, there’s been a lot of talk about multiple intelligences or, or um authentic assessment that, that really is addressing this issue. There’s a student of verbal presenter, they, they’re very good at explaining their concept, they may not be as good at writing it, writing it. Can students apply what they uh are able to do in a performance setting, but they may be pretty shy about talking it. I think as teachers um, um think about the full range of, not only instruction presentation that they can do, but response things that students can do, they’ll naturally fit into, uh, this role of getting multiple evidence um to get a more accurate picture of what students can do.
Teachers have a, a role of um fostering students, and most teachers I, I know want students to become more independent. So there’s this great tension between um giving feedback and and helping students be more independent and sort of jumping in and showing them how, how to do it, and sometimes taking over a little bit too much. And every teacher has that, that a tension. Um, so what we want to do is give feedback and structure it enough, or um in research there’s talking about scaffolding so that students can be successful in what they are doing and as, and but helping them become more and more independent, primarily along more complex uh lines. So, an, as an example, um, in mathematics, uh students may be able to understand an idea but not be able to know when to apply it. And, and we want to be able to give students feedback on on that and gradually lead them to more, uh both complex problems and it may be that it’s a more open ended problems so that, that it’s not only how to apply it but of the three or four possible ways to approach the students need to, um choose-choose an appropriate one for them. Um, I think that teachers um are always thinking about how can I um, help these students and feedback is essential and then um having uh feedback it helps students become more independent along time I think is another dimension to think about.
I, I think one of the unfortunate things going on is that there’s so much public attention on accountability assessments, s-sponsored by the states that uh many people are thinking that’s the primary thing to do, or that’s the most important thing to do. I- I don’t think that’s – that’s true at all. Um, teachers really need to be primarily responsible for students learning and diagnostic assessments, classroom assessments uh will be the things that both help students learn and will provide the information to teachers to , to help them do that. And I’m pretty confident that, if the um, efforts are aligned, if, if, we pay attention to our classroom instructional assessments the performance on the state assessments will follow. The problems that we usually have are if there’s disagreement about what is important so that the state assessment may emphasize um, a certain type of math problem or a certain type of um reading um, understanding and uh it may emphasize it 60% of the time or at 60% of the test and we don’t think that that’s the right proportion, that’s where there’s usually some uh disagreement. But I think if teachers pay attention to helping students learn that includes good classroom assessments, then the other part should primarily take care of itself.
Because, um, there is in some state laws, and and some federal law, the call for multiple measures, some states in their accountability systems have included things in addition to student performance on, on state tests or state assessments. Um, I think about half the states have some other measures whih may be attendance, or drop out, those two are the most common ones, but other states, um may use things, some have teacher attendance, some have student, um attendance of students uh, at risk, many have some process variables of credentialing of teachers and so on. I think that the important part to realize there is that um just a student test scores don’t provide a sufficient, um uh characterization of what a good school is, is probably also true that good test scores don’t provide an indication of what a good student is. Um, and I think that, um, parents especially will say, uh, we want our, we want our child to learn, but there are many other things that we would, we would be very grateful if the school could uh help us develop those in, in our children. Which isn’t to say that schools are responsible only for character development and and in families and communities—they are not but uh schools that, that have high learners, but the children are are selfish or disrespectful or too competitive, uh I think parents will say, Th-this is not what I want the school to be helping my child develop. I think parents um pretty well communicate those to teachers and, and teachers would do well to be sensitive to those other indicators of, of um, values that the community has for, for what a good student is.
One of the most important movements going on now that the accountability movement is going for, I guess over a decade uh in the United States, is that people are realizing that that we need to take apart the data or disaggregate it more in order to have the, um influence on on improving student performance and school performance that, that uh we’d like. So, one score saying this is how your school did um is important for the public. The public doesn’t really want to have to have 15 scores, but for a school it’s very important for a school to look at data. Are the uh, uh, how are the boys doing in relation to the girls. It’s not uncommon to have um boys score lower than girls in certain subjects and certain grades and girls score higher and it’s very interesting when the trend reverses. So schools want to look at, uh, how males and females are doing, they, they usually look at how significant um racial and ethnic sub groups are doing. They’ll look at different learner groups, ELL learners, Special Ed students, sometimes they’ll look at students who have been in the school longer uh if they are coming from particular communities. And the primary, the real important thing to, to realize is that, we’re not trying to explain a way or make excuses for any differences. We’re trying to be aware of them so that we can be proactive and take uh actions for them that will help, um, uh every child be able to, to a, meet his or her potential. Teachers need to do that same thing and uh there’s been a lot of talk about whether we teach to the middle or teach to the top or teach to the bottom and how, how often um do we do that. We really need to be aware of all the different students. One of the nice things that a teacher has is usually, she or he has an opportunity to get to know every student um, somewhat on an individual basis at least in elementary school and more in the middle school concepts, sometimes in Junior Highs and High School we have a harder time doing that. But um, when we disaggregate our data on on and individual level as a teacher, or in these group levels as the school, it helps us, um focus our, our instruction to make sure that, that uh every student is being affected in a good way.
In um, statistical in professional areas, there’s a lot of talk about reliability and validity. Reliability is the um tendency of scores to be consistent or stable over time or over different populations. Validity is the, um, uh correct or true interpretation of those scores, did-did the students really learn and also their use. So um. Validity I think is the, the thing that teach-we as teachers should be most concerned with. We’re, we’re trying to get a, a true picture of where the students are, and where our own teaching is. Uh, if we have an inaccurate picture then, then our decisions won’t be um, as well focused as they ought to be, in fact they may be doing some disservice. So for example, if um, I think that a student um, cannot do certain things but I’m not really sure that I’ve made some assumption about their, their ability. And I may be making instructional decisions based on that perception. If I found out and that could be through a variety of ways, that, in, in, and I would call that assessment, it could be through talking with the student and finding out more from the parent or maybe a more formal way of, of um, uh, having them do something, uh whether it’s take a test or some other performance. Then I would be able to more accurately uh say this is what the students, uh ability is at that point in time and then adjust, the, uh, instruction accordingly. So I think validity is really an essential thing that every teacher should be concerned with. Maybe not with that, that term, but um accuracy of portrayal of students both where they are and where they can be, I think is essential to our, our educational, um endeavors.
As uh, we have more history over time, um, it seems that the, there’s a performance gap between um, Caucasians and non-white students. That’s true in uh every state in most subject areas, with the exception that, that in, in many states Asian students are s-uh sometimes higher or equal than white students in mathematics and in some communities they’re, they’re somewhat equivalent in reading and uh writing. But there remains a persistent gap in performance between Hispanic and African-American students in particular which are a large population uh groups in many states as well as in uh Philippino and some other groups that, that uh uh are present in in some of the other states. I think that there has been a lot of research shown that that this is not a genetic thing. It’s much cle-more closely aligned with social class and poverty than it is with these convenient racial or or ethnic subgroups. The way that research goes is that um when students are provided um, powerful instruction their improvement and their attainment o-often comes up quite a- quite a bit. Um, and that happens at the classroom level. The state can call attention to it, but that type of teaching and training happens uh at the classroom school and district level. The instruction is providing people of the, the right um-the right types of instruction and um I think what people are finding is that it’s easier to keep people from falling behind than it is to close the gap once it happens. The other thing I think that’s coming up that’s really important is that individual teachers um are key that they cannot do everything on their own. That is, many of these solutions are at the school or district or sometimes even a regional level. So for example, um is there a tracking system within the school? What will the counselors play in helping students choose their courses? Uh, we did one study for a state where uh there was a large performance gap between white students, Hispanic students and African-American students. Turns out that the entire, uh variance, that – in –in that uh performance gap was explained by course taking patterns. That is if you knew the course taking pattern then um you didn’t need to know about whether the students were a minority student or not. Uh, but it didn’t answer the question about why the non-white students were not enrolling in the more academic uh, uh course sequences and and for that we would have to look at what their preparation was. Some it may be cultural, um some of it may be personal, and uh also you know something about the, preparation uh that they had in the school, in as much counseling is often a very powerful factor, especially at High School.
I thought – I mean, here- here’s one. Um, in terms of large scale accountability, there’s a question but who gets in- who gets included and who doesn’t. And then there’s a question about is it beneficial or not. I think that um, it differs from state to state, but in, most states, the, the, that I’m working with or acquainted with, most people say, if you can get into the accountability system it will draw attention to where things are and given the right political environment, that can be constructive. And if you aren’t in the picture, then, then uh you’ll continue to be ignored. Now some people want to be ignored. But, uh, in most programs that have traditionally been um not given enough resources or have been given resources but told you’re really out of the mainstream—you’re on the side, um, and so go do whatever uh you want to do, but, but we don’t particularly um care about how, how you fit in with things. Being included in accountability uh has been a good, uh, uh, has risseled (?) in in um good effects. Uh, an example is is um in one state, uh where they are working with um special ed students who are moderately to severely handicapped, they were really being warehoused, they were saying ‘we have money for you and, and you can have someone to be with you, but we really aren’t interested in in helping you learn.’ As those students were brought into the accountability program, then, and the standards were how uh are you learning, how are, are those students learning, then um, the, it wasn’t about funding anymore, it was really about learning and that has been positive thing. Um, so, I think getting that-that positive political environment where it is about learning and not about blaming, but also about including everyone in learning, is uh, is uh important thing. That’s, uh, I think true for teachers of, of, of English language learning students as well. There is a distinction between assessments of proficiency in English, which is more of uh an issue about uh inclusion than there is, that’s different than the issue about uh assessment of students performance in content areas. So, uh, can a student um, do mathematics? Or, can a student do science? Uh, and I think that one of the ongoing issues is, uh, for states, whether they care about literacy or whether they care about English literacy. I think that argument uh needs to happen at a state by state level. Uh, if the student can communicate well in Russian, um, do they meet the state standards for uh reading and writing or must it be in uh, must it really be communication in terms of reading English or being able to write in English. And states vary on that, uh, on where they come down on, on that. But I, I think that uh, teachers of, of English language learners uh, need to both look-have an eye on the state because uh the state does have, have quite a bit of power and, and uh responsibility on setting common standards and uh individually and within their professional organizations and circles are saying what do these students really need, and uh, helping students be as successful as possible.
Ok. Uh, I think and idea that is especially relevant to um teachers of English as a second language or English language learners is that there’s a lot of research going on now about the effect of language on students ability. Now this like a straightforward idea, but um actually, people are, are just starting to do serious research on large scale assessment. One is, if you, if you change the language, how does it effect the performance? That is, is, is, uh, is there really anything that is language independent? And, and I think that language is so powerful, it’s really, find-it’s finding it hard um area where we can say ‘you can really do a math test that um is uh doesn’t depend on, on English to some extent or if it’s presented in another language, Spanish or Russian, does that effect the language? You have to be very careful and very sensitive about how that’s presented. I think that’s true of the classroom level. That’s true for instruction, that’s true for assessment. How we present things, whether it’s in the students um native language or in, in English if they’re trying to learn that, uh, can make a big difference. And it, and so we need to be very careful just as uh, we would expect a test developer or a test publisher to be very careful. We need to take that, a similar type of care in our own um classroom action so that we don’t present a student a task and then conclude that he or she can’t do it or can do it when really, our, our translation, or our presentation of it may have gotten in the way. Another example of the research that’s coming up is, um, because of the need to um include English language learners and large-scale assessments for state accountability, people have been trying to simplify the English presentation in um tests. The general research finding has been that that’s made a big difference. It’s made a difference not only for the English language learners, it’s also made a difference for the native speakers. And uh, that’s probably something that’s good instruction. If we, are careful about what is that we really want to know and present it in a straightforward and um uh simple a manner as possible, we’ll probably get better results than if we uh make them too complicated. I think test publishers are finding out, that, that much of what they are doing, really could be simplified and it would benefit all the, the students who are taking the tests and not just English language learners.
I’ll mention this… I don’t know how relevant it is to your audience. Um, some states that we’re working with are, are entertaining the notion of, of more comprehensive assessment systems. That if there are some things that the state is responsible for, there’s some things the districts, or schools, or individual teachers, or even students themselves may be responsible for. And the ideal there is that there are some standards that cannot be assessed well in a large-scale assessment. In addition, there are some standards that cannot be specified well at the state level. Uh a common example speaking and listening can’t be assessed well and often those standards are, are set primarily at the local level. So, some states are, are really working and many districts and schools are working at how those standards should be elaborated and what should the evidence mean. And schools that are, are taking it seriously, w-are doing things like setting up, a, a graduation requirement or a um, um promotion opportunity. Or it may just be an endorsement thing of --- here’s a super project that students can be involved in to really show what they can do. And then they’ve found that, uh, that has been valuable in, in uh usage for things such as college admissions or to take to employers, or to be part of a cumulative record. For the students it’s just a super uh learning experience. Something that they can be, be proud of. From what I’ve seen of those um projects, these comprehensive uh assessment systems, they really get at things that teachers value, parents value, uh community members value, and students value, often more than just a test score. Um, I think that if we, were clear in our own minds what we really valued and what we hope for and wanted students to be able to do and then said, “what would be good evidence of that?” Um, “How could we show it to someone else?” And then the instructional questions, “What are the opportunities it would take in order to, for a student to be able to demonstrate that and how can I help the student get there? How can they develop those talents?” I think it would strengthen our, our instruction across the board.