Sharif Shakrani

SHARIF SHAKRANI

Um, my name is Sharif Shakrani. I’m the Deputy Executive Director of the National Assessment Governing Board in Washington DC.

It’s very important for us to include all students eh---in our sample, because the National Assessment for Educational Progress a, is designed to develop and estimate what American students know and can do. And by law we are required to assess in three grades, four, eight, and twelve, and we select a national sample from these three grades. And in the national samples we have students who are general education, students with limited English proficiency, students with disabilities, and students who are gifted and talented. And it is imperative that these students participate in the assessment as much as possible in order for the result to be representative of the nation. If we end up with excluding a lot of students from any one particular group, then we don’t have a good sample. And if you don’t have a good sample, then the results would not be valid and reliable for the nation as a whole.

In the past, a, quite often, a, for students with disabilities, a, they may have a---a plan and a committee that decide whether the students should or should not participate in the assessment. And quite often that committee deemed it inappropriate for these students to participate in a large scale assessment, or in test, because they felt it may be harmful to their student, to the students, or the results for these students may be lower than other students, and overall it may reduce the average scores for the school or the district or the class. And so there was a, maybe not a, intended a, decision to exclude, but they were excluded for sometime good reasons, as they viewed it on the part of the district. But a, we thought that students, if they are given accommodation, they can participate affectively in the assessment. The purpose of an assessment is to find out what students know and can do. So if you provide accommodation that would help us find out what the students know and can do, then that is the purpose of the assessment. And accommodation is not affecting their scores. And so we have learned over the past few years, that there are certain accommodations, very easy to do, that will bring about a significant increase of participation of the students with disabilities and limited English proficiency.

Well, a, for our assessment, which is a survey assessment, and students a, are asked a variety of questions, some of them are multiple choice, some of them are open-ended, some of them students have to write, maybe, half a page or more. It’s derived to assess certain content and topics that we as educators, and as a society, value. And we consider essential for students to know in order to progress in their educational ladder and in order to become contributing members of society. And that certainly we want for all students, not only students who are in general education, but students with disabilities and limited English proficiency, and students who may have special needs. So when we want to find out about these students, what they know and can do, the questions are not geared for a specific subpopulation, the questions are geared toward all students. The accommodation are intended to assist the students in giving us the information. I’ll give you a very simple example for students who’ve disabilities. Quite often what students ar---mi---commonly referred to as special education students may have a special classroom where they receive their instruction, and they feel comfortable in that setting. So what we suggest for an accommodation is that the test administration be conducted in that classroom so that they do not leave thei---the normal environment of where the instruction is given, and put into a new environment that may affect their ability to answer the questions. Now another thing is that some of these students may not be able to read the questions or answer them in the same speed as other students, so we may give them extra time to provide with the response. Our test are not speed test, they are power test. They are intended to find our what you know and can do. And so if a students require more time to be able to read the questions and to provide the answers, then certainly we should do that. Because that will tell us what they truly know and are able to do.

A, or tests are not speeded test, a, a speeded test may be like a norm reference test. A, the reason, a, it’s a speeded test because it is given a specific time, because in the past they have developed norms for that test. And the norms, as a same amount of time, should be given to all students in order to compare to the norms you have to use the same conditions or the testing. Ours is a power test. It’s intended to delve in depth to what students know and can do. And because our test have open-ended items that may require the students to write, to think, they may have to write a first draft of the response and then a---a, improve upon it, or refine it. So, we---a---we make sure that they have ample time to do the testing. And if a student require more time than others, then we allow them to do so. Because we do not norm the test using the time as one of the criterion for a demonstration. We, a, we allow them to use extra time if they need it.

Research on accommodations a----is---relatively new, not much was done before. Because a, of the a, allowance for exclusion in the past a, there was no reason to do a lot of research about accommodation. But with the advent of the IDEA, Individual with Disability Education Act, that required that all students participate equally in assessment, a, this is a part of their educational program. Then educators, the researchers, and test development specialists and cycle attrition, these are the people who work with testing, start look at this a concept of---thi---es---inclusion and exclusion very seriously. So, since 1990 a, in the education field, we no longer talk about exclusion, we start talking about inclusion. And in order to provide a meaningful inclusion you must provide accommodation. So that if you take, for example, a student who’s mother tongue, or home tongue, is other than English, and may have been in this country for a short period of time, and you want to test them in mathematics, what do---what are you trying to find out? Is it ho---ma---how much mathematics they know and can do, not how much English they know and can do. So do---you do not want to have two variables in the assessment, so you may consider providing an accommodation by having the test administered in their home tongue, or in the language that they are familiar with. Because you can derive a good math score from that. But if you give it to them in English and they do not do well on that test, you don’t know if they do not know the math, or they do not know the English language. A, another example: if you take a students who has some visual impairment, and a, and their normal instructional a, procedure, they may have a large print, they may have a magnifying glass, they may have the teacher or the aide read a part of the instructions to them. But if you bring them to a testing situation, and you do not provide accommodation, then you don’t know what kind of information you are getting out of them. They may look at the test and not be able to read it. And therefore, the information you are guar---gathering from them is basically worthless. So you must provide them with an accommodation, either by having a large print, provide them with the same accommodation that is given to them during their instruction. And this way they are---an---a---in a similar of---a setting and familiarity as were their instruction. And the information we gather from them will be me---will be meaningful and relevant. And so these accommodations are not changing their scores, they are giving you a more reliable and accurate information.

What we have learned a, about accommodation is different for students with disabilities a, then for students with limit English proficiency. A, for students with disabilities, they are a very diverse groups. Some of them have some physical disabilities; some of them have some, maybe, mental or emotional disabilities that impact their scores. So the accommodation vary, depending on their disabilities. But what we have discovered, that simple accommodation such as: providing the students with extra time, or testing them one on one, or reading the instruction to them, or testing them in the same setting where they received their instruction, will bring about inclusion of eighty-five percent of these students. Eighty-five percent. And the other fifteen percent who are not included, many of them a, would be in a special education program that provide different general curriculum. These are students with severe mental imper---impairment that may not be able to learn the same mathematics as the other students because of their disability. So, a, these accommodations would not benefit them. But certainly these simple accommodations provide tremendous impact on inclusion. And as we all know, whether it is at the classroom level, at the building level, the state level, of the national level, assessment is an integral part of the educational process. Provide information to the teachers, to the parents, to the system, about what these students know and can do, and what they need, what a----remedial training, a change in program. Not only for that individual student that’s tested, but fu---for future generation that are coming to, to the classroom, because we will learn from these experiences to build a better assessment. And kind of a, an assessment instructional curriculum, kind of like troika, you know, or a threes---three st---three, a, legged stool, they must work with each other and if we do not have information about assessment, a, it is not gonna help an individual or help the educational system in general. So it’s important to have a valid and reliable information. And what we have done, term of research at the national level, I think a, without a shadow of a doubt, help the state level and at the district level. But also, we have learned, at the national level, from what goes on at some very innovative practices at the state and at the district level. Let me give another example for students with disability. A, sometime we, a, ask students who may have a, problem with eye to hand coordination. Now a student, general education students may be able to read the question and be able to write their answers. But a students who have some problem with eye to hand coordination may affect wh---which a, bubble they, they select, and in may be in the middle, and this is scored by a machine. The machine doesn’t recognize what’s between the two bubbles, if it’s not at the, in the bubble, it’s gonna treat it as wrong. Well that’s not fair for them, so the teacher, the resource teacher at the school, being aware of the disability of that particular student, then can say: ‘I will read you the test, or you read the question, and tell me what you want to answer.’ And they can record it for them. Now we rely on the honesty of the teacher, they will---they will---a check the appropriate answers, but, when you analyze a test then you truly have information about what that students know and can do. We do not want the testing procedure to determine what the score should be because of the disability of the student.

The United States is a very unique country, that, we have people coming---a---who immigrate to this country from many, many places. We have more than a hundred and twenty-three languages spoken in---in our schools. In Los Angeles, for example, ninety-seven languages are spoken by students. It’s very easy to say; let’s translate the tests to them, that’s not at---a easy task at all, in fact it may be a bad task for some students. We ge---we get some students, for example, who may be the children of migrant workers, so they are working part of the year in Oregon, part of the year in Michigan, part of the year in Texas. The educational system in these states are different. One may have a program of teaching them in Spanish while they are learning English, another may have a prohibition against teaching them in Spanish, such as in Florida, for example. And these students are moving around, so to assume that they have the same base of knowledge is incorrect. So that, for example, for a---students, Hispanic students, in the fourth grade, whose English i---is a limited, to give him Spanish translation is not an answer. He may not have learned in his home, or before, what is a term for a parameter, or a radius. So to see it in Spanish is not gonna help him. So one way you may consider is to give him a bilingual, a, version of the test, English on one side and Spanish on the other side. So that if you ask him a question like: ‘If you had four dollars and you spend one dollar and sixty-seven cents, a, how much do you have left?’ He can, he can read the words in Spanish because that’s what he understand. But then if you ask him a ques---a question about the radius of a circle, he may, if he doesn’t understand the word radius, he may turn to the other side and see what radius mean. Because he was---he received instruction in English on that word. So that is very helpful. But the minute you do that, then you must give the students more time, because the students is reading twice as much as the general education students, so you have to accommodate them for that. Now there are some students that, a, they come from countries who, a---a, their language may be, a---an oral le---language rather than a written language, a, the Mung, for example, of Cambodia, it’s---they have an oral language. So there’s not a written language, so how do you write it for them. A, then you may get some a, students who may have been, because of war in their countries, they have been out of---out of school for a year or two, and they may not have learned the concept in their, a, mother tongue. And so providing them with a translation is not a solution. So what can you do, there are---are other methods of doing, a---th---the assessment. You can do one on one assessment, and then you, as a teacher, if you recognize that the students is not familiar with the word, may be able to use a glossary or a dictionary so that he has an understanding, or she has an understanding, of what the mir---meaning of the word or the context of the question. But it may be for some students, a, the assessment is done by a teacher rather than by a test. But, a, what I, I am concerned about, and we have done lot of research on this area, is that the translation has to be meaningful. For example: Hispanic students, who constitute about seventy-eight percent of students with limited English proficiency in this country, come from different background. Some of them have a Puerto Rican origin, some of them have Cuban origin, Central American, Mexico, and some of them are, a---m, like the children of migrant workers, are born and raised in this country. So t---to assume that one translation fit all may not be appropriate. So we have to make sure that th---the translation is done in a manner consistent with the population that you are assessing. And that literal translation of Spanish may not work for any one particular group. And so care has to be done with that. Now, what do you do in a place like Los Angeles where you have ninety-seven languages and maybe one language is only understood by five students, do---can you go through the expense of developing a translation? Well, no, then you use other method, you use glossary of terms, use maybe a simple dictionary that they use in their instruction, and this provide meaningful accommodation. But all of these must be done in conjunction with other accommodation, such as extra time, one on one testing, reading the test items. Sometime a student a, may know what a radius is when he or she hears it, but they may not be able to read it very well, because, for students who are learning a new language, a, listening skills are by far more powerful than the reading skills.

Well, a, the National Assessment for Educational Progress is a national test, but we do not have a national curriculum in this country. So the question is: what should NAEP test? So, what the congress of the United State have done, a, they said: we will establish a board, a, call the National Assessment Governing Board, which is where I work, and it has twenty-six members. It has two governors, two state legislators, two chief state school officers, teachers, principals, parents, business representative, basically a national representation. And congress asked the board, the National Assessment Governing Board, to identify what students should know and be able to do, (pause) in mathematics and sciences and reading and writing, all the academic area. But congress told, a, the board to do it through a consensus process, that mean, involve people from across the country, getting them together and let they discuss which stu---what students should know and be able to do. And so that’s how we define the framework for NAEP. These people get together, they discuss what students should know in mathematics at the fourth grade, at the eighth grade and the twelfth grade. The need for progression from what they know from the fourth to the eighth to the twelfth, a the same thing with geography, history, civics, a, reading, writing, and these frameworks then---are reviewed and approved by the board and are distributed widely across the nation. They---they form the basis for the assessment, so we develop test items to measure that framework, and then these test items are used in the NAEP assessment.

This is a very important question. A, the purpose of the test quite often dictate what kind of accommodation should be provided. NAPE is what is referred to as a low stake assessment. Nobody’s gonna be denied graduation, nobody’s gonna be denied promotion, no state or district gonna lose fund as a result of the test. It’s basically like a thermometer to tell us if we are doing well, if we are moving in the right direction or not. And it’s a group estimate, it tells us what the United States is doing, what a---a male students at the fourth grade, female students, a, African American students, Hispanic students, we do these---the segregation. But we never report on a individual student result, we are prohibited by law from doing that. We provide group estimate that are used to gauge the progress of American education. So, for example, a, the United State saw the need to empath---emphasize math and science, a, since the nation at risk report. And so NAEP is an excellent instrument to gauge whether make---whether we are making progress in math and science. Whether we are closing the gap between the performance of males and females in math and science. Whether we are closing the gap, the performs, between white students and African American students, it’s an exe---excellent instrument. Then, in 1990, congress allowed states to participate in NAEP, and it’s the same instrument used at the national level is also used at the states that volunteer to participate. And, a, approximately forty to forty-five states do volunteer to participate. So NAEP becomes the only instrument in the country that allow one state to compare their performance to another, and allow one state to compare their performance to the nation. There is not other instrument that can do that. So, as it related to the issue of accommodation, because we select the sample, it’s imperative upon us to make sure that as many people who are selected in the sample, these are students, participate in the test, so that the estimate become valid and reliable. But let’s suppose that we cannot include three or four students in a particular state because of their disabilities, or because of their---a---the committee report says they cannot participate, it’s what they call the IPE committee. The Individual, a, Program for Education, for the students with disabilities, they say they cannot participate. It is not gonna affect the overall estimates, because our sample size is about ten thousand students. But that’s different at the district and school level, where every student is tested, then they produce individual report, classroom report, school report, and district report. So if my child, a, is a students with this, a, disability, for example, if my child has a leaning disability (pause) and he is not tested, and I have no information about how well he, or she, is doing, then certainly puts me at disadvantage. But more importantly it put the child at disadvantage. So we have to make sure that we’re, when there are individual tests and individual results, that accommodation are provided so that the students can be tested. And these accommodation should vary depending on the disability or the need of that particular individual, whether students with disabilities or limited English proficiency. And so, a, the schools, a---at---or the district or the state may not have, may not have the resources to test all possible accommodations, and they can rely on the experiences that we have develop, because we, we have the funding for a, government provide the funding, to do all these experimentations. And I must say that a---a the states have been doing a better job of including students in their assessment over the past five years than, and the seventy years before that.

A, I have a very strong point of view on this issue. Um, and I’ve expressed that vividly in many, many conferences across the country. There are---lot of research about what type of accommodation that will get students to take the test. That will get the gatekeepers to allow the students to participate. But the research is very weak about what do these, a, test result tell us about the students knowledge and ability. So, for example, if we are to look at the impact of giving the students more time, how does that impact on their overall scores? Which areas---a---would improve their score or not make any difference? Why would time, extra time, provide more information? For students who may need to, because of their a---a specific disability (background chair creaking) may not be able to sit still for one hour and be able to tell you at the end of the hour what they really know, in the same way they would at the beginning of the hour. If we chunk the time for the testing, if we test the students, for example, for a half an hour, and then another half an hour, or maybe ten minutes, but not do it all in one time. How does it affect the scores? Very little research is done in that area. A, the student teacher, resource teacher, or someone else may say; this students, a, attention span is limited to this much time (pause) ans---and therefore go a---on chunking the time, but it not research about looking at the results and being able to compare it. What would be a more appropriate, a, breaking of the testing time for a particular students and for other students depending on their disability? Very little research done on this area. Another thing is that a, some states, for example, they read the test to the student, a reading test. Well a---is it now a reading test or is it a listening test? Little research have been done on this area. What is the difference between the two? If a---my child, wh---if---a he or she has a special need, and a, they have some visual impairment, to some items that rely on a picture to answer the question, disadvantageous to these students. We have to be cognoscente of students with disabilities from day one. From the time we start writing the items, we should not wait until the test is develop to determine what kind of accommodation we should give. Because the test item may, in fact, be a detriment tos---to some students. And the---they would have been no reason to write a different item that reliably measure the objective, but does not discriminate against certain students with disabilities.

A, I’m a, I’m a psychomatrician, and I’m a---very much interested in a---making sure that students are included. I have----through all of my professional career fought for additional---a---inclusion. Until 1990 I was very disappointed. A, I saw people with good intention excluding their students. (Pause) But, a, since 1990 I have seen an improvement, and over the past five years, a, I feel positive and I am, am at ease that we are seeing more and more research, not enough, but more and more research into inclusion and into the analysis of the data. But, and I see more, a, of the testing companies a---addressing the issues of students with special need, both the---a---those with limited English proficiency and students with disabilities, from day one of the development process. Until 1995 they were not considered. So that---let me just give you an example of what was happening at the national level. I described to you how we develop our test for NAEP. We have the framework and then we contract with a company and the company brings people who are specialists in this area to write the items. And ‘til 1995 they didn’t have a person whose specialty was special education, or students with disabilities, and how they learn. They had content specialists. In 1995 we brought in the expert in the area of students with disabilities. (Door closes) And their input has been profound, it improved our test, and if you improve the tests, at that point, then it decrease the amount of accommodation that is rev---needed later on. And I think that the advent of---a computer, and being able to use the computer and technology, is going to help tremendously in the future. Because it allow the students to move at their own pace. It allow them to see, with clarity, the items. They can make them larger, they can make them smaller, they can look at a particular part of the test, there is a picture or a graph associated with it. A---technology allow a tremendous variability in addressing individual needs. So, I---I---I am very positive that in the future we are going to be able to include more students meaningfully and effectively in the assessment.

Well---a---I think in the future we’re gonna see more high stakes assessment. I think legislators, state boards of education, local boards of education, are going to hold teachers, a, principals, assistants accountable. And accountability is quite often associated with assessment. And so as these assessment become more high stakes, and by high stakes I mean that---a they are linked to the students being able to be promoted from one grade to the next, or to graduate from high school, or funding to the district or school, a---or incentives. And a it would require us to insure that the assessment is reflective of what the students are instructed on, what’s in the curriculum. And that is very reasonable, but at the same time we wanna make sure that the test will be able to derive from the students the maximum of what they know and can do. And the test should not be a hindrance. That is of interest to the schools, to the parents, to the board of education, and to the system, and to society in general. So that---a whenever there is a high stake testing (Pause) the need for meaningful accommodation is by far more important than at with NAPE or othe---other low stake assessment---a---those intended to prod---provide---a picture of what the st---what the state is doing on a particular area or not. And---a whenever you are dealing with high stakes that produce individual results the involvement of the teachers who are teaching these students is extremely important. Because they tell you about the specific needs of these students. You know one thing we tell everybody; we say; we allow NAPE the same accommodation that the students receive during their instructional program in their school, because they are used to it. But how do we know about these accommodations? We know by talking to the teachers, so the teacher has an integral part in assuring that the accommodation are provided, that are meaningful. As---a teacher, for example, will tell you, giving a translated version of this test to Julio is not appropriate, giving a translated version of this test to Miguel is very appropriate. He has been in the system longer, he is, in essence, he, well he does not know the English, but he is proficient in Spanish. So---a---how do we know that when we come and test if we do not talk to the teachers? And so the system---a---that is in contact with the students must be consulted very carefully---a---on testing students with limited English proficiency, or students with disabilities prior to administering the test and accommodation. One solution does not fit all.

I am personally vehemently opposed to what is called: alternative assessment students with disabilities or limited English proficiency. (Interruption)

A, I have a particular point of view on that. A, I am personally vehemently opposed to---a---what is called: alternative assessment for students with disabilities or limited English proficiency. Not because alternative assessment is not meaningful, but what I have seen is that an alternative assessment tend to be easier, a---does not measure the same content that we want for all students, and this harms the student. Sooner or later is gonna catch up with them. And they will be tested on---a part of the curriculum, not the whole curriculum, and they ar---go---the difference between what they know and do not know grow as they move up---a---th---the ladder. And if that is continued then by the time they are able to lo---leave school voluntarily, they will do so. And we wanna avoid that. So that is we consider the curriculum to be for all students, then the assessment must be reflective of that. If the alternative assessment measure the same content, but using a different approach, I have no problem with it. But, in fact, my analysis of many of the assessment that are being used with students with disabilities, tended to be for, what is called, dumming down the curriculum, and we should avoid that at all cost. We would be doing these students a tremendous disservice if we do not---a---have the same expectations for them as we do for the general education students, assuming that they have the same ability as the other students. That we give them that opportunity to learn, and denying them that opportunity to learn is---a---to me, is---is---is paramount to an immoral act.