Reading Assessment and Evaluation Alternatives

I. Definition of Training Piece

A. Purpose for Instructor

Effective programs of reading instruction include assessment at three critical points: prior to instruction for the purpose of diagnosis and placement; during instruction for the purpose of monitoring progress and modifying plans accordingly; and after instruction for the purpose of evaluating outcomes. This training piece will present information regarding instruments and activities designed for these three phases of assessment.

B. Material covered

This training piece will include a survey of research and activities that address a range of assessments that may serve various purposes for an instructor. Some assessments are broad and designed to be used with groups of students (see KWL as summarized in the research section). However, others are highly individualized and reveal specific reading strengths and weaknesses. The most detailed example of this is the Informal Reading Inventory, which is administered one-on-one and requires a student to read aloud.

II. Foundation

A. Definition of Concept and Theory

The best instructional practices and programs are developed from clearly articulated outcomes. Assessment of these outcomes may be very general or very specific according to the needs of the program and the learner. Tasks may be selected from preexisting models or banks, or they may be created to enable students to demonstrate an outcome. Assessment that occurs prior to instruction will establish baselines, identify strengths and weaknesses and inform curriculum planning. Assessment that occurs during the course of instruction will monitor progress and may provide the opportunity to give feedback for the teacher and / or the student. Assessment that occurs after instruction will determine whether the desired outcomes have resulted.

B. Summary of Relevant Research

1. "Due to the rapid growth of college reading assistance programs throughout the United States, assessing the reading abilities and reading needs of entering college students has become a fairly standard practice." This sentence introduces one of the most comprehensive chapters on Reading Tests in the annals of reading research. It should be part of every college reading teacher's collection. The topics range from "Selecting the appropriate Test" to a "Review of the Literature."

Flippo, Rona and David C. Caverly, eds. Handbook of College Reading and Study Strategy Research. Mahwah, New Jersey: Lawrence Erlbaum Associates, 2000.

2. KWL is a procedure that can help teachers get a general idea of their students' knowledge about a topic and that can help students connect prior knowledge to a topic. It is a metacognitive activity that asks students to determine what they know, what they want to know and what they have learned after a course of study.

Ogle, Donna. "KWL: A Teaching Model that Develops Active Reading of Expository Text."The Reading Teacher, 39, 564-570.

3. An informal reading inventory (IRI) is "the use of a graded series of passages of increasing difficulty to determine students' strengths, weaknesses, and strategies in word identification and comprehension." It identifies students' independent, instructional, and frustrational reading levels.

Betts, Emmett A. Foundations of Reading Instruction. New York: American Book Co., 1946

4. "This portfolio of tools serves two purposes: first its creation provides us with tools for assessing a range of tasks within the learning domain and setting standards for college-level performance of those tasks; second, the scoring rubrics allow all members of the learning community to describe accurately a performance of a given task along a defined continuum, to name specifically what has been and what remains to be accomplished in order for that performance to match a college-ready standard."

Bloom, Tom, ed. Community of Classrooms: A Handbook for Preparing Students for Reading and Writing in College. Minnesota Community College System, 1994.

5. Some of the assessment that goes on in college classrooms measures how well the teacher is doing. Angelo and Cross have explored this topic in great depth; their expertise translates theory into practice so that teachers can easily utilize or create assessments to fit just about any teaching situation.

Angelo, Thomas and Patricia K. Cross. Classroom Assessment Techniques, 2nd edition. San Francisco :Jossey Bass, 1993.

III. Benefits

A. Teachers

For most college teachers, time is a precious commodity. No one wants to spend time teaching what has already been learned or to skip material or concepts that provide the foundation for greater learning. Effective assessments yield accurate placement and support maximum progress. Wasted time can be reduced. The results of assessments can be used to design curriculum that responds to students' needs and even interests.

B. Students

Students benefit from assessment, too. When they are assigned to read text written at their instructional level (resulting in 95-98% word recognition and 75-90% comprehension), they make the most reading gains. Accurate placement and ongoing monitoring create a learning environment that will allow students to progress efficiently.

IV. Implementation

A. Exploration exercises for Instructors

1. Informal Reading Inventory. This can be used as a follow-up to standardized norm-referenced or criterion-referenced assessments. They can also be helpful for monitoring progress during the course of instruction. Some IRIs are available through commercial resources, but an IRI is as much a process as it is a product, so instructors can administer an IRI with any piece of text once they have learned the technique. Try your hand at evaluating whether a student has been able to read a passage successfully.

Refer to the passage "Visualize Your Ideal Future." It has been marked according to the oral reading performance of a student. A code sheet is provided. Finally, a set of responses to comprehension questions has been recorded. Based on the criteria for reading levels, determine whether the reader has performed at the independent, instructional, or frustration level.

Visualize Your Ideal Future by Skip Downing

(On Course, Houghton Mifflin Co., Boston, 1996)

We human beings pursue what gives us pleasure. And we do our best to avoid what causes us pain. Use this psychological truth to your benefit. To make or strengthen a commitment, visualize yourself accomplishing your dream and imagine the pleasures you'll experience when you do. Consider also the pleasures you'll experience on the journey to creating your dream. Let these positive outcomes and positive feelings draw you like a magnet toward your dream. You see, all of our accomplishments are created twice. Before we can create them in the world, we must create them in our minds. Some years ago, I happened to glance at a three-ring notebook carried by one of my students. Taped to the cover was a photo showing her in our college graduation cap and gown. In the photo, her face was aglow with success.

"Tell me about that photo," I asked. "Have you already graduated?"

"Not yet. But that's what I'll look like when I do."

"How did you get this photo then?"

"My sister graduated from this college a few years ago," she explained. "After the ceremony, I put on her cap and gown and had my mother take this picture of me. Whenever I get discouraged in school, I look at this photo. I imagine myself walking across the stage, receiving my diploma from the college president. I hear my family in the audience cheering for me, just like we did for my sister, and then I stop feeling sorry for myself and get back to my school work. This picture reminds me what all my efforts are for."

A few years later, at her graduation, I remember thinking, "She looks just as happy as she did in the photo. Maybe even happier."

Life will test our commitment to our dream. To keep our commitment strong in times of challenge, we need a clear picture of our desired results. We need a mental image that, like a magnet, will draw us steadily into our ideal future.

Here is a sample oral reading performance of the same passage by a student:

Here are the student’s responses to the Comprehension Questions:

Visualize Your Ideal Future Worksheet

In 100 words: 7.1 sentences and 154 syllables.

Readability according to Fry: 9th grade

COMPREHENSION QUESTIONS AND RESPONSES

1. What is the main idea of the selection?

"We all face difficult times and we should strengthen ourselves ahead of time by having a clear picture of what we want for ourselves."

2. Why did the student in the story tape a picture of herself on the cover of her notebook?

"To remind herself what she looked like when her sister graduated."

3. What order or pattern does the writer use to organize the details of this selection?

"Narration."

4. Why does the author state "all of our accomplishments are created twice?"

"Because he wanted to show that accomplishments are hard to come by and you have to really work hard to get what you want."

5. Why does the author compare a mental image to a magnet?

"He believes that the image acts as something that will draw people toward success, just as a magnet draws metal to it."

Determine the student’s reading level by using the following formula:

# of Word Recognition Errors in 100 words __________ = __________%

# of Comprehension Errors __________ = __________%

INDEPENDENT? INSTRUCTIONAL? FRUSTRATIONAL?

This student scored at the instructional level in word attack but at the frustration level in comprehension. Some college level developmental readers have great strength with word attack, but their ability to comprehend is poor. They need guided instruction in reading strategies.

B. Student Exercise

Assess what students know about their knowing by using the following instrument. The following answers should be marked "No," indicating they are poor strategies: 3, 7, 8, 12, 16, 17, 21, 26, 27, 30, 34, and 35. Students who score below 75% may have difficulty making progress in a reading course. They have a limited set of strategies from which to choose and may need direct instruction in ways to respond to reading difficulties.

Assessment of Metacognitive Awareness in Reading (Form 3)

DIRECTIONS: Read the scenario that introduces each section. Respond to the choices by using your scantron answer sheet. Mark "A" for "Yes" and Mark "B" for "No."

A. Vickie is a first semester college student. Her professors give the students difficult reading assignments. Think about what you typically do before you start to read something challenging for a class. Then answer the following questions.

Would you advise Vickie to:

1. think about the purpose for the reading Yes No
2. look over the material's headings Yes No
3. just start reading Yes No
4. think about what she already knows about the topic Yes No
5. read over the introductory notes Yes No
6. make predictions about the content Yes No
7. count the number of questions that are at the end of the assignment Yes No
8. memorize the title Yes No
9. think of a personal experience that relates to the topic Yes No

B. Vickie begins the reading assignment. While she is reading she realizes the text is somewhat difficult for her to understand. Think about what you typically do while you are reading difficult material.

Would you advise Vickie to:

10. mark confusing passages so she can ask about them later Yes No
11. make a connection between the reading and her life Yes No
12. read the assignment from beginning to end without stopping Yes No
13. periodically ask herself, "Does this make sense?" Yes No
14. look for transition words and organizational patterns Yes No
15. adjust her pace Yes No
16. assume all sentences are equally important Yes No
17. skip the parts she doesn’t understand Yes No
18. underline or summarize periodically Yes No

C. Vickie also notices that some of the vocabulary is unfamiliar. In fact she wonders if it is causing her comprehension difficulties. Think about what you typically do when you encounter difficult or unfamiliar vocabulary.

Would you advise Vickie to:

19. use the words around it to figure it out Yes No
20. look it up in a dictionary Yes No
21. skip it and keep reading Yes No
22. consider whether the word has word elements with meaning Yes No
23. sound it out to determine whether it may be familiar Yes No
24. ask someone else what the word means or how to pronounce it Yes No
25. stop reading until she knows the meaning of the word Yes No
26. copy the word down in case she runs into it later Yes No
27. substitute familiar words that might fit the context for unfamiliar words Yes No

D. Vickie has completed her reading assignment. She knows there will be a class discussion on the selection and that questions about it may turn up on an exam. Think about what you typically do when you have finished reading an assignment.

Would you advise Vickie to:

28. reread any notes, underlining, or markings Yes No
29. make a study guide Yes No
30. reread the assignment from beginning to end Yes No
31. review the material immediately Yes No
32. write a summary or an outline Yes No
33. note anything she still does not understand Yes No
34. consider her assignment complete Yes No
35. delay a review of the material until the day before the exam Yes No
36. create some questions that might appear on an exam Yes No

This assessment is modeled after a scenario described by: El-Hindi, Amelia E.(1996). Enhancing metacognitive awareness of college learners. Reading Horizons, 36, 214-230. Permission for use granted by Reading Horizons.

Additional resources: Miholic, Vincent. (1994). An inventory to pique students’ metacognitive awareness of reading strategies. Journal of Reading, 38, 84-86.

Miller, Wilma (1995). Self-Monitoring (Metacognitive) or Self-Correcting Assessment Device. The Center for Applied Research in Education, West Nyack, New York.

C. Skill Connections

1. Vocabulary Development: A student’s knowledge of vocabulary and methods for determining the meaning of unknown words greatly influences his/her reading placement as well as his/her reading comprehension of college textbooks. Assisting students to develop strategies for learning vocabulary increases their likelihood of success in college. For more information, visit the Vocabulary Development Module.

V. Frequently Asked Questions

Q: Why do I need to do assessment at the beginning of the semester if my

college has already given a placement test to the students?

A: Placement tests are gross instruments. They will help put the student in the best general group. However, they do not always reveal diagnostic information. Also, at least two measures should be used whenever placement is the concern because there are many factors that may cause a student to score poorly on one particular test.

Q: My school uses a timed assessment for placement and evaluation. What do I need to know about timed tests?

A: Timed tests are controversial. Because they are normed, the role of rate is important. However, ESL students tend to score more poorly on timed tests than they do on untimed tests. If you have access to the actual responses, take advantage of that and check: Did a student score low because he or she answered many questions and answered them incorrectly? Or did a student answer a low number of questions but answered them correctly? The latter student may simply be a decent but slow reader who needs some instruction on adjusting rate.

Q: My college is considering the use of a computer adaptive test. How do these work?

A: These tests present a passage for a student to read. If the student responds to the passage successfully, then the next passage will be more difficult; if the student is not successful, then the next passage is easier. This process is repeated a number of times until the student has reached a level that seems neither too easy nor too hard. This branching yields very accurate scores.

VI. Helpful Resources

A. The Burns and Roe Informal Reading Inventory (Houghton Mifflin Company, Boston, 1999) contains background information, instructions for use, and graded passages for K-12. It has been cited in the research literature (see Flippo and Caverly) as a good IRI for college students. If you want to learn how to do one or how to make one, refer to this book.

B. Web sites

http://thetwocrows.com/ahclem/
Links to a wide range of websites related to exam skills

http://www.brazosport.cc.tx.us/~lac/studyand.htm
Ways to improve your test taking skills

http://www.Collegeboard.org/accuplacer/html/accupla1.html
Learn about the College Board's ACCUPLACER Online program. It has an explanation of Computer Adaptive Testing.

Workshop Information
www.facultytraining.com to attend a workshop on this topic or bring one to your campus, visit this site or call Faculty Training at (800) 856-5727.