Repair Kit: Fix 13 Summative Assessments Only in Final Grades

Fix 13: Don’t use information from formative assessments and practice to determine grades; use only summative evidence.

When I began reading this book I didn’t understand how you could calculate grades solely on summative assessments.  Now I’m thinking counting practice would be inaccurate as not every child gets the knowledge the first time; we need to allow them a second chance or third or fourth.  I think I would still track formative/practice/homework just to see how students are progressing and to compare to summative scores, but I would not count them in the final grade.

Small aside – The quote at the beginning of this chapter comes from a nearby school district.  I just thought that was kind of cool.

“It is important that teachers, students, and parents recognize that learning is a process in which learners increase their knowledge, understanding, and skills as a result of effort, instruction, feedback from teachers and peers, and self-assessment and adjustment.” (page 106)

This is the explanation we were given about our son’s standards based report card as to why he may have a 2 one quarter, 3 the next, and then back to a 2.  As additional instruction is taking place the level of knowledge being assessed is different therefore while he may have improved in second quarter maybe he’s struggling with the higher level in third quarter and so drops back to a 2.

Figure 5.1 lists three purposes of assessment, in short

Diagnostic – what is the child’s starting level

Formative – assessment for learning – provides direction for instruction

Summative – assessment of learning – used to see where a child is at the end of learning

“Feedback has to be descriptive not evaluative” (page 109)

This is what I have not done on a regular basis.  I need to make the 2 Likes and 1 Suggestion that I do and have my students do for each other a habit for myself and not just as a peer assessment on occasional assignments.  I wonder if it would be conflicting with the design of this fix to have a “format” feedback sheet.  If I could circle certain things like on a descriptive scoring guide and then add just a few sentences of tailored feedback it would be easier to do this for every student on multiple assignments/assessments.

“Careful consideration has to be given to the purpose(s) of homework.  Sometimes homework requires students to show what they know by extending or integrating their knowledge and understanding through projects or assignments done partially  or completely outside of the classroom.  This is clearly summative assessment  and is legitimately part of grades as long as there is careful monitoring to ensure that it is the student’s own work… Most often, however, homework is a practice of whatever was learned in class that day – any assessment of this work should be considered formative.” (page 109-110)

This makes sense!  I finally understand what they mean about not counting formative grades in final grades and better understand WHAT should be counted for final grades.

“It becomes an issue of compliance so it really doesn’t matter who does the work” (page 110)

Oh is this ever true, this would be why some teachers get homework turned in that is clearly the work of the student’s parents and not the student..

I’m paraphrasing sections from pages 111 and 112 here that really summarized this whole idea for me: homework is final grade is harmful for struggling kids who may avoid failure by not trying.  Students who ace tests don’t do homework because they don’t need to, they don’t need the practice and so they get lower grades that are an inaccurate representation of knowledge.  Using the mean to calculate rather than replacing scores or looking at more recent evidence of growth means that early low scores may mean final grades are inaccurate representations of student knowledge.  Teachers should create an assessment plan that begins determining the desired results/learning targets, writing a summative assessment to determine what students learned, then creating a diagnostic assessment to determine what knowledge students are starting the unit with, and finally determine what formative assessments will be used.  There should be a direct link between the formative and summative assessments.

“Once teachers have become clear about the appropriate uses for formative and summative assessment and abandoned the practice of including everything in grades, especially homework, it is acceptable to consider formative assessment evidence when determining grades.” (page 113)

OK again clearing things up for me.  I can still look to see if there are significant differences between homework scores and summative scores.

Student Involvement: Students have to be retrained to understand practice and performance and trained how to self-assess.

Teacher Vignette – “…students making a first attempt at practice new learning should be permitted to practice and take academic risks without having it cost them in the grade book.  We also emphasized that if everything counts, then the student’s focus would be on compliance (meeting the deadline) through any means possible, including cheating.” (page 116)

Wow is this an important statement to consider.  It really makes me think and it goes so well with fix 4.  Just another way to prevent cheating.

“The fear with implementing risk-free practice, of course, was ‘If I don’t grade it, they won’t do it.’” (page 116)

This is a very real fear for me, if students choose not to do the practice I won’t know if they need it.  I suppose that falls under behavioral consequences for incomplete work though, according to the vignette though “when practice is not counted in grades students complete the work more often, took more chances, and pushed themselves to learn.”  This gives me hope but I still think there would be a transition period.  Children would need to see that they do better on assessments when they do the practice and how do you explain that to parents.  Something to think about…

I may have to copy the guiding principles to help my set up my classroom policies and share with parents.

Advertisements

Repair Kit Fix 10: quality assessments

Fix 10: Don’t rely on evidence gathered using assessments that fail to meet standards of quality, rely only on quality assessments

This is a fix I’m particularly interested in reading because I’m not sure I write quality assessments and I want to know what the author suggests the identifiers are of a quality assessment.

“To be quality an assessment must be accurate.  Accurate assessment requires attention to three questions:

1) Why are we assessing?

2) What are we assessing?

3) How will we assess it?” (page 82)

Four criteria/features of test design quality are listed in the chapter.

In short:

1) Match question style to context, make sure questions will gather the evidence needed to show student achievement.

2) Tests must have good questions (specifically multiple choice) not bad ones.

This particular criteria could stand to be more specific.  I attended at least two different workshops where I learned to write quality  multiple choice, well at least for DOK (Depth of Knowledge) level 2.  I never could find anyone who could explain to me how to write a good DOK 3 level question.  The best answer I ever got was “Write a DOK level 3 constructed response then write answers to turn it into a multiple choice/selected response.”  The problem was every time I wrote the answers it seemed to become a DOK 2.  Without further help I think many teachers could struggle with writing a good question.  It’s been a long time since I’ve been in college but I don’t ever remember being taught how to write assessment questions, most teachers just used the tests that came with the textbooks.  That being said I made it a habit to review the tests I had given (and yes by that time I was writing my own using various sources) looking for questions missed by a high percentage of students to eliminate bad questions.  When I came across those questions I discounted those points from the exam so as not to penalize students.  If a student had happened to answer correctly I let them keep the points as extra credit.

3) Gather enough evidence to make valid judgements of proficiency. “We know we have enough evidence when we can confidently say that, if we gathered one more item it would simply confirm what we know now.”

Ok so is this done all on one test or through number of exams?  Does this depend on the student?  I need more information here or maybe I just need to put this fix into practice and see it work for myself, that might get me the answer I’m looking for.

4) Avoid bias that can distort results.  There can be problems with the student, the assessment setting, the scoring process, or the assessment itself can cause the score to misrepresent student achievement.

I’d say to the best of your ability because there are things you can’t control like students coming to school sick or without eating breakfast. I guess though this is where chances for retakes come in and the part from Fix 3 about sufficient evidence.

Teacher Vignette:

“My tests are composed of 10-12 high quality multiple choice questions that follow all conventions of multiple choice items designed to pinpoint where the errors are in student understanding.”

Ok this is all well and good but if teachers don’t know/aren’t taught those conventions how do they write quality multiple choice questions.  It might have been nice to get a reference to a good place to learn about that.

 

 

Repair Kit For Grading Chapter 1, part 1

It’s finally time to get into the book starting at the beginning with Chapter 1: Setting the Stage.  I had a lot of notes on this one so it will probably be at least two posts.  I will try to turn my notes into something more fluent and easy to read but it may not always happen so it something doesn’t make sense please ask.  Also I read this first chapter before I found the study guide online so I may repeat some things I said there, just so you know.

Starting right away on page 2 this quote caught my attention “How confident am I that the grades students get in my classroom/school/district are accurate, meaningful, and consistent, and that they support learning?”

Have I ever thought about this before?  Did the grades I gave at SMS really reflect the students’ learning?  I’m definitely one of those who argues that standardized testing, specifically the MAP test (Missouri Assessment Program) don’t always reflect a child’s knowledge due to any number of reasons.  My son is in third grade and taking the MAP for the first time this year, after seeing the responses he’s given on MAP like assessment for class I wonder what his score will be.  I know he understands the topics taught because he can easily discuss and explain them however sometimes he reads the questions differently than they are meant.  I’ve seen this happen with students as well, however since I have had that experience I know that his scores may not be indicative of his actual level.

I remember looking at the noun assessment provided in the teacher’s edition of the writing and grammar series I used and noticing that it wasn’t written the way the practice work had been.  At that point I rewrote the exam, I’m ashamed to say this was the third or fourth year I’d used the test and hadn’t noticed this before.  I had mistakenly assumed (and we all know what happens when you assume right?) that because the test was written by the same people that wrote the practices they would be similar in style.  Yep wrong!  Never again will I make that assumption, I will be checking the assessments and changing them as need be.

My partner teacher and I made it a practice to review our tests, specifically which questions were missed most often.  As we graded exams we often sat together and would discuss things we noticed such as questions that seemed to be missed frequently, we would then look at the answers to see if we had possibly given more than one correct answer by accident or if the question was in any way confusing.  We eliminated those questions from the exam for the next time it was given.  We also did not count that question in the final score on the exam, if students had answered it correctly that counted as bonus points for them, hmmm wonder if that would be bonus points that are a no no or bonus points that are considered showing extra knowledge?

It seems to me that this question just might be the reason so many teachers/districts have gone to backwards design, starting their lesson/unit planning by asking “what do I want my students to learn?  What is the learning target/objective?  How will I assess that learning?” and once they have determined that asking “Now how will I teach so they learn what I’m assessing?”  I know that I saw an increase in test scores when I began planning this way.

On pages 2 and 3 there is a blog post from the Washington Post that tells the tale of two students in the same school.  A student who had not achieved a high enough GPA was asked to leave, another student from the same school wrote in to say he had a similar schedule but with different teachers and some of his teachers offered a great deal of extra credit points.  He said he believes had he had the same teachers as student A he would have also been asked to leave because his grades would not have been as high.  

This shows to me the importance of having a school wide policy on extra credit.  If two teachers offer equally challenging curriculum but one of them also offers a substantial amount of extra credit the grades for the students in that class will be higher.  The office will be inundated for requests to be in that teacher’s class.  This is not effective, this doesn’t really show us that our students are learning what we want them to learn.

Ok on to page 5 where this made me stop to think: grades need to support learning – When teachers assign point values to simply turning in work (uh oh guilty, I have done this, there have been times where I was just so behind, so busy that I checked off each student had turned in the assignment and gave them 10 points for it.  I tried not to do this often but it did happen.) or put a mark on everything students do and use every number when calculating the grade the message sent to students is clear: success lies in the quantity of points earned.

I agree with the first part, if I simply assign points for turning in an assignment then yes students get the wrong message though I don’t know that it’s a matter of success lying in the quantity of points earned rather I think they learn that turning it in is all that is necessary, making an effort is not.  I have a problem with the idea that we shouldn’t put marks on everything students do, maybe my problem lies in the thought that for this book marks seems to mean scores or grades, points earned.  To me marks can simply be a plus or minus, a star that shows it was done not the same thing as points.    And if we don’t count points on everything how do we calculate grades?  Scores must be given on at least some items, how do we choose which ones?  It can not only be tests, I just can’t get behind that thought.  At what age do we start with point scores?  How do we explain why they aren’t getting points on some work?

Oh boy not even off of page 5 and they have me filled with even more questions: With some limited exceptions, only evidence from summative assessments should be used when determining grades.

If you’ve read my previous blog posts I bet you can guess what my big concern is here…TEST ANXIETY.  I’m trying not to harp on this topic as I’m hoping maybe it will be addressed later on but here are my original thoughts from reading this: What about those kids with testing anxiety or the child who went without breakfast that morning, who came to school sick, was beaten or berated that morning, experienced the loss of a pet or family member?  Do we allow a second or more chances on summative? (breaking into my old thoughts to say it appears this is exactly what is expected) Teachers know their students.  What do they do when the summative does not show the knowledge the student has demonstrated in homework or in class work?

I think that’s more than enough for today as this post is getting rather long.  What are you opinions?  What do you think?  Am I way off base in my thoughts?  Are you on the side of the author?  Maybe you’ve read the whole book already or your district is implementing these fixes and you can help me understand and feel better.  More on chapter 1 next time.

 

Repair Kit Study Guide: designing and preparing assessments, grades that support learning

This is it the last part of the study guide pre-assessments, next time I can finally move on to the actual book.  YAY!!  Ok truth time here I’m way into the book but I’m behind on posting, it’s ok, it’s better that way, less stress LOL.

 

So anyway this section is three questions and the idea is to assess your confidence level on each statement.

 

20: I can design or find assessments that provide an accurate picture of student learning on particular learning targets/objectives.

My last principal was a big proponent of backwards design, so I began to write units looking at learning targets/objectives, then writing the test, and finally planning instruction and formative assessments.  I also evaluated scores on summative assessments looking specifically for questions that students struggled with.  With all that in mind I’d say I’m somewhat confident on this one.

 

21: I can prepare assessment plans for units that show when formative and summative assessments will occur and how they will interact.

See the answer for #20.  No really I feel somewhat confident on this one for the same reasons.

 

22: I can assign grades that support learning.

On this one I chose a little confident mainly because I’ve never thought about grades supporting learning, I’ve always thought of grades as a reflection of learning.  I’m not sure if the grades I have given support learning in any way. I would guess that since they are not standards based that they don’t but I’m not sure.  So the question I’ll be looking to answer is how do I assign grades that support learning.

 

Well that’s it the end of the pre-assessment.  I really think I learned something through this, reflecting on my grading practices in the past and I have changed my mind on some things already from what I’ve read in the book.  Any opinions on today’s statements?

Repair Kit Study Guide: Feedback, homework, assessments

Ok so time to get back to a post on Repair Kit for Grading, you know the reason I began to blog in the first place 😉

8) I provide detailed comments to students about strengths and weaknesses in their work.

I didn’t do this on everything my students turned in, so my answer is sometimes.  I mainly provided more feedback on big projects and tests, especially on rough drafts of projects.  Even then looking back I feel like I didn’t provide enough feedback.

The daughter of one of my best friends has written her first book, she was 14 when she finished it btw and it is good.  I read it for her and when I read it I popped it up in Microsoft Word and began adding comments. I really concentrated on giving her feedback that would help her improve, if I had a question about what was happening, if I was confused, if I just had to say something in response to a character I put it in a comment.  I’d love to be able to do that for every student I have but I’m just not sure how to handle that in the time constraints teachers have.  I’ll be interested see what the book says about this, how often to give feedback, what kinds of feedback, how much feedback.  Guess I’ll find out when I get to that fix.

9) I include performance on homework into final grade.

I imagine you can already guess my answer to this is almost always.  I have a hard time fathoming it any other way, especially when you take into consideration what must be coming across as my biggest concern: test anxiety.  Let me make this clear, I have other concerns about this but I have had students who I knew were prepared and yet were reduced to tears or nervous tics before a test.  It makes no sense to someone who doesn’t suffer from the problem but teachers see it all the time.  I worry that if we did not ever include homework performance in final grades that the grades of students who do struggle with this fear would not be an accurate representation of their knowledge, the main reason for this book. I will definitely be looking for some answers to this concern and if I don’t find any in the book I may have to find a way to contact the author.

10) I keep separate track of information from formative and summative assessments.

I don’t know if I’ve ever tracked formative assessments other than notes I’ve taken during reading/writing conferences and homework grades.  Those would be separate but are they the types of formative assessments they refer to in this book.  I guess I’ll find out, for now I’d say sometimes is my answer.