SGO2.0: 2.0: SGO fromCompliance Complianceto toQuality Quality from
SGO2.0: 2.0: SGO fromCompliance Complianceto toQuality Quality from Increasing SGO Quality through Better Assessments and Target Setting 1 Note for Districts Using this Presentation and Resources This presentation has been designed by the Department for use by educators in districts to help them increase SGO quality. Read the notes below each slide carefully for additional information and context for the contents of the slides. (For PDF format, download file to view notes.) Links to resources in PDF format are embedded in the presentation. Other formats are available on the AchieveNJ website SGO page. Even though the contents of this presentation represent emerging best practices in SGOs and well established rules for assessment design, districts should understand that these are guidance materials only. They should be adapted and
modified to meet district-specific needs and priorities. For clarification on any of the topics covered by this presentation please visit http://www.state.nj.us/education/AchieveNJ/ or email [email protected] 2 Objectives for Today 1. Clarify what SGOs are and what they are not. 2. Develop a foundational understanding of how to develop and choose high quality assessments. 3. Investigate appropriate ways to set targets using readily available student data. 4. Develop a series of concrete next steps that will allow you to increase the quality of SGOs in your district. 3 Part 1 Clarify what SGOs are and what they are not. 4 Requirements for Student Achievement
Measures TEACHNJ Act The standards for approval of educator evaluation rubrics at a minimum shall include: a provision ensuring that performance measures used in the rubric are linked to student achievement. A Student Growth Objective is an academic goal that teachers and evaluators set for groups of students. It shall be specific and measurable, based on available student learning data, aligned to Core Curriculum Content Standards (or other standards adopted or endorsed by the State Board), and based on growth and/or achievement. 5 The Value of SGOs
For Educators SGOs provide a method by which teachers can improve their practice through high quality goal setting while clearly demonstrating their effectiveness through the learning exhibited by the students for whom they are responsible For Evaluators SGOs provide an authentic measure of teacher effectiveness that is aligned to the learning exhibited by students through an educators daily practice of teaching For Students When well-designed, SGOs promote reflective and collaborative teaching practices, alignment among standards, instruction and assessment, and improve student learning. 6 What SGOs Are, and What They Are Not Misconception SGOs need to be a significant addition to the work of a teacher. #1
Reality SGOs should be a reflection of what effective teachers typically do. 7 SGOs should be a reflection of what effective teachers typically do Assessment SGO Instruction Standards 8 Excerpt from SGO Quality Rating Rubric
Excellent Number of students in combined SGOs represents all or a large majority of the teachers students. Includes start and stop dates that include a significant proportion of the school year/course length. Includes a significant proportion of standards for which the teacher is responsible during the instructional period. SGO Quality Rating Rubric 9 General and Specific SGOs General Specific 0 .
1 4 O 1 G S 13 0 2 Captures a significant proportion of the students and key standards for a given course or subject area Most teachers will be setting this type of SGO Focuses on a particular subgroup of students, and/or specific content or skill For teachers whose general SGO already includes all of their students, or those who receive an SGP
10 2014-15 SGO Form Name School Significant proportion of students, standards and course Grade 9 Course/Subject Physics 1 Number of Students 55/55 Interval of Instruction
October-April Rationale for Student Growth Objective Name the content standards covered, state the rationale for how these standards are critical for the next level of the subject, other academic disciplines, and/or life/college/career? Name and briefly describe the format of the assessment method. Standards NJCCCS physical science 5.2.12 C, D and E NJCCCS science practices 5.1.12 A-D Impact of Standards This SGO includes all of the NJCCCS related to physics creating a foundation important for students who will take AP and/or college-level physics and is fundamental to many careers including architecture, mechanics, engineering, medicine. The SGO also includes all of the science practice standards, standards crucial in helping student become scientific thinkers. This mindset is valuable for making decisions when a large amount of information is available and must be analyzed for value and accuracy. It is critical in most academic disciplines. Assessment Physics departments common assessment administered at the end of the 3rd marking period Written: 60 multiple choice (4 choice), 5 short response questions, Practical: Students design a simple apparatus, take measurements and collect data.
2014-15 SGO Form High quality test normally 11 administered at this time #2 What SGOs Are, and What They Are Not Misconception SGOs are an administrator-driven compliance exercise Reality SGOs are driven by teachers, supported by administrators, and centered on student learning 12 SGOs are driven by teachers, supported by
administrators, and centered on student achievement 13 Part 2 Develop a foundational understanding of how to develop and choose high quality assessments. 14 Turn and Talk What is the relationship between assessment quality and SGO quality? 15 SGO Quality depends upon Assessment Quality
Poorly designed assessments do not accurately measure student knowledge and learning. If SGOs are based on low-quality assessments, then the SGO process cannot yield accurate or meaningful results. If SGOs do not yield accurate or meaningful results, they will fail to promote good instruction and improve student learning. 16 Types of Assessments for SGOs Teachers may use but are not limited to: Portfolios Performance Assessments
Benchmark Assessments Finals (modified as needed) Program-based Assessments Standardized Tests, e.g. AP Whether locally-developed or commercial, multiple choice or rubric-based, assessments should follow the rules of good assessment design. 17 What Does Good Assessment Look Like? 18 Elements of Assessment Design Purpose Note Taking Handout 19 Elements of Assessment Design Begin with the End in Mind
Purpose SGO assessments are measures of how well our students have met the learning goals we have set for them 20 Elements of Assessment Design Align to Standards 21 Elements of Assessment Design Align to Standards 22 Elements of Assessment Design Align to Standards
Given limited resources, especially time, on which standards do we focus our SGOs and assessments? 23 Elements of Assessment Design Align to Standards Determine the relative importance of the standard being taught during the SGO period 1. How much time is spent teaching the standard? 2. Does the standard have value beyond the current course in: i. the next level of the subject, ii. other academic disciplines, or iii. life/college/career? 24 Practice Time Using the criteria described, assign a score between 1 and 4 (1 is low priority, 4 is critical) for the four standards provided.
Rank the standards in order of importance (1st, 2nd, 3rd, etc. - ties not allowed). Provide a justification for your decision. 25 Determine the relative importance of the standard being taught during the SGO period Standard CCSS.ELA-LITERACY.RL.5.4 Determine the meaning of words and phrases as they are used in a text, including figurative language such as metaphors and similes. CCSS.ELA-LITERACY.RL.5.6 Describe how a narrator's or speaker's point of view influences how events are described. CCSS.ELA-LITERACY.RL.5.9 Compare and contrast stories in the same genre (e.g., mysteries and adventure stories) on their approaches to similar themes and topics. CCSS.ELA-LITERACY.RL.5.2 Determine a theme of a story, drama, or poem from details in the text, including how characters in a story or drama respond to challenges or how the speaker in a poem reflects upon a topic; summarize the text. 26
Determine the relative importance of the standard being taught during the SGO period* Standard Name Rating* Rank* CCSS.ELA-LITERACY.RL.5.4 Determine the meaning of words and phrases as they are used in a text CCSS.ELA-LITERACY.RL.5.6 Describe how a narrator's or speaker's point of view influences how events are described CCSS.ELA-LITERACY.RL.5.9 Compare and contrast stories in the same genre CCSS.ELA-LITERACY.RL.5.2 Determine a theme of a story, drama, or poem from details in the text * Answers will vary based on many factors.
Activity Handout 27 Determine the relative importance of the standard being taught during the SGO period* Standard Name Rating Rating* Rank Rank* Assessment Design 4 1st More Questions/Points 4
2nd 2 3rd 2 4th Fewer Questions/Points Rationale for Rating and Rank* * Answers will vary based on many factors. 28 2014-15 SGO Form Rationale for Student Growth Objective Name the content standards covered, state the rationale for how these standards are critical for the next level of the subject, other academic disciplines, and/or
life/college/career. Name and briefly describe the format of the assessment method. Standards NJCCCS physical science 5.2.12 C, D and E NJCCCS science practices 5.1.12 A-D This SGO includes all of the NJCCCS related to physics creating a foundation important for students who will take AP and/or college-level physics and is fundamental to many careers including architecture, mechanics, engineering, medicine. The SGO also includes all of the science practice standards, standards crucial in helping student become scientific thinkers. This mindset is valuable for making decisions when a large amount of information is available and must be analyzed for value and accuracy. It is critical in most academic disciplines. 29 Using Commercial Products for SGOs 30 Elements of Assessment Design Valid/Accurate Inferences 31 Elements of Assessment Design
Valid/ Accurate Inferences 32 Elements of Assessment Design Valid/ Accurate Inferences Valid/Accurate Inferences Why does The assessment should measure what it sets out to measure. it matter? The assessment is aligned to standards, and rigor of the instruction and What does skills, content of the course. it look like? The assessment is accessible to all students.
33 Analyze This Item How valid is the inference we can make about student learning using this question? How can we make this a better assessment item? 6.2.12.C.1.b - Trace the movement of essential commodities (e.g., sugar, cotton) from Asia to Europe to America, and determine the impact trade on the New Worlds economy and society. Perhaps the most famous of all the arts of the Ming Era was: A. the elaborate puzzles of the period, which were popular even in Europe. B. blue-and-white porcelain, which Europeans collected in great quantities. C. the construction of large, elaborate palaces, the finest example of which is the Imperial City in Beijing. Handout D. high-quality Berber rugs, which are still popular today.
34 35 Item is not aligned to standards 6.2.12.C.1.b - Trace the movement of essential essential commodities commodities (e.g., sugar, cotton) from Asia to Europe to America, and determine the impact trade on the New Worlds economy and society. of Ming the Ming Era Perhaps the most famous of all the arts arts of the Era was: A. the elaborate puzzles of the period, which were popular even in Europe. B. blue-and-white porcelain, which Europeans collected in great quantities. C. the construction of large, elaborate palaces, the finest example of which is the Imperial City in Beijing. D. high-quality Berber rugs, which are still popular today.
36 Elements of Assessment Design Range of Rigor/Depth of Knowledge 37 Elements of Assessment Design Range of Rigor/DOK 38 Elements of Assessment Design Range of Rigor/DOK Range of Rigor/Depth of Knowledge An assessment that accurately reflects the range of of the course and instruction increases the Why does rigor validity of inferences educators can make about
it matter? student learning. Provides access points to students of varying ability. assessment requires a range of thinking skills as What does The it look like? proposed by Blooms taxonomy and Webbs Depth of Knowledge (DOK) that reflects the rigor of the course. 39 Elements of Assessment Design Depth of Knowledge Wheel Range of Rigor/DOK 4 minute video explaining DOK using the Gettysburg Address Handout 40
Determine the Rigor of this Item What DOK level does this item represent? What modifications could you make to the question to make it more rigorous? Examine the following political cartoon and answer the following questions. 1.What does the snake in this cartoon represent? 2.Whom is the snake attacking? Handout 41 Determine the Rigor of this Item What DOK level does this modified item represent? Examine the following political
cartoon. Use details from the cartoon to: 1. Explain the symbolism of the snake in the political cartoon. 2.Explain why the artist used children to represent free press, free speech, and honest opinion. 42 Elements of Assessment Design NOT Rigor for Rigors Sake Range of Rigor/DOK A high quality assessment has a range of rigor that: Is representative of the rigor of instructional level and content delivered in the course, and Provides stretch at both ends of ability levels 43 Elements of Assessment Design Accessible
44 Elements of Assessment Design Accessible 45 Elements of Assessment Design Accessible Accessible Assessment Why does it matter? What does it look like? Promotes similar interpretations of the data and informs sound instructional decisions. Its fair to all students. Provides equal access to all students regardless of personal characteristics/background and pre-existing extra-curricular knowledge.
Questions and structure do not disadvantage students from certain groups or those without particular background knowledge. Appropriate modifications for students with learning plans. Format, wording, and instructions are clear. 46 Examples Directions: Directions: Choose the one answer that best solves the problem. Choose the one answer that best solves the problem. If one card is taken at random from a deck of playing cards, what is the probability that the card will be an ace?
A) 8% B) 50% C) 25% D) 10% There are 4 aces in a deck of 52 playing cards. If one card is taken at random from the deck, what is the probability that the card will be an ace? A) 8% B) 50% C) 25% D) 10% 47 Examples Directions: Directions: Choose the one word or phrase that best completes the
sentence. Choose the one word or phrase that best completes the sentence. The soldiers and their wives excitedly attended the _________. A) funeral B) celebration C) meeting D) workshop The soldiers and their spouses excitedly attended the _________. A) funeral B) celebration C) meeting D) workshop 48 Examples
49 Check for Understanding Directions: Choose the one word that completes the sentence. Directions: Choose the one word or phrase that best completes the sentence. Quarterbacks are often sacked during games _______ they do not have a good offensive line protecting them. Some students are often late to class _______ they visit their lockers too frequently. A) even though B) although C) in spite of D) because
A) B) C) D) even though although in spite of because 50 Elements of Assessment Design Reliable/Consistent 51 Elements of Assessment Design Reliable/ Consistent Reliable
le Unreliab 52 Elements of Assessment Design Reliable/ Consistent Reliable/Consistent Assessment Why does it matter? Provides information about student learning that can be trusted. What does it look like? Assessment administration and scoring is
standardized and comparable. Assessment items yield consistent results over time. 53 Turn and Talk Discuss the items in the table below. How do these enhance the reliability of the assessment? Which do you have in place now? Are there others you could add to this list? Provide a physical and emotional environment that encourages students to do their best. Provide clear directions and scoring criteria to students before they start the assessment. Allow enough time to complete the assessment. Make the assessment long enough (longer assessments are generally more reliable).
Ensure scoring is done by educators trained using clear criteria; use multiple scorers when possible. Keep the assessment secure before and after test. 54 Check for Understanding Day Monday Weight Scale (lbs) 130 Bathroom Time of Day Morning Tuesday 130 Bathroom
Morning How would you describe the reliability of this scale? How about the validity of the information you get from it? 55 Elements of Assessment Design Bringing the elements together into a coherent whole Blueprint 56 Elements of Assessment Design Blueprint
A blueprint document describes the content and structure of an assessment. It defines the: Standards measured Relative importance of the standards on the assessment Item types, number and point value DOK of each item 57 Elements of Assessment Design 0 2 3 1 Blueprint
4 1 58 Elements of Assessment Design PRIOR TO TEST DESIGN Standard and Relative Description of Importan Standard ce of (NJCCCS, CCSS, Standard etc.) 4.NBT.B.4 Add and subtract multidigit whole numbers 4= High 3= Mediumhigh 2= Mediumlow
1= Low 4 Blueprint DURING TEST DESIGN Type of Question (multiplechoice, constructedresponse, Depth of Question Total Point Knowledge Number/ Value/ of Question Points Percentag 4= Extended e of Test essay, etc.) Thinking
Assessment Blueprint and Completion Guide 30 pts /10% 59 Elements of Assessment Design Blueprint AFTER TEST DESIGN CHECKLIST Is the assessment of a length and format that is appropriate for subject/grade level? Is the complete assessment and each assessment item accessible to all students? Can the assessment be administered under comparable conditions across classrooms? Can the assessment be scored consistently with a readily accessible scoring guide and/or rubric? Does each item follow the rules of assessment item design? Rules of Item Design 60
Part 3 Investigate appropriate ways to set targets using readily available student data. 61 #3 What SGOs Are, and What They Are Not Misconception Reality SGOs are a statistically precise measure of growth based on a pre-test/posttest model of performance. SGOs are learning targets for key concepts and skills that students can be expected to master in a course based on a rough sense of where they start.
62 Pre-tests - The Siren Song of Simplicity 63 Important Considerations if Using the Pre-test Post-test Model Inherent Testing Error Error, present in all tests, is compounded in a pre- post- model, and often greater than the learning gains of the students. Reliability of Results Especially in Pre-test Dont worry about it this doesnt count. Stretches Teacher and Student Capacity Two high quality assessments must be developed and administered. Unnecessary tests can interfere with other important work occurring at the start of the school year. Lack of Value for Instructional Purposes Yep, just as I thought my kids dont know any Mandarin yet. Difficult to Set Reasonable Targets 64 Impossible to extrapolate future learning from one data point. What is the Alternative to Pre-/Post-testing
Model for SGOs? Create learning targets for key concepts and skills that students can be expected to master in a course based on a rough sense of where they start using a variety of typically-collected information about student learning 65 Predict the Final Picture 66 Predict the Final Picture 67 Predict the Final Picture 68 100 90 80
70 60 50 40 30 20 10 0 ? ? ? 0 0.5 1 1.5 2 2.5 3 3.5 4 Time Expected learning cannot be determined using one data point. Learning Learning Predicting Student Learning Based on a Rough
Sense of Where They Begin 100 90 80 70 60 50 40 30 20 10 0 0 0.5 1 1.5 2 2.5 3 3.5 4 Time Expected learning is betterdetermined using multiple measures of starting points. 69 List the information you have used or could potentially use to determine students starting points. 1.
2. 3. 4. 5. 70 List the information you have used or could potentially use to determine students starting points. 1. 2. 3. 4. 5. Current grades Recent test performance Previous years scores Important markers of future success Well-constructed and administered, high-quality pre-assessments 71
Sample Rubric for Important Markers of Future Success Criterion Active Participant Level 4 Always prepared Engaged in all of the learning process Level 3 Mostly prepared Engaged in most of the learning process Frequently Consistently demonstrates demonstrates intellectual
Academic intellectual curiosity curiosity Independence Consistently self Usually selfmotivated and motivated and independent independent Class Attendance Never absent Rarely absent Rubric for Important Markers of Future Success Level 2 Level 1 Sometimes prepared Engaged in some of the learning
process Rarely prepared Engaged in little or none of the learning process Sometimes demonstrates intellectual curiosity Sometimes selfmotivated and independent Rarely demonstrates intellectual curiosity Rarely or never selfmotivated, frequently depends on prompting and/or teacher assistance Sometimes absent Frequently absent
72 What data could be used to get a rough sense of students starting points? STUDENT PRIOR YEAR TEST SCORES MARKERS OF FUTURE SUCCESS (see rubric) CURRENT YEAR TEST SCORES Unit 1 Unit 2 Average Score Active
Participant (1-4) Attendance (1-4) Academic Independence (1-4) Total Points PREPAREDNESS GROUP 1. 252 100 97
98.5 4 3 3 10 High 2. 201 62 83 72.5 2
4 3 7 Medium 3. 143 57 75 66 2 1 3
6 Low Key Prior Year Test Score Current Year Test Score Average Markers of Future Success Preparedness Group 250 300 85 100 9-12 High
200 249 70 84 5-8 Medium <200 <70 0-4 Low 73 Physics 1 SGO Using Multiple Measures of Starting Points Student ID
Prior Test Scores Current Year Test Scores Markers of Future Success Preparedness Group NJ ASK 8 Math Unit 1 Unit 2 Average Score Participates in Class Completes
Retakes Completes Homework Total Points 1 230 100 97 98.5 Yes Yes No 2
High 2 202 90 95 92.5 Yes Yes Yes 3 High 3
211 95 95 95 Yes Yes Yes 3 High 4 241 85
86 85.5 Yes No No 1 High 5 263 90 92 91
Yes No Yes 2 High 6 284 90 85 87.5 Yes No
Yes 2 High 7 199 91 88 89.5 Yes Yes Yes 3
High 8 201 57 75 66 No Yes No 1 Low 9
144 50 58 54 No No No 0 Low 10 182 58
58 58 No No No 0 Low 11 143 62 83 72.5
Yes Yes No 2 Medium 12 171 78 83 80.5 No Yes
No 1 Medium NJ ASK Math Score <200 200 249 250 300 Current Year Test Score Average <70 70 84 85 100 Number of Future Success Markers 01 12 23
Preparedness Group Low Medium High Target Score on Summative 70 80 90 Determining Appropriate Target Scores Previous Years Test Score Current Year Test Score Average
Markers of Future Success (points) Preparedness Group Target Score on Summative 79 or below < 70 0-6 Low 70 80-89 70-84 7-9
Medium 80 90 or above 85-100 10-12 High 90 What level of performance on the assessment would indicate a sense of competence/mastery of the content and skills I am teaching? How should I modify this performance target based on the preparedness level of my students. 75 76
What is the appropriate role of pre-assessments in SGOs? Where improvement in a set of skills is being evaluated When assessments are high quality and vertically aligned When normally used for instructional purposes In combination with other measures to help group students according to preparedness level 77 Effectively using high quality pre-assessments in combination with other data to set targets Grade 1 Reading Student High Frequency Initial Markers of
Word DRA Level Future Success Recognition Preparedness Group DRA Target 1. 3 25 7 Medium 14 2.
1 26 4 Low 4 3. 3 35 8 Medium 14 4.
6 62 10 High 18 78 2014-15 SGO Form Starting Points and Preparedness Groupings State the type of information being used to determine starting points and summarize scores for each type by group. Add or subtract columns and rows as needed to match number of preparedness groups and types of information used. Preparedness Group Preparedness Group Information #1 Information #1 Prior Year Test Score
Information #2 Information #2 Current Year Test Score Average Information #3 Information #3 Markers of Future Success High 250 300 85 100 9-12 Medium 200 249 70 84 5-8
Low <200 <70 0-4 Preparedness Group Prior Year Test Score High Medium Low 250 300 200 249 <200
Current Year Test Score Average 85 100 70 84 <70 Markers of Future Success 9-12 5-8 0-4 79 Part 4 Develop a series of concrete next steps that will allow you to increase the quality of SGOs in your district. 80 Possible Next Steps Share information from this workshop with all members of your DEAC and
develop a strategy for developing higher quality assessments and SGOs throughout the district. Review the materials from this workshop and plan the time and method for delivering to staff in a PD session. Ask building leaders to create an SGO assessment inventory and check quality against the elements of assessment design and item design rules. Ask teachers to identify 3 sets of data to determine student starting points. Build in time during PLC/team time for assessment development early in the next school year. Use the SGO quality rating rubric to determine quality of SGOs during the approval process (deadline - October 31st, 2014). Activity Handout 81 Resources Updated SGO guidebook and forms Expanded SGO library
Assessment quality webinars Teacher practice workshops Information www.nj.gov/education/AchieveNJ Questions [email protected] 609-777-3788 82
HOLD UP EACH OF THE TWO SAMPLE BOOKLETS (session one and session two) - HOLD UP THE STUDENT ANSWER SHEET SO THEY CAN SEE WHAT IT'S GOING TO LOOK LIKE. ... Ontario Secondary School Literacy Test (OSSLT) Last modified by:
Changes in collagen structure can have adverse effects. Previous studies in my mentor's lab have shown that DDR1 modulates collagen structure in-vitro (Flynn et. al., 2010). Glycoprotein VI (GPVI) is a blood platelet receptor protein that is known to bind...
Carbon and iron data from the Mesilla Valley Formation indicate a shoaling-upward suboxic-anoxic ferruginous depositional environment. Increasing TOC in conjunction with unchanging δ13C. carb. is characteristic of OAE 1d. Correlation of geochemical data with existing biostratigraphic data substantiates the association...
Identify family members in Spanish. Use family vocabulary, adjectives, colors, and clothing vocabulary to describe people in Spanish. Listen for specific details about family members in Spanish. Use present tense verbs and appropriate vocabulary in Spanish to write a paragraph...
-In the above, each read command adds a latence of t_WTR + t_RTW.-Each write command adds a latency of t_WL + t_BUS. Worst Case AnalysisCumulative Latency . tAC:-t_CD is not dependent on the previous tasks. ... Worst Case Analysis of...
Chiefdom A society with social ranking in which political integration is achieved through an office of centralized leadership called the chief. Citizenship Membership in a state. Chapter 12, Political Organization Key Terms Achieved status A social position that a person...
Emergency Management Planning Business Continuity IT Partners Business Continuity Planning A few definitions to get started: … a "disaster" The cake I was baking to bring to Xmas dinner He lost a laptop with the only copy of his thesis...
Ready to download the document? Go ahead and hit continue!