Ne SAW Grades 4811 Writing Assessment Nebraska Department

Ne. SA-W Grades 4/8/11 Writing Assessment Nebraska Department of Education Statewide Assessment Pat Roschewski – pat. roschewski@nebraska. gov Edward Foy – edward. foy@nebraska. gov Julie Barger – ESU #16, Ogallala, NE 2/26/2021 1

The Ne. SA-W/DRC Partnership Working together to improve teaching and learning for Nebraska students. 2/26/2021 2

The Nebraska Department of Education and Nebraska educators continue to direct the process of statewide assessment. 2/26/2021 3

Nebraska Educators continue to be involved in these aspects of Ne. SA-W: • • • 2/26/2021 Prompt Creation Field Testing Prompt Selection Rangefinding Appeals Continued training and communication 4

The Rangefinding Process – Included 7 -10 Nebraska educators from a variety of educational backgrounds – Two DRC reps facilitated the work 2/26/2021 5

The Rangefinding Process – Analytically scored, discussed, and came to consensus on 90 -100 papers from the field testing of the test prompt. – DRC will use the papers selected and scored by this group to establish the anchors and train the raters prior to the scoring of the assessment. 2/26/2021 6

Grade 8 & 11 Online Assessment 1. Grade 8 will remain a descriptive essay. Grade 11 will remain a persuasive essay. 2. The 8 th & 11 th grades Ne. SA-W assessment will only be available online. 3. Students with IEP’s or 504 Plans may take the test in pencil and paper if documented in an IEP. 4. Students responding in Spanish will take a pencil and paper assessment. 2/26/2021 7

5. Paper/pencil responses at 8 and 11 must be written in #2 pencil. Papers written in ink will not be scored and will result in zero scores. 6. In paper/pencil testing student may use only the paper provided. If additional sheets of paper are added to the booklets, they will not be scored. 2/26/2021 8

7. When making copies of the paper/pencil booklets for local scoring or possible appeals, do not take the booklets apart. 8. Do not copy the entire booklet, but only make a copy of the actual student’s response. 2/26/2021 9

9. Ne. SA software is scheduled for release November 14 th. (All subjects, including writing and practice tests. ) 10. Districts will need to schedule a download or update of software prior to administering the assessment. 11. Students will be allowed to use up to 6, 000 characters in their writing responses = approximately three pages. A pop-up warning will occur at approximately 4, 000 characters. 2/26/2021 10

12. The TAB button is not compatible with the software. Student should be advised to space over 3 -5 spaces. This does not affect scoring. 13. The results will be analytically scored. 2/26/2021 11

Grade 8 & 11 Online Assessment Printing options are available. Ø Both the practice and operational test may be printed in 2012. Ø The printed copy may be used for local scoring, placed in a student portfolio, or shared with the student/parents. Ø Districts will need the printed copy for appeal purposes – This copy is used to decide if an appeal is warranted and, if so, a copy must be attached to the appeal form. 2/26/2021 12

Rubric Changes have been made to the 8 th and 11 th grade rubrics. Ø Ø The rubrics were changed to analytic rubrics in the fall of 2010 from the previous holistic rubrics. Ø The content of each new rubric has been tightened, and although most of the previous criteria remain, they have been redistributed into fewer domains. Ø Grade 8 and 11 analytic rubrics will be in use this year with the 2012 assessment. 2/26/2021 13

Rubric Changes Ø The domains of the rubrics will be weighted as follows: – – Content/Ideas – 35% Organization – 25% Word Choice/Voice/Tone – 20% Sentence Fluency/Conventions - 20% 2/26/2021 14

2/26/2021 15

2/26/2021 16

Rubric Changes were made to the rubric for a number of reasons. Ø A revision of the Language Arts standards occurred in 2008 -2009. Ø An external study determined that the new standards and the writing assessment rubrics did not match, necessitating a review/revision of the rubric. 2/26/2021 17

Rubric Changes Ø Influence favoring analytical assessment was expressed in an effort to provide more specific data to data users. Ø The use of a computer based assessment reflects a “real world” application of writing. Ø Five vendors responded to the RFP (Request for Proposals. ) 2/26/2021 18

Rubric Changes Ø The new rubrics are available on the Assessment page of the NDE website. http: //www. education. ne. gov/Assessment/Ne. SA_Writing. htm 2/26/2021 19

Ne. SA-W Assessment Ø Ne. SA-W testing window is January 23, 2012 February 10, 2012 Ø 8 th and 11 th grade tests will be administered online. Ø Students with IEP’s or 504 plans may take the test using paper/pencil if documented in an IEP. Ø Students responding in Spanish or a language other than English will take a paper/pencil assessment. 2/26/2021

Ne. SA-W Grades 8 & 11 On-line Ø The recommended testing time is 90 minutes to occur on one day. Ø The test is not timed, but 90 minutes is recommended by NDE as the amount of time to schedule for the assessment. Ø If a student needs more than 90 minutes, that is allowable, the same as for Ne. SA-R. 2/26/2021 21

Ne. SA-W Grades 8 & 11 On-line Ø Pre-writing may be done on paper furnished by the district. Students may choose to prewrite and produce a rough draft on paper before entering it on the computer, or they may choose to work entirely on the computer. 2/26/2021 22

Ne. SA-W Grades 8 & 11 On-line Ø A new thesaurus, dictionary, and cut/paste are available. The spell check will not be available. Ø An online practice test for 8 th and 11 th grade writing will be made available. You can access this assessment through the CAL software on November 14 th. 2/26/2021 23

Practice Test Instructions – 8 & 11 2/26/2021 24

Practice Test Prompt 2/26/2021 25

Practice Test Self Assessment Tool 2/26/2021 26

Ne. SA-W On-line Grade 11 Pilot Test Times Data Time Span in Minutes % in Each Span 0 - 30 minutes 30 - 60 minutes 60 - 90 minutes 90 + minutes 6% 58 % 31 % 5% 2/26/2021 27

Grade 8 & 11 Online Assessment Additional Information Ø Graphic organizers, story starters, and story frames are not to be provided to students as part of the state testing program. Ø In the testing room, visual aids and clues should be removed or covered and remain covered throughout the administration of the writing assessment. 2/26/2021 28

Grades 8 & 11 Online Assessment Additional Information Ø Teachers are NOT to provide editing assistance to students on the Ne. SA-W. Ø No Ne. SA-W online tests will be provided in Spanish. All Spanish tests will be administered via paper/pencil test booklets. Ø Districts submitted requests for Spanish prompts via e. Direct. 2/26/2021 29

Grade 8 & 11 Online Assessment Additional Information The composite score is calculated from the domain scores as: 1. Composite = 1. 4 D 1 + 1. 0 D 2 + 0. 8 D 3 + 0. 8 D 4 D 1, D 2, D 3, and D 4 are the sums of the two reader scores for the four domains. The table below illustrates the calculation for a student who received a 2 or 3 from each reader on each domain. R 1 R 2 D=R 1+R 2 W W*D Ideas / Content 3 3 6 1. 4 8. 4 Organization 2 3 5 1. 0 5. 0 Voice / Word Choice 3 2 5 0. 8 4. 0 Sentence Fluency / Conventions 2 2 4 0. 8 3. 2 Composite Score 20. 6 The corresponding Scale Score can then be retrieved from the table supplied by DRC. 2/26/2021 30

Grade 8 & 11 Online Assessment Additional Information Current information about the Ne. SA-W assessment and all Ne. SA assessments can be found on the NDE website. Go to http: //www. education. ne. gov On your left, click on Statewide Assessment. Click on Ne. SA Writing. 2/26/2021 31

The Process of Analytical Scoring The New Statewide Process 2/26/2021 32

Holistic vs. Analytical Scoring Analytic scoring involves a score being given for each domain and its criteria. These individual weighted scores are combined to calculate the final score. 2/26/2021 33

2/26/2021 34

2/26/2021 35

The Analytical Scoring Process The Process Ø First read: focus on Ideas/Content and Organization--assign a score to each domain of 1, 2, 3, or 4. Ø Then review paper: focus on Word Choice/Voice and Sentence Fluency/Conventions-- assign a score to each domain of 1, 2, 3, or 4. 2/26/2021 36

The Analytical Scoring Process The Process Ø Each paper is scored by two readers. Ø Agreement: Exact matches or adjacent scores. Ø A Third Read by the table leader is required for each domain if scores are not in agreement. 2/26/2021 37

The Analytical Scoring Process Rules for obtaining the final score for a student after the third read: 1. If the third score is an exact match to one of the original scores, the two matching scores are used. 2. If the third score is adjacent to one of the original scores but not the other, the third score and the score adjacent to it are used. 3. If the third score is adjacent to both scores, the third score is used twice. 2/26/2021 38

The Analytical Scoring Process Possible impact of analytical scoring: Ø Information: Each student report will include four sub scores and one scale score. These reports are shared with students and parents. Ø Trends: Over time, data from this assessment can indicate domain strength or weakness at a classroom, school, and/or district level. 2/26/2021 39

Scoring Practice 2/26/2021 37

8 th Grade Prompt Think about your dream house. It could be any place you wish. Write an essay describing your dream house by creating a vivid mental picture using sensory details for the readers so that they can see it, experience it, and be able to understand why you would want to live there. 2/26/2021 37

My dream house is located. . . #8 – see top right 2/26/2021 42

Analytical Scoring Ideas & Content: 2 The picture of what is being described is somewhat limited; some digressions from the topic. Organization: 3 Intro, body, conclusion are functional; pacing is controlled; transitions functional; paragraphing is generally organized. Voice & Word Choice: 3 Wording is generally expressive; language generally precise; adequate vivid words and phrases. Sentence Fluency & Conventions: 3 Sentences generally vary in length or structure; occasionally the phrasing sounds unnatural; fragments do not distract the reader; grammar, usage, punctuation do not distract. 2/26/2021 43

Have you ever wanted to see, . . . #6 – see top right 2/26/2021 44

Analytical Scoring Ideas & Content: 2 Limited sensory details; limited picture of what is being described. Organization: 2 Structural development of an intro, body, & conclusion is limited; pacing is inconsistent; transitions are weak. Voice & Word Choice: 1 Wording is inexpressive and lifeless; little sense of the writer; language is neither specific, precise, or varied; few vivid words or phrases. Sentence Fluency & Conventions: 2 Sentences seldom vary, phrasing occasionally unnatural, conventions do not distract the reader 2/26/2021 45

Grade 11 Scoring Guide The packet you have in your handout represents a shortened version of the actual scoring packet that raters use when scoring Nebraska papers. In the actual rater’s packet two sample papers are included for each score point in each domain, totaling 32 papers that are used by raters as anchors for the rubric. Also note the rationale that is provided for the score on each paper. 2/26/2021 46

Grade 11 Scoring Guide Ideas/Content: 4 example The writer conveys a clear opinion regarding the benefits of taking a class in Medical Terminology. The content is well focused throughout the response and the reasoning is logical and compelling. The supporting examples are numerous and relevant. 2/26/2021 47

Grade 11 Scoring Guide Organization: 2 example A limited structural development is evident in this essay about the need for a weights class. A weak introduction and a brief conclusion are present, but the main body is a list of somewhat random ideas regarding the benefits of a class, which are connected using a few weaker and repetitive (also) transitions. Paragraphing while present is not successful and pacing is somewhat inconsistent. 2/26/2021 48

Grade 11 Scoring Guide Voice/Word Choice: 3 example The writer demonstrates a general commitment to the merits of a gym class. While the voice is appropriate for the purpose and audience, it is not particularly enthusiastic or passionate about the topic and projects only a generally persuasive tone. The language is generally specific and engaging and the writer generally anticipates the reader’s questions. 2/26/2021 49

Grade 11 Scoring Guide Sentence Fluency/Conventions: 1 example Frequent grammar, usage, punctuation and spelling errors distract the reader through this response. Sentences seldom vary in length and structure, and the run-on sentences confuse the reader. Phrasing sounds awkward and unnatural. 2/26/2021 50

What scores would these papers receive in other domains? 2/26/2021 51

Questions? Nebraska Department of Education Statewide Assessment Pat Roschewski pat. roschewski@nebraska. gov Edward Foy edward. foy@nebraska. gov 2/26/2021 52