TAGG A New Online Transition Assessment Jim Martin

TAGG: A New Online Transition Assessment Jim Martin & Amber Mc. Connell Zarrow Center for Learning Enrichment University of Oklahoma Zarrow Center

Transition & The Courts • Prince, Plotner, & Yell (2014) examined district court findings and recommend – Use multiple assessments across transition domains – Do not solely use informal assessments • This means at least one transition assessment needs supporting validity evidence – Maximize student participation in the transition planning process Zarrow Center

Case Law Decision Case involved inadequate transition assessments. Decision: • Ruled in favor of the family. School needed to focus on meaningful non-academic goals to prepare students for post-school life. Massachusetts Bureau Of Special Education Appeals And Currently Under Appeal In Federal Court, Dracut Public Schools, BSEA #085330, 15 MSER 78 (2009). Zarrow Center

Agenda 1. 2. 3. 4. 5. 6. TAGG Overview TAGG Development In-Brief: Validity Evidence Sample TAGG Screen Shots Obtain TAGG Teaching TAGG Skills Zarrow Center

Thanks to the National Center for Special Education Research (NCSER) • The TAGG was developed with – A grant from the National Center for Special Education Research (NCSER) and – the University of Oklahoma Zarrow Center for Learning Enrichment Funds Zarrow Center

TAGG. OU. EDU/TAGG Zarrow Center

TAGG OVERVIEW Zarrow Center

TAGG Purpose • Assesses non-academic skills associated with and predictive of post-school further education and employment • Provides IEP team student strengths, needs, a written summary, and annual transition goals matched to common core standards to facilitate writing Indicator 13 compliant IEPs Zarrow Center

Whom? • TAGG Designed to Assess – Secondary-aged students with IEPs who plan to be competitively employed and/or enrolled in further education after graduation • Each TAGG set includes 3 versions – Student – Family – Professional Zarrow Center

Versions and Format • Online written English – May be printed and taken by hand, but item scores must be entered into website to produce results – Spanish and English audio • Users may choose to listen to audio or watch ASL videos for TAGG instructions and items Zarrow Center

Reading Level • Professional 10. 4 grade level • Family 5. 7 grade level • Student 4. 8 grade level Zarrow Center

TAGG Web-Generated Results Profile • • • Graphic results by constructs Written summary Relative and greatest strengths Relative and greatest needs Annual transition goals Components may be copied and pasted into IEP Zarrow Center

TAGG DEVELOPMENT Zarrow Center

• Standards for Educational and Psychological Assessment • Guided TAGG Development Zarrow Center

Development of TAGG Items • TAGG items derived from research studies that identified behaviors of former students with disabilities engaged in post-high school employment and/or further education • The research team initially used the research studies to develop – 10 construct definitions – Items developed from constructs • 15 iterative TAGG versions were created before field testing began Zarrow Center

Initial Structure: Ten Initial Constructs • Knowledge of strengths and limitations • Actions related to strengths and limitations • Disability awareness • Employment • Goal setting and attainment Zarrow Center • • • Persistence Proactive involvement Self-advocacy Supports Utilization of resources

After FA: Professional and Family TAGG Stayed 1. Strengths and Limitations 2. Disability Awareness 3. Persistence 4. Interacting with Others 5. Goal Setting and Attainment 6. Employment 7. Student Involvement in IEP 8. Support Community Zarrow Center Dropped 1. Actions Related to Strengths and Limitations 2. Utilization of Resources TAGG-P: (c 2=1043. 62, df=499, RMSEA=. 058, CFI=. 92, TLI=. 91, RMSR=. 0597) TAGG-F: (c 2=862. 74, df=499, RMSEA=. 057, CFI=. 91, TLI=. 90, RMSR=. 058)

Student Version Constructs After FA Constructs Dropped Constructs 1. Strengths and Limitations & Support Community 1. Actions Related to Strengths and Limitation 2. Disability Awareness 2. Utilization of Resources 3. Persistence 4. Student Involvement in IEP 5. Interacting with Others Combined Constructs 1. Strengths and Limitations 2. Support Community 6. Goal Setting and Attainment 7. Employment Zarrow Center TAGG-S: (c 2=819. 00, df=505, RMSEA=. 047, CFI=. 89, TLI=. 88, RMSR=. 064)

Replicate Factor Structure • Two additional studies from TAGG users across the country • Results confirmed strong factor structure Zarrow Center

Three Years of Data Collection • 2, 556 participants from 42 states and 162 schools – 1, 291 secondary students with disabilities who had postsecondary further education and/or competitive employment goals – 172 professional completed TAGG on 7 to 8 of their students – 847 family members completed TAGG on their child Zarrow Center

Final TAGG Constructs • Strengths and Limitations • Disability Awareness • Student Involvement in the IEP • Persistence Zarrow Center • Goal setting and attainment • Interacting with Others • Employment • Support Community

IN-BRIEF: VALIDITY EVIDENCE Zarrow Center

Internal Reliability • Generally, a score between. 7 and. 8 is considered “good” – Each TAGG version has great overall internal consistency and satisfactory subscale consistency (ranging from α =. 89 to α =. 95) Zarrow Center

Test-Retest Reliability • Scores of. 7 or higher represent good or satisfactory test-retest reliability – 14 weeks after the first TAGG was completed, same users completed the TAGG again – A large correlation was found between the first and second administrations • . 80 for professional TAGG • . 70 for family TAGG • . 70 for student TAGG Zarrow Center

Fairness Validity Evidence: Gender • Do differences exist by gender? – No overall difference by gender on TAGG-P, TAGG-S – On TAGG-F small overall differences – Some construct differences exist. On TAGG-S – females rated themselves higher on student involvement in IEP than males – males rated higher on employment Zarrow Center

Fairness Validity Evidence: Disability Categories • Some construct differences do exist, but these are small differences – 5 Disability Categories were included • Autism, ED, ID, OHI, and SLD – On most constructs OHI and LD scored higher than Autism, ED, and ID Zarrow Center

Students with Autism • Students with Autism scored themselves as having no problems with interacting with others. – (Professionals and Family members scored students with Autism lower than ID, OHI, SLD) • Students Slide with ED scored lower on interacting with others Zarrow Center

Fairness Validity Evidence: SES • Free/reduced lunch eligibility • No significant differences for construct scores on TAGG-P or TAGG-S. Only small differences for TAGG-F scores. • Family education • Significant differences- Highest family education lower TAGG scores Zarrow Center

How Close Are Students, Professionals, and Family TAGG Scores? • How closely do the different TAGG versions assess the same student? – Medium correlations across Parent, Educator, and Student versions when assessing the same student – This is excellent for this type of assessment Zarrow Center

TAGG & AIR Self-Determination Assessment • Same users completed TAGG and AIR Self. Determination Assessment – Medium Correlation • This implies the TAGG addresses some selfdetermination skills and assesses other skills, too. Zarrow Center

Predictive Validity Process • Follow-up of 297 former high school students who completed the TAGG while in high school • Logistic regressions examined relations between TAGG non-academic behavior constructs and postsecondary education and employment Zarrow Center

Constructs Predicting Further Education • • Interacting with Others Student Involvement in the IEP Support Community Goal Setting and Attainment Zarrow Center

Constructs Predicting Employment • • Employment Student Involvement in IEP Support Community Interacting with Others Zarrow Center

Item Response Theory • Advantages of IRT include – The ability to scale different item types – Provides a common metric for scales with different number of items – Weights items differentially by their validity for assessing the construct of interest Zarrow Center

IRT Algorithms Produce Results Profile 1. Placed each scale onto a common score metric 2. Projected item characteristics (e. g. item difficulty) onto the scale score metric 3. Conducted a within-student comparison of scale scores across constructs to determine relative strengths and weaknesses 4. Conducted a within-construct comparison of a student’s scale score to item responses (e. g. difficulty) to generate appropriate goals for identified weaknesses Zarrow Center

Overall Score • The overall score is a weighted combination of all items. • The overall score is not an average of all the construct scores. Zarrow Center

The Online TAGG and Result Profile SAMPLE TAGG SCREEN SHOTS Zarrow Center

Zarrow Center

Zarrow Center

Zarrow Center

Disability Awareness Profile Zarrow Center

Combined Score Profile Zarrow Center

Combined Score • The overall score is a weighted combination of all items. • The overall score is not an average of all the construct scores. Zarrow Center

Greatest and Relative Strengths Zarrow Center

Areas of Greatest and Relative Need Zarrow Center

Summary Statement for IEP Wilma Mankiller’s skills were assessed using the TAGG, a norm-referenced assessment with research-based items known to be associated with post-school employment and education. Compared to similar students, Chad’s scores are average. Results indicate greatest strengths are in the areas of Goal Setting and Attainment. Chad’s relative strengths include Disability Awareness and Student Involvement in the IEP. Greatest needs are in the area of Strengths and Limitations, with Employment being a relative need. Zarrow Center

Suggested Annual Transition Goals To prepare for success in employment, the student will write an essay describing three situations where the student used his or her strengths with 90% grammar and context accuracy by the end of the essay writing unit. Zarrow Center

OBTAIN THE TAGG Zarrow Center

TAGG Details • $3 per set (Professional, Student, Family versions) – Used to pay for on-going TAGG development and operational costs • TAGG profiles saved for 7 years • Data kept on OU high-speed secure cloud servers • Purchased credits may be transferred to other registered TAGG users Zarrow Center

Zarrow Center

Zarrow Center

TAGG Web Site Location 1. The OU Zarrow Center’s Web Page http: //zarrowcenter. ou. edu 2. The TAGG Section of the ZC Web Page https: //tagg. ou. edu/tagg/ Zarrow Center

Teaching TAGG Skills Zarrow Center

Lesson Package to Teach • Strengths and Limitations • Disability Awareness • Interacting with Others Zarrow Center

Screenshot of the Me! logo. Lessons for Teaching Self-Awareness & Self-Advocacy Zarrow Center

• Getting Started Units – Understanding Self-awareness & Self-advocacy – Understanding What It’s all About • Learning About Special Education – Learning About the History of Disability – Learning About Special Education: How & why did I get here? – Creating My History • Understanding My Individualized Education Program – Getting to Know My IEP – Still Getting to Know My IEP Zarrow Center

• Understanding My Rights and Responsibilities – Learning About My Rights & Responsibilities in High School – Learning About My Rights & Responsibilities After High School – Where do I go from Here? • Improving My Communication Skills – Learning How to Communicate Effectively – Knowing What to Share and Who to Share It With Zarrow Center

• Increasing My Self-Awareness – Starting My Self-Awareness Project – Completing My Self-Awareness Project – Presenting My Self-Awareness Project • Advocating For My Needs in High School – Planning How to Advocate – Learning From Experience Zarrow Center

• Advocating For My Needs After High School – Using My New Skills on the Job – Using My New Skills at Postsecondary School – Reporting My Findings • Developing My Resources – Completing My Summary of Performance and Goals • Assessing My Progress & Portfolio – Assessing My Progress – Assessing My Portfolio Zarrow Center

• Time to Teach Details – Approximately 20 hours – 23 sessions taking 45 -60 minutes each • Where to Teach – Resource English, Social Studies, Transition class or Study Skills class • Cost – FREE – download at http: //zarrowcenter. ou. edu/ Zarrow Center

Screenshot of the Me! logo. Transition Bell Ringers Zarrow Center

Details • Time to Teach – 5 -10 minutes once a week at the beginning of the class period – 150 Transition Bell Ringers Total (50 Elementary Transition, 50 Secondary Financial Literacy) • Skills – self-awareness – disability awareness – goal setting – knowledge to lead an IEP • Cost – FREE – download at http: //zarrowcenter. ou. edu/ Zarrow Center

Lesson Package to Teach • Persistence • Goal Setting and Attainment • Support Community Zarrow Center

Zarrow Center

Take Action Major Steps: Plan • • Establish standards Determine how to get feedback Identify motivation to attain goal Select strategies, support and schedule Long term goal is get my drivers license. Short term goal is learn to drive. Student plan row has 6 boxes. Standard – what will I be satisfied with. Motivation - why do I want to do this. Strategy – what methods should I use? Schedule – when will I do this? Support – what help do I need. Feedback – how will I get information on my performance. Student standard is get her drivers license by the time school starts. Motivation is to be able to drive herself places and not be picked up by her mom. Strategy is practice driving. Schedule is 4 to. A 5 friend on Wednesday and To be able to her Get her Practice 4: 00 -5: 00 who What Thursday. is a herself friend whodriving can drive. Feedback is what her can passenger about her drivers Supportdrive Wednesday drive sayspassenger places not says about her driving. license by and be picked up the time by her mother school and brothers starts. Zarrow Center Thursday driving

Take Action Major Steps: Action Check each plan part for action Get her drivers license by the time school starts To be able to drive herself places not be picked up by her mother and brothers Zarrow Center Practice driving 4: 00 -5: 00 Wednesday and Thursday A friend who What her passenger can drive says about her driving

Take Action Major Steps: Evaluate Understand why action was or was not taken Screen shot of evaluate section – understand why action She wasdidn’t or was not taken say I really want my driver’s license before school starts. Zarrow Center I really need to I had to wait until anything and made My friend my friend was practice! didn't’t give me me laugh. ready. any!

Take Action Major Steps: Adjust Propose changes to actions that did not work Screen shot of proposed changes to actions that did not work. Standard and schedule marked no for the action row. In the evaluate row standard motivation, and strategy marked yes. Schedule, support and feedback marked no. In the adjustment row, student opted to keep standard, motivation, and strategy. Changed schedule to when my mom off friend didn’t I had to. Changed wait until feedback work. Changed support to only go with my. My mom knows She didn't’tto say I really want my I really needmy to mom. my friend was driver’s license before anything and made give me any! what practice! schoolshe starts. is talking about. ready. me laugh. Keep Zarrow Center Keep When my mom is off work. Only go with my mom! My mom knows what she is talking about.

Take Action Major Steps: Adjust Adopt suggested plan adjustments Screen shot of sample Take Action Major step for adjust. Student wrote Keep in the first row for standard, motivation, strategy. For schedule change to when my mom is off of work. Support changed to only go with my mom. Feedback revised to read my mom knows what she is talking. My mom knows When my kept mom and about. The Keep second states the adjustment steps new Only goones with that what she is Keep is off work. my mom! talking about. were added. Get her drivers license by the time school starts To be able to drive herself places not be picked up by her mother and brothers Zarrow Center Practice driving Mom’s day off, Saturday @ 4: 00 Mom and only mom! What mom tells me.

Revisions to Take Action Addition of Reminder Screen shot of revisions made to the Take Action Schedule Plan. A reminder titled How Will I remember is added to the schedule section. Reminder How will I remember? Zarrow Center

Employment Zarrow Center

Student Involvement in the IEP Zarrow Center

Self-Directed IEP Steps • State Purpose of Meeting • Introduce Team • Review Past Goals • Ask for Feedback • State School and Transition Goals • Ask Question If Don’t Understand Zarrow Center • Deal with Differences in Opinion • State Support Needs • Summarize Goals • Close Meeting • Work on Goals All Year

Zarrow Center

Zarrow Center

FREE – download at http: //zarrowcenter. ou. edu/ Zarrow Center

Use to Collect Meaningful On-the-Job Assessment Data Results Zarrow Center

Curriculum Guide Sections • • • Making Choices Exploring Choices Testing Choices Final Choices Finding a Job Matching Skills & Preferences Solving On-The-Job Problems Zarrow Center

Methods • Assessments – Use Repeated Measure Situational Assessment Process – Characteristics I like – Jobs I like – Tasks I like • Solving On-The-Job Problems – Use of self-management skills • Self-Evaluation • Adjustment • Goal Setting Zarrow Center

Zarrow Center

Zarrow Center

Find These Lessons and More at FREE – download at http: //zarrowcenter. ou. edu/ Zarrow Center

Contact Information Jim Martin jemartin@ou. edu Amber Mc. Connell ambermcc@ou. edu Phone: (405) 325 -8951 Website: https: //tagg. ou. edu/tagg/ Zarrow Center