CEM NZ Centre for Evaluation Monitoring College of

CEM (NZ) Centre for Evaluation & Monitoring College of Education Using Mid. YIS 9 to inform Teaching & Learning Dr John Boereboom Director Centre for Evaluation & Monitoring (CEM) University of Canterbury Christchurch John. boereboom@canterbury. ac. nz

The Centre for Evaluation and Monitoring • Established in 1999 • Part of Canterbury University • Each year about 300 NZ schools and 70 000 students use our services. • Evaluates and monitors progress of students, subjects and schools to enable evidence based decision making. • Provides Entrance Tests to assess mathematics, English and Reasoning Skills for class placement and identifying strengths and areas for development. • Administers student attitude and engagement surveys to investigate factors that can affect progress.

Why use Mid. YIS 9? • League tables are an unfair comparison of schools and do not recognise the impact a school has on student progress. • Unique value added analysis. • Comprehensive on line easy to understand share feedback eg subject, HOD and staff meetings and BOT. • Sound data for evidence based educational decision making at the student, subject and school levels. • An independent measure of school effectiveness. • Based on international research adapted for NZ.

Mid. YIS Yr 9 Middle Years Information System • Provides value-added measurement from the beginning of Year 9 to the end of Year 11 in about 20 different subject areas. • Students sit a baseline Mid. YIS test at the beginning of Year 9. • The first diagnostic feedback is provided to schools and targets are provided based on national data from the previous year. • At the end of Year 11, CEM receives NCEA results from NZQA. • The predicted scores are compared to the actual NCEA scores and the value added is calculated. • Comprehensive online feedback is provided to analyse the performance of: • The individual student • The subject cohort • The school

What does Mid. YIS 9 measure? Literacy • important to the prediction of all subjects, especially English, foreign languages and language-related subjects such as history. Numeracy • important for prediction to mathematics and mathematics-related subjects such as design, technology, science and economics. Reasoning skills • how well students undertake tasks involving 3 -D visualisation, spatial aptitude and pattern recognition. Non verbal skills are particularly important for predicting to a range of subjects that includes mathematics, visual arts, drama and design. Processing speed and accuracy • proof reading and perceptual speed and accuracy.

The first feedback Mid. YIS scores are standardised on a nationally representative sample to have a mean score of 100 and a standard deviation of 15. The scores for each sub test and the overall score are normally distributed. • The average pupil in the nationally representative sample will score 100 • 95% of pupils score within about two standard deviations of the mean, which means 95% of pupils will score between 70 and 130 • Pupils scoring over 130 are in the top 2½ percent of the national sample and meet the traditional definition of being gifted • Pupils scoring less than 70 are in the lowest 2½ percent of the national sample and may have special educational needs

Interpreting the IPR Information from the IPR: Overall the pupil is a fairly average pupil but her mathematical ability is significantly less than her Vocabulary (which is well above average). This might suggest there will be some underachievement in mathematics and in parts of subjects where mathematical skills are required, notably science. The best score is in Non-verbal ability where, like in Vocabulary, the pupil has scores in the Band A, the top 25% of all pupils nationally. Her Skills score is roughly average and although not significantly, is lower than her Vocabulary. This might affect her performance in examinations.

The second ‘Value Added” feedback Comprehensive online feedback is provided at the: • Student Level • Subject Level • School Level http: //www. atlanta. k 12. ga. us

How is the value added score calculated? Year 11 NCEA results are combined into a single number (the NCEA discrimination score) using a simple algorithm and plotted against the Mid. YIS Year 9 results to establish a national regression line which represents the predicted expected performance. The individual student’s NCEA discrimination score is plotted on the same graph. The vertical distance between this score and the regression line represents the value added scores. Value added scores are standardised:

The concept of Value Added http: //www. slideshare. net/Robert. Leneway/aclcurinstassess Beyond Expectation +ve Value. Added In line with Expectation 0 Value-Added Below Expectation -ve Value-Added

The Student Report

The Subject Report

The School Report

How can schools use the feedback? The comprehensive on-line feedback can be used to: • Identify student strengths and weaknesses and set realistic and motivational targets; • Identify students requiring support at the individual subject level or across a range of subjects; • Identify and share best practice and measure teachers’ influence on the academic growth rates of students; • Identify high achievers or under-aspirers; • Inform departmental planning and the School Improvement Plan; • Determine where curriculum and instruction is having the greatest impact on student learning; • Evaluate the effectiveness of programs.

What about SES? Each student serves as his or her control and thus eliminates factors like socio economic status that may affect the learning outcomes of students. “Given similar student abilities, the choice of school a student attends is not among the more powerful influences on later achievement. ” Hattie, J. (2002) Schools Like Mine: Cluster Analysis of New Zealand Schools Technical Report 14, Project as. TTle, University of Auckland

Other services provided by CEM Student attitude surveys • Student Welfare & Safety • Non-academic activities • Support • Social and personal development • Extent of Bullying • Extent of Racism • Healthy Lifestyles School entrance tests • Mathematics, English and Reasoning skills Ye. LIS and Se. LIS