Creating a Comprehensive Bank of Materials for Use

Creating a Comprehensive Bank of Materials for Use in Applied Statistical Methods (Stat 1000) DR. BRIAN LEVENTHAL, PSYCHOLOGY IN EDUCATION (CURRENT: ASSESSMENT SPECIALIST AT JAMES MADISON UNIVERSITY) DR. NANCY PFENNING, SENIOR LECTURER, DEPARTMENT OF STATISTICS

Introduction v. Proposed a transformation of materials for Stat 1000 in order to consolidate and enhance existing materials • Goal: ensure fair and appropriate content for all instructors – focus on new hires v. Starting point: examination of existing instructional and assessment materials developed by various faculty to date. • Considered Power. Point slides, assignments, projects that require statistical software, quizzes, midterm and final examinations v. Goal: organize a coherent collection of materials that make it possible for a less experienced lecturer to “hit the ground running” with high-quality tools that are consistent with Stat department’s goals for instruction in that course BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 2

Stat 1000 Teacher Evaluation Spring 2016 v. Adjunct X: Mid-semester survey “It seems like he was tossed to the wolves at the last minute and given some class materials and told to go run this class… We have had sporadic quizzes(2 so far) and they have been on material that is several lectures back. We could be on lecture 12 in lecture, talking about lecture 7 in recitation and doing homework on lectures 4 and 5 and a quiz on lectures 7 -10. That seems how this class has been going. It was like that one week for recitation/homework/quiz. Im honestly thinking about sitting down with someone in the STAT department to talk about this class. He's well prepared for class in a sense of knowing the material and teaching it. He is not well prepared for class in regards to the other areas like quizzes, homework, recitation, stat lab, etc. ” BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 3

What has been developed? v Welcome document by Leventhal, Pfenning, and Bodenschatz v. Example Syllabi v. Example Power. Point Slides v Homework Assignments v Example Quizzes v Example Lab Documents and Assignments v. Teaching Assistant and Grading Guidelines v. Final Exam Item bank BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 4

Example Syllabi There are three example syllabi provided: 1. Mon/Wed/Fri (50 min) Fall course 2. Tues/Thurs (75 min) Fall course 3. Tues/Thurs (75 min) Spring course To be developed: schedule for MWF (50 min) Spring semester course BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 5

BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 6

Example Power. Point Slides The provided [optional but recommended] Power. Point slides are developed to be paired with the syllabus of the Tuesday/Thursday course. You may use these exact slides or make modifications. You may also use them for a course that meets on Monday, Wednesday, and Friday, making the necessary adjustments to match the schedule. If you do use Power. Point presentations, it is recommended that you post your notes on the class’ Course. Web website (Pitt’s version of Blackboard) prior to lecture. This will allow your students flexibility if they prefer to print the notes prior to lecture and take notes directly on the slides. Students have provided positive feedback to lecturers who provide the notes before the class that they present the material. ” BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 7

Homework Assignments “Having weekly assignments for this quantitatively intensive introductory statistics course is highly recommended…Most STAT 1000 instructors use items from the textbook for homework assignments. Provided are example homework assignments (unrelated to provided syllabus). We like to recommend that STAT 1000 instructors choose their own items from the text for homework assignments. Since not all lecture sections follow the same schedule, we do not want one lecturer providing solutions for the same items that students from another lecturer are still trying to solve. There will be overlap but ideally all instructors should choose the items that they think best work with their teaching style and emphasis of material. “ BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 8

BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 9

Example Quizzes “We recommend that you assess your students weekly in recitation. You may do this using an open book/open note quiz that counts for a grade, a closed-book/open-note quiz that counts for a grade, or lab problems for which full points are awarded as long as they are completed. Having students assessed most weeks provides them with extra practice in the material. Additionally, it provides the students and the instructor an idea of how well the students are retaining, understanding, and applying the material. Typically, attendance is not required for recitation but a quiz or lab is a good way to incentivize attendance. ” BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 10

BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 11

Example Lab Documents and Assignments “Use of MINITAB is required for STAT 1000. It is recommended that you have your students attend recitation in the Stat Lab at least three times throughout the semester. Recitations in the Stat Lab may consist of lab activities or assignments. Although you do not have to assign students homework using MINITAB, MINITAB output is typically seen on exams. Thus, it is important for students to have practice either in recitation or on assignments generating and interpreting output using MINITAB. Note: MINITAB is free to download for students from the Pitt Software Download center through the My. Pitt Portal. ” BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 12

BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 13

Teaching Assistant Recommendations The instructor for the STAT 1000 course will be assigned a teaching assistant to cover each of the recitations. It is typically one TA for all recitations but due to scheduling conflicts there may be more than one assigned teaching assistant. If one TA is assigned to a single recitation and another TA is assigned to the other three recitations, consider having the first TA completely grade about one fourth of the weeks that assignments are turned in, and the other completely grade the remaining three fourths of the time. (Students often compare graded papers and are quick to pick up on inconsistencies if more than one TA grades the same assignment. ) You should grant your TA access to your course’s Course. Web website with a Teaching Assistant role. This way they can directly input grades, access course material, and keep up to date if you make any announcements to the class. BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 14

Teaching Assistant Grading Guidelines It is recommended that the TA grade weekly homework assignments and quizzes. Since recitation does not fall on the same day for all students in a given lecture, it is recommended that students submit assignments during lecture. This assures fairness in time to complete the assignment. You may have your TA return the assignments and answer questions about the assignments in recitation or office hours*. If quizzes are given in recitation, it is recommended that the TA be the grader and enter the scores into Course. Web. This is only a recommendation and the instructor may grade any quizzes or assignments according to preference. It is recommended that the instructor grade the examinations. You may seek assistance from your TA in grading but it is strongly encouraged that, at a minimum, the instructor has oversight of the examination grading. BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 15

Final Examination - Topics and Number of Items DESCRIPTIVE STATISTICS AND DESIGN (180) INFERENCE ITEMS (185) Choosing the correct test: 20 items Confidence Interval and Tests for Mean (Z) : 26 items Data Types; Study Designs: 17 items Exploratory Data Analysis: 38 items Exploratory Data analysis for two-way tables: 17 items Exploratory Data Analysis for regression: 42 items Probability: 15 items Probability distributions: 27 items Sampling Distributions: 4 items Confidence Interval and Tests for Mean (t): 17 items Confidence Interval and Tests for Independent t: 15 items Confidence interval and tests for paired t: 12 items Confidence interval dn tests for proportion: 32 items Confidence interval and tests for two proportions: 6 items Chi-Square Analysis (goodness of fit and tests of independence): 24 items ANOVA: 30 items Inference for Regression: 15 items Multiple Regression: 8 items BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 16

Example Multiple Choice Items BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 17

Example Constructed Response BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 18

Step #1 v. Surveyed adjunct lecturers about usefulness of materials provided • STAT 200 Instructors: Overall material provided was helpful and established a useful starting point in developing the course • STAT 1000 Instructors: Lack of material provided. BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 19

Step #2 v. Material gathered from experienced lecturers Drs. Bodenschatz and Chen. • Midterm and final examinations • Quizzes • Homework assignments • Exam practice material • Lab assessments v. Course syllabi: frequency of quizzes, examinations, lab assessments, homework assignments v. Proportions with A grades for students from various lectures of STAT 1000 in recent semesters are being examined to arrive at reasonable guidelines BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 20

Step #3 v. Ongoing Discussion with experienced lecturers Drs. Bodenschatz and Chen v. Discussions include gaining recommendations for • Homework assignments • Laboratory assessments • Number of examinations • Other useful ideas relevant to incoming adjuncts or new full-time faculty • What material is considered most important for assessment in final exams BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 22

Step #4 v. Maintained and handed off prepared Stat 1000 materials in AY 2016 -2017 • Brian Leventhal taught with all provided material, confirmed all aspects to be in working order [extremely high OMETs] • Stats junior faculty was provided with material, found it to be very useful BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 23

Step #5 v. Created annotated test bank for final exams • Contains a variety of constructed response and selected response items • Consists of statistical basics, choosing the correct analysis procedure, and interpreting output from statistical software • LOTS of editing of previously developed items. • Items are classified by topic BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 24

Step #6 v. Handed off more comprehensive packet of materials to Summer and Fall 2017 Stat 1000 adjuncts, who were given a list of • Final examination assessment items • Sample syllabi • Sample homework assignments • Sample lab assignments • Sample midterm examinations • Course organization recommendations. BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 25

Overall Discussion Had initial meeting with 5 new STAT 1000 instructors to discuss details; prospective instructors were truly grateful for the material. Questions? Live documents will continue to grow as more experienced lecturers/instructors contribute to the content. brl 38@pitt. edu Comments? nancyp@pitt. edu Next Step: STAT 1100 BRIAN LEVENTHAL (BRL 38@PITT. EDU); NANCY PFENNING (NANCYP@PITT. EDU) 26