What is a Knowledge Survey Introduced by Nuhfer

What is a Knowledge Survey? • Introduced by Nuhfer (1993, 1996), Knipp (2001), and Nuhfer and Knipp (2003) • Series of questions that cover the full breadth of a unit or course • Content and skills objectives • Questions cover different levels of understanding (e. g. , Bloom, 1956) • Students indicate perceived ability to answer questions

Example Knowledge Survey Questions Question According to Bloom, what are the different levels of understanding? Bloom Level Knowledge Explain how the “wiring in the brain” can change with age and use. Comprehension Use the elements of critical thinking to analyze a sample of writing. Application Compare the different levels of reasoning according to Bloom’s taxonomy. Analysis Construct a diagram that synthesizes Bloom’s levels of understanding with Fink's (2003) model of significant learning Synthesis Using the list of “qualities that distinguish outstanding students, ” appraise your own study and learning habits and suggest a course of action. Evaluation

Knowledge Survey Answers a) I do not understand the question, I am not familiar with the terminology, or I am not confident that I can answer the question well enough for grading purposes at this time b) I understand the question and a) I am confident that I could answer at least 50% of it correctly, or b) I know precisely where to find the necessary information and could provide an answer for grading in less than 20 minutes c) I am confident that I can answer the question sufficiently wellenough for grading at this time

Creating Knowledge Surveys Compile old exams, quizzes, activities, exercises Define enduring concepts & learning objectives Determine content and skills Write questions that illustrate levels of understanding Sorted by textbook chapter, source, or topic

Administering Knowledge Surveys Paper Electronic Out-of-class Data management and analysis more efficient Survey Responses Not Graded Beginning of Course Pre-Exam Post-Course In subsequent years?

Electronic Knowledge Surveys Text Format of Input File Explain the elements of critical thinking? {~No answer at this time ~Know more than 50% =Know detailed answer} Format in Moodle

Graphics in Knowledge Surveys

Utility of Knowledge Surveys Course Design Clarification of course objectives and structure Improved organization and preparation

ject s A Typical Course … Quizzes F s a t Skills cts n e m n s g t i s c S s k A a ills F Readings Facts Papers Discu s s i Facts o s n s s t c m a a Facts x F E Fa cts ct s Fa Pro Objectives Enduring Concepts lls i Sk

A Course That Utilizes Knowledge Surveys Objectives Enduring Concepts Supporting Skills and Facts Assessments (Formative & Summative) Instructional Practices & Learning Experiences

Utility of Knowledge Surveys Course Design Clarification of course objectives and structure Improved organization and preparation Students Full disclosure of course objectives and expectations Study guide Formative assessment tool Development of self-assessment skills

How Students Use Knowledge Surveys

How Students Use Knowledge Surveys How has the knowledge survey contributed to your learning or changed the way you learn? Higher Bloom level questions force me to think critically about the material before the exam It challenges me to consider the level of understanding, and I can now apply that to all my classes. It has made me evaluate my knowledge and way of thinking. It is good to have such an in-depth guide because sometimes I don’t even realize I don’t understand something until I am asked to explain it.

Utility of Knowledge Surveys Course Design Clarification of course objectives and structure Improved organization and preparation Students Full disclosure of course objectives and expectations Study guide Formative assessment tool Development of self-assessment skills Instructors Assessment of learning gains Course assessment Assessment of instructional practices

Topics and Bloom Levels of Questions

Two years results for 190 questions: confidence before and after course

Knowledge Survey & Exam Results Exam II

Confidence Before and After Course 2004

Confidence Before and After Course 2005

Survey and Examine Results 2004

Survey and Examine Results 2005

Course Grade vs Survey or Exam

Comparing 2005 Students with those from Several Years Ago

Confidence by Bloom Level: Current and Past Students

Learning Through the Curriculum

How Students Use Knowledge Surveys Should the use of knowledge surveys in this course be continued? Yes, they should be continued, but one possible danger is that it can be good practice to try to figure out [on your own] what is most important about certain topics/ideas.

How Students Use Knowledge Surveys Should the use of knowledge surveys be continued in this course? No. I spent considerable time in high school discussing learning and feel comfortable with the way I learn and study. It is almost ridiculous to think a college student will take the time to not only do an assignment, but then reflect on what they learned and assess themselves.

How Students Use Knowledge Surveys Describe how you use knowledge surveys in this course? I read through the relevant sections every couple of weeks to see what is expected of me. Before the test I didn’t use it, but now I answer the questions as I go through the reading. Used it as a guide before I re-read the course readings as preparation for the exam. Before the exam I didn’t use it, but now am following along with every class period and reading to keep it up to date.

How Students Use Knowledge Surveys Introductory Course Upper Level Course

Knowledge Survey 1. What is a knowledge survey? 2. Describe how knowledge surveys differ from exams. 3. Use a knowledge survey to assess learning and instructional practices in your course. 4. Compare courses with and without knowledge surveys. 5. Design an integrated assessment plan for your course that includes knowledge surveys. 6. Appraise the validity and utility of knowledge surveys. a) I don't know the answer b) I know more than 50% c) I know the answer

Knowledge Survey Results

Knowledge Surveys An Indispensable Course Design And Assessment Tool Karl Wirth Macalester College Dexter Perkins University of North Dakota