Coding OpenEnded Comments Lib QUAL Survey Results Best

Coding Open-Ended Comments Lib. QUAL+® Survey Results Best Practices: Stories from the Field January 14, 2009

The search for coding information began Feb. 24, 2008 General Query sent to LIBQUAL-L@LISTSERV@TAMU. EDU “Can anyone share information about how they coded the open-ended comments from the Lib. QUAL+® survey? ”

Lib. QUAL+® Listserv Query - Results • Software utilized • Influence of theories • Coding to • Themes/Topics • Lib. QUAL+ dimensions • Value/Emotional Content • Question Numbers • Named departments/persons • Lib. QUAL+ questions • Demographics

Software. Local Decision Influences • Campus Users • Review (Journal of Mixed Methods Research) • Support, company and local • Cost

Assistance/Consultations for Coding Process Lib. QUAL+® listserv software; categories; resources OSU Qualitative Research Faculty importance of data immersion; resources QSR Website and Customer Assistance tutorial; questions; tips; importing data Faculty member using NVivo demonstration; questions Systems / IT software; importing data

DATA REDUCTION At OSU, 1, 067 Comments (38. 8% of respondents) Reduced to 853, by eliminating • small, specific data sets • “other library” • “no comments” • very general comments that didn’t say anything of use • survey population (faculty, staff) • comments about the survey itself

Of the demographics provided by Lib. QUAL+® We considered: • What Fields Mattered to Us? • What would most impact our services? Our decision: • Classification • Discipline • Library Used Most Often

Sample Tree Nodes, dictated by the data • Books • Hours • Building • Instruction • Online • Library Resources • Services • Journals • Resources • Study • Parking • Technology

Values, defined locally Praise- commend; “express favorable judgment”; approval; satisfaction (positive comment) Words such as: like, love, enjoy, happy, good, important, glad, pleased, satisfied, positive, helpful Strong Praise- effusively positive; detailed exclamatory positive statement; explanatory or descriptive reason for praise Words such as: great, excellent, amazing, fabulous, best, outstanding, always, thanks, grateful Merriam-Webster’s collegiate dictionary (11 th ed. ). (2004). Springfield, MA: Merriam-Webster.

More Values, defined locally Complaint- expresses discontent; mildly negative; unfavorable (a negative comment) Words such as: more, cannot, never, hardly, often, sometimes, isn’t, lacks, lack of Strong Criticism- severely unfavorable; hostile; pessimistic; fault-finding; points out flaws; clearly negative incident or encounter (explains or describes reason for criticism) Words such as: hate, frustrating, embarrassing, irritating, nightmare, unacceptable, never, impossible

More Values, defined locally Suggestion- “mentions or implies a possibility”; “offer for consideration”; “propose as fitting”; (concrete suggestion) Words: more, should, would like, would be, would appreciate, wish, could have, could be, needs, add, improve Neutral Observation- “not engaged on either side”; indifferent; not positive or negative, just an observation Merriam-Webster’s collegiate dictionary (11 th ed. ). (2004). Springfield, MA: Merriam-Webster.

Number of Comments by Value

Coding Open-Ended Comments Challenges • Timeframe • Importing data into software • Inter-rater reliability Benefits/Uses of Data • Software • Strategic Planning • Library Development • Innovative Ideas • Emergence of strongest themes

Karen Neurohr, Assessment Librarian Oklahoma State University karen. neurohr@okstate. edu