User studies LIS 670 BairMundy Users Potential users

User studies LIS 670 Bair-Mundy

Users Potential users Expected users Actual users Beneficiary users

Studying users What we need to know Types of user studies Designing a research plan Data collection methods Problems in user research User study assignment

What we need to know (1) Information needs of users Request for Assistance Name______ Address_____ Telephone_______

What we need to know (2) How users utilize information

What we need to know (3) How the user attempts to fulfill his/her information needs Information-seeking activities User reasoning processes

What we need to know (4) Means of presentation of information that are intelligible and thus useful to the user

What we need to know (5) Trustworthy sources of information for the user

What we need to know (6) Elements of a user interface that expedite or hinder information retrieval Enter search statement in the box below START CAT FIND XT GONE WITH THE WIND

Intelligible and useful means of presentation of information

Searching by color http: //portia. nesl. edu/screens/well_its_red. html

Types of user studies User oriented Systems-use oriented Utility oriented

User-oriented studies Seek to utilize demographics to predict information use Age Education level Economic status Language

Demographics - age http: //www. lapl. org/

Demographics - language http: //www. lapl. org/espanol/index. html

Systems-use oriented studies Extent to which users utilize one or more kinds of information system Degree of difficulty or satisfaction in using information services Characteristics of the system contributing to success of information transactions

Elements examined in systemsuse oriented studies (1) Number & type of reference questions asked Number of items loaned in-house Number of items accessed through interlibrary loan

Elements examined in systemsuse oriented studies (2) Number of times users access online catalog from remote sites Number of times users access particular online databases Search and retrieval statistics for online databases

Ethical consideration in systems -use studies: user privacy Aggregate data about systemsuse is acceptable Remove any information that could identify individual Keep only data necessary for providing services for only as long as needed

Utility-oriented studies Critical incidence studies Subjects report on information needs and their resolution at each decision-making point in a specific search Citation analysis Analyze citations to articles by particular author or published in a particular journal

Process of user research 1. Design of research plan 2. Implementation of research plan 3. Implementation of results 4. Evaluation of results

Designing a research plan: Step 1 Identify a problem area or need to study Not as easy as it sounds!

Problem: Line at the reference desk? Reference

Actual problem: lack of signage Reference Adult Fiction Children's Room Reference Desk Restrooms Science & Tech

Designing a research plan: Step 2 Conduct an initial literature review How have others approached the problem?

Designing a research plan: Step 3 Define the specific research question to study What is the optimal arrangement of information resources in the reference area?

Designing a research plan: Step 4 Cost-benefit analysis of study Study may be expensive. Will resulting suggestions be implemented?

Designing a research plan: Step 5 Conduct a second literature review Specific search now that research problem is defined

Designing a research plan: Step 6 Formulate hypotheses (specific testable question(s) to ask in the study) "Changing the configuration of the furniture to segregate incoming and outgoing traffic will result in less crowding in the lobby. "

Designing a research plan: Step 7 Identify the population to study Everyone in certain zip codes?

Designing a research plan: Step 8 Select data collection methods Phone survey? In-house questionnaire? Capture keystroke data?

Designing a research plan: Step 9 Develop data collection instruments Questions for phone survey or questionnaire http: //www. isworld. org/surveyinstruments. htm

Survey of currently-enrolled LIS students From 1 (strongly disagree) to 6 (strongly agree), rate the following statements by completing this sentence: From the courses I took in the LIS Program, I feel I gained sufficient knowledge of the. . . Rating (circle one) Strongly disagree Strongly agree 2. 1) Philosophy, principles, and ethics of library and information science 1 2 3 4 5 6 2. 2) Historical development and communication of knowledge 1 2 3 4 5 6 2. 3) Use of strategic planning techniques to create 1 short- and long-term plans for improvement 2 3 4 5 6 …

Surveying a multi-ethnic community A continuación puntúe de 1 (muy deficiente) a 6 (muy satisfactorio) las siguientes variables Grado de conocimiento que usted tiene sobre los servicios que ofrece la biblioteca 1 2 3 4 5 6 Localización de la biblioteca 1 2 3 4 5 6 Indicación de su ubicación 1 2 3 4 5 6 Infraestructura de la sala 1 2 3 4 5 6 Tamaño de la biblioteca 1 2 3 4 5 6. . .

Designing a research plan: Step 10 Design a data-analysis plan Statistical tests you plan to use will influence way you collect and encode data.

Salary data Statistical tests you plan to use will influence way you collect and encode data. What is the starting salary for librarians at your institution? $20, 000 - $29, 000 $60, 000 – $69, 000 $30, 000 - $39, 000 $70, 000 – $79, 000 $40, 000 - $49, 000 $80, 000 – $89, 000 $50, 000 - $59, 000 $90, 000 – $99, 000

Designing a research plan: Step 11 Formulate a data-collection plan Stopping people who walk in the door to question them?

Designing a research plan: Step 12 Conduct pilot studies of methods, instruments, and analysis First try questionnaire on friends and staff, then on sample from group you plan to study

Data collection methods Questioning Observing Studying records or documents

Questioning (1) Questionnaire survey—can reach widely dispersed geographic group of subjects; low return rate; low validity Interview—can give high quality data due to follow-up questions; costly; limited sample size; inconsistency among interviewers

Questioning (2) Diary method—rich informational content; low validity and reliability (answers get shorter over time) Group interview and questions— less expensive than individual interview; good response rate; includes Delphi technique

Delphi technique Ask each expert same questions Summarize responses and distribute to the group Ask for revisions of original opinions based on group response Report group response to individuals again Elicit new revisions of opinions

Observation Can be less intrusive Can be highly reliable Objective Expensive Doesn’t reflect motivation Hawthorne effect—activities of observed subjects are affected by interested presence of the observer

Hawthorne effect Lighting productivity

Documentary evidence Publications Statistical reports Citations

Publications Library Science Research Information Science Research Bibliometrics

Citations

Citations

Statistical report - databases

Statistical report - searches

Reference question tally Directional Hawaiian Pacific Rare

System-use printout OPAC Logging Report DATE TYPE SEARCH STRING LIMIT_STRING INDEX HITS 9/28/01 Keyword harry potter N K 9/28/01 Author N LOCA=Main Library B 9/28/01 Keyword birds N K 9/28/01 Keyword alligators N LOCA=Main Library K 9/28/01 Complex (NKEY alligators) AND N K 0 N K 103 N K 6848 tolkein 4068 1 2863 30 (TKEY "crocodiles") 9/28/01 Complex (TKEY rats) AND (TKEY "mice") 9/28/01 Keyword harry potter goblet

Classical experiment Formulate hypothesis Experimental group is tested in comparison with control group Must keep all other variables constant (difficult to do with human subjects)

Different reactions excellent Welcome Subject A poor Welcome Subject B

Problems in user research (1) Many variables affect informationseeking behavior Variables are difficult to identify Variables are difficult to control

Problems in user research (2) Link between cause and effect difficult to establish Notice Change of borrowing policy to take effect ? X Quality of service not readily measurable circulation

Users’ articulated demands may not reflect their true information needs Request: Need: Articles about New Zealand in National Geographic Information about business opportunities in New Zealand

Studies are often disruptive to normal operation of information-provision system Reference

Difficulty in implementing changes suggested by result of studies. Reference. .

Factors that affect quality of data Validity Is data representative of total population? Does measuring tool actually yield desired information? Reliability Stability of the measure over time (e. g. interviewer consistent with every subject? )

Assignment: Recommendation for a User Study Read scenario Analyze the problem Decide on research methodology Write letter describing & justifying methods