Survey exercise followup Survey population o Sample n

Survey exercise follow-up

Survey population o Sample? n n n o Random Purposive Convenience Entire population n Small, bounded population o o All the people who work here Non-response bias an issue in each case

Sample design o Survey element – unit ABOUT WHICH information is collected; unit of analysis n o Survey universe – theoretical and hypothetical aggregation of elements n o E. g. , Person; household; event E. g. , Americans; users; visits Population – specified aggregation of survey elements: n n Households in the US, defined as one person, or a collection of related or unrelated persons…as of March, 2002 Visits to any web page in defined domain during March, 2002

Sample Design, cont. o Survey population -- Aggregation of elements from which survey sample is actually selected n o Households in 50 states with listed telephone numbers Sampling unit – elements consider for selection at some stage n n listed residential phone numbers If multi-stage sample: o o Sampling frame – list of sampling units n o State; telephone directory; phone numbers Telephone directories for specified places Observation unit – element from which info collected n person over 18 who either answers the phone or is fetched by a person under 18 answering the phone

Sample selection o o Consider possible biases of sample design Mixed methods may give uninterpretable samples n E. g. mail to a group AND hand out in Sproul Plaza – so what population does your sample represent? o o And people on the mailing list who walk through Sproul have *2* chances of being sampled Consider practical issues n Confidentiality often precludes getting a list, e. g. from the university n If you can’t quantify units (people, events), you can’t sample proportionately n Non-users or non-affiliated users (e. g. , potential students; web users with no particular connection to the site) are hardest to track down

Introducing survey o Purpose of introduction n Justify your asking Induce their cooperation Allay possible fears o n o “this is not a test…” Help them understand domain of survey, questions Describing purpose of survey n Enough to gain cooperation, not enough to bias answers o n E. g. “a study of compensation in this field, ” not “a study of whether women’s salaries lag behind men’s in this field” SHORT but not at the expense of clarity.

Getting cooperation o Worthwhile topic, survey n n n o Good design n o Not threatening Not a waste of time Interesting to them If you are not credible, not worth their effort Short and easy to answer

Directions o o o Short and clear People skim! The more you ask of them, the more likely they are to mess up n o “please rate from 1, most important, to 7, least important” Give instructions n n For skip patterns o “Did you use the library today? If not, go to question 3. If yes, … For what to do if question does not apply

Choice of questions o o Think through: what will you do with the answers? If you don’t know, don’t ask If you do know, word in a way that helps you E. g. “Have you ever used a library? ” n What difference will it make if they answer “no”?

Choice of questions o Respondents must be qualified to answer n o “What would be the best way to bring peace to the Middle East? ” Respondents must think the topics of the questions are: n n n Not invasion of privacy Not “dumb questions”; worth their time Not a test o Of them o Of some idea or proposal they may not like (unless you mean it) n “if we had to cut the budget, should we lay people off or reduce salaries? ”

Time dimension to questions o General, usual, most often: n o Specific n n o What did you do today? What have done in the last month? NOT: n o What do you usually do? What do you use this site for? (today? Ever? ) Memory n n What’s important to you may not be to them o “what did you do the last time you used this site? ” Time and frequency are hard to remember o “Approximately how many times in the last month…”

Question wording o Resolve any possible ambiguity! n Definitions, illustrations, examples o o Have you ever used a library? What’s a library? “Did you use the library’s homepage? ” n o What proportion of your income do you pay for housing? n n o Which page is that? Individual or household income? Housing: rent or mortgage; taxes? Insurance? You cannot use “etc. ” unless the reader can complete the list n n NO: “borrow library books, ask ref questions, etc. ” YES: “borrow library books, ask reference questions, or otherwise use the library’s resources or staff, in person or remotely. ”

Use parallel construction o What did you use this site for today? Searching for information n Placing an order n Checking a previous order NOT n Searched for information n I wanted to check on an earlier order n o What did you use this site for today? If yes… n What was the question to which “yes” was the answer?

Answers o Closed-ended are easier (which is usually, but not always, a good thing) n n o Easier for users (recognition, not recall; less writing) o They are less likely to think of different choices Easier for you to summarize Be as exhaustive as possible with choices n n If you leave out an answer that a user might reasonably give, they may or may not think to add it They may become confused about the question or the survey

Answers, cont. o Allow degrees of answers n n o Yes/ No, always/never are rarely the case E. g. , Usually; Sometimes; Rarely Allow multiple dimensions of answers n How satisfied are you with your visit today? o Very: found what I was looking for. o Not very: did not find. . o What if I found what I needed but with difficulty? I may have been successful but am dissatisfied with the experience.

Answers, cont. o Give clear instructions n n o Check one? Check the one that best describes…? Check as many as apply? Define! Lay out clearly n E. g. Ordinal scales all on one line o How satisfied are you? n n o Very satisfied OR Very Satisfied somewhat satisfied not at all satisfied NOT: How satisfied are you? Very satisfied Somewhat satisfied Not at all satisfied

Ranges and intervals o o Want answers to be easy to for your reader to interpret Generally, use either: n n Intervals of equal size OR Intervals reflecting meaningful groupings o n Age: under 18 25 -40 40 -55… OR intervals get larger when you expect fewer responses o How many years have you worked here? Less than 1 o 18 -25 1 -3 4 -10 10 -20 more than 20 Avoid: n Years worked here o o 1 year 2 -5 years 6 -7 years 8 -15 years 10% 15% 20% 50%

Examples o Have you checked out a book from the UC Berkeley Library system? n o Yes no If you use Pathfinder, Gladis, or Melvyl to find books or periodicals, where do you use them? n n n Terminal at main library Terminal at my departmental library I don’t use these services

Examples o If you could allocate funds for more salary and had to pick from the following places, how would you prioritize where you would obtain these funds? (1 – first choice to find money through 7 – last place you would go) n tuition n Pension plan n Taxes n Decreased health coverage n State bonds n Fund raising n Less vacation/sick leave

Examples o o How long has your library offered live online reference service? How long have you been a Reference Librarian? (providing reference services comprises at least 40% of your day) n n Up to 5 years Between 5 and 10 years Between 10 and 15 years Over 15 years

Examples o o Which part(s) of this web site interests you the most? Which part(s) of this web site do you find the most uninteresting? What parts of this web site do you like the most? Which parts of this web site do you dislike the most?

Examples o 1. Have you ever used the following online services: n o [list] 2. If you checked any box in question 1, how important do you think online help would be in each of these activities? (only respond as applicable) n [Item] n Absolutely unimportant |--|--|--| absolutely important