Research Design Procedures 1 Research Design n Forming

  • Slides: 39
Download presentation
Research Design Procedures 1

Research Design Procedures 1

Research Design n Forming your action plan n Deciding on the Who and When

Research Design n Forming your action plan n Deciding on the Who and When n Defining all concepts and terms 2

Research Design n Three purposes for research: n Exploration n Description n Explanation n

Research Design n Three purposes for research: n Exploration n Description n Explanation n Or Descriptive- existing conditions n Normative- observed vs. intended n Impact- can it be attributed to programs? n 3

Exploration n To gain familiarity with a topic n Typically done to: n Satisfy

Exploration n To gain familiarity with a topic n Typically done to: n Satisfy a researcher’s curiosity and desire to understand n “Test the water” for a more extensive study n Develop methodology to be used in a subsequent study 4

Description n Provide context for situations and events n Typically based on observation and

Description n Provide context for situations and events n Typically based on observation and reporting n Observation is systematic n Example: U. S. Census 5

Explanation n Attempts to address the question of “why” n Tries to get at

Explanation n Attempts to address the question of “why” n Tries to get at reasons and underlying causes n Example: Not “would you vote for Mc. Cain” but “why would(n’t) you vote for Mc. Cain? ” 6

GAO- Designing Evaluations n Considerations 1. 2. 3. 4. 5. 6. 7. Kind of

GAO- Designing Evaluations n Considerations 1. 2. 3. 4. 5. 6. 7. Kind of information to be acquired Sources of information (i. e. types of respondents) Methods of sampling (i. e. random) Methods of collecting data (i. e. interviews, surveys) Timing and frequency of information collection Basis for comparing outcomes Analysis plan 7

Research Design: Experiments n Experiments aim to control one variable or set of variables

Research Design: Experiments n Experiments aim to control one variable or set of variables in order to determine their relationship to and/ or impact on another set of variables. n Types of studies: n Experiment n n Uses a random sample Quasi-experiment n Does not use a random sample, must try to correct for error through statistical tests 8

Impact of a Program/Service Involving an Experimental Design IMPACT = Outcome of experimental group

Impact of a Program/Service Involving an Experimental Design IMPACT = Outcome of experimental group receiving the treatment compared to control group Information Literacy Instruction 9

Classic Design n Two group pre- and post- test n One experimental group n

Classic Design n Two group pre- and post- test n One experimental group n One “untreated” control group n Compare outcomes to assess impact n Problems with this? 10

Solomon 4 Group Test n Classic Design expanded to include two sets: n One

Solomon 4 Group Test n Classic Design expanded to include two sets: n One set has experimental group and control group who both receive pre- and post-tests n One set has experimental group and control group who receive only post- tests. n Advantages over classic model? 11

n Solomon Four Group Design Group 1 Control 1 Group 2 Control 2 Before

n Solomon Four Group Design Group 1 Control 1 Group 2 Control 2 Before Treatment After no no yes yes yes no 12

Time Series Design n Repeats testing twice (or more) to establish a trend in

Time Series Design n Repeats testing twice (or more) to establish a trend in the data independent of the experiment Pretest Treatment Posttest Experiment yes Control Group yes no yes yes yes no REPEAT yes 13

EXAMPLES n Pretest/posttest design with control group Experiment Control Group pretest treatment posttest yes

EXAMPLES n Pretest/posttest design with control group Experiment Control Group pretest treatment posttest yes yes yes no Pretest/posttest design without control group Experiment A Experiment B pretest treatment posttest yes yes yes 14

Case Study n Basis of selection: representative, typical, cluster, probability, etc. n Multiple methods

Case Study n Basis of selection: representative, typical, cluster, probability, etc. n Multiple methods of data collection 15

Populations and Statistical Sampling 16

Populations and Statistical Sampling 16

Populations n Population- the entire group/ universe under study n Sample- a portion of

Populations n Population- the entire group/ universe under study n Sample- a portion of a population of possible information sources n Sampling- methods for selecting these sources 17

Research Design: Action Plan (continued) n Who is studied n population n Sample n

Research Design: Action Plan (continued) n Who is studied n population n Sample n Is sample reflective of population? n Where n Sampling? n When n Sampling? 18

Who (or What) is being studied? n Units of analysis: the what or whom

Who (or What) is being studied? n Units of analysis: the what or whom being studied. In social research the most typical units of analysis are individual people. n Can be: individuals, groups, organizations, social interactions, social artifacts n Examples: n Library Users or Non-Users n First-year Students n Senior Citizens n Public Libraries n Also n ILLs n Biographies, Mysteries, Audiobooks (i. e. collections) n Web sites 19

Beware: Pitfalls of Analysis n Ecological Fallacy: n Something learned about a group says

Beware: Pitfalls of Analysis n Ecological Fallacy: n Something learned about a group says something about the individuals making up that group. n Reductionism n An attempt to explain phenomena in terms of limited or lower-order concepts. 20

Who is being studied n How to select a sample? 21

Who is being studied n How to select a sample? 21

Sampling- Three Options n Census- collecting information from the entire group making up a

Sampling- Three Options n Census- collecting information from the entire group making up a population n Like the decennial census n Judgment sampling- making conscious choices n Convenience Sampling- what’s available n Probability/ Statistical Sampling- left to chance, each member of a population has an equal chance of being chosen 22

Sampling: Purpose n Representativeness n Sample has roughly the same distribution of characteristics as

Sampling: Purpose n Representativeness n Sample has roughly the same distribution of characteristics as the population from which it is drawn. n Nevertheless, each sample will differ from each other, as well as from the population n Can determine the amount of error 23

Probability Sampling (1) n Random sampling: Each member of the population has an equal

Probability Sampling (1) n Random sampling: Each member of the population has an equal and known probability of being selected n Systematic sampling: Each member of the population is either assembled or listed, a random start is designated and then members of the population are selected at equal intervals… nth intervals 24

Probability Sampling (2) n Stratified: Each member of the study population is assigned to

Probability Sampling (2) n Stratified: Each member of the study population is assigned to a group or stratum, then a simple random sample is selected from each group or stratum 25

Probability Sampling (3) n Cluster: Each member of the study population is assigned to

Probability Sampling (3) n Cluster: Each member of the study population is assigned to a geographically-defined group or cluster. Clusters are then selected at random, and members of a selected group are represented in the sample. § http: //www. claritas. com/My. Best. Segments/Default. jsp n Role of GIS and TIGER files § http: //www. census. gov/geo/www/tiger/index. html 26

Non-probability Sampling n Convenience: selecting cases based on their availability n “Typical” cases: selecting

Non-probability Sampling n Convenience: selecting cases based on their availability n “Typical” cases: selecting cases already known and not considered “extreme” n Snowball: group members identify additional members to be included in sample n Quota: sample is in same proportion as population 27

The Sample n How selected n Sample size n Determine the actual individuals or

The Sample n How selected n Sample size n Determine the actual individuals or “things” included 28

Sample Size n A larger sample does not necessarily mean better results, but n

Sample Size n A larger sample does not necessarily mean better results, but n Too small a sample can lead to error 29

Sample Size- 3 Considerations n Precision (sampling error)- the range in which the “true

Sample Size- 3 Considerations n Precision (sampling error)- the range in which the “true value” is estimated to be: ± 5 n Confidence Level (Central Limit Theorem)when a population is repeatedly sampled, the average value is = to the true value, and values in each survey will be normally distributed: 95% confidence level n Degree of Variability- distribution of attributes within a population. The more homogenous the population, the smaller the sample size. 30

Sample Size http: //edis. ifas. ufl. edu/PD 006#TABLE_1 31

Sample Size http: //edis. ifas. ufl. edu/PD 006#TABLE_1 31

Sampling Customers (Example) n Present n Lost n Never-gained n Nonuser 32

Sampling Customers (Example) n Present n Lost n Never-gained n Nonuser 32

Users/Uses of Electronic Resources (More Examples) n Home page users in general n Users

Users/Uses of Electronic Resources (More Examples) n Home page users in general n Users of a database 33

Questions of When n How might time effect our study? n How do we

Questions of When n How might time effect our study? n How do we choose a time frame? n What is an appropriate time frame based on the research problem? n Address time through: n Cross-sectional studies n Longitudinal studies 34

Cross-Sectional Studies n A study based on observations representing a single point in time.

Cross-Sectional Studies n A study based on observations representing a single point in time. n A “snapshot” n Best for exploratory and descriptive studies n U. S. Census n Explanatory cross-sectional studies aim at drawing causal relationships over time based on observations made at one time. n Issues? 35

Longitudinal Study n Permits observations of the same phenomena over an extended period. n

Longitudinal Study n Permits observations of the same phenomena over an extended period. n Researcher may “follow” a group over time n Researcher may become part of a group n Researcher may study artifacts developed over time 36

Types of Longitudinal Studies n Trend Studies n Type of longitudinal study that examines

Types of Longitudinal Studies n Trend Studies n Type of longitudinal study that examines changes within a population over time n Cohort Studies n Examines a specific subpopulation (cohort) as they change over time- often based on age. n Panel Studies n Examines the same set of people over time 37

The Learning Organization n http: //www. lib. umd. edu/groups/learning/learni ngorg. html 38

The Learning Organization n http: //www. lib. umd. edu/groups/learning/learni ngorg. html 38

Group Activity Selecting a Sample l l l l Archive/ Special Library Public Library

Group Activity Selecting a Sample l l l l Archive/ Special Library Public Library Academic Library/ School Library Identify a research question Define your population Describe how you would select a sample Could you design an experiment around this project? 39