Research Methods An Overview of Survey Techniques Sampling

Research Methods An Overview of Survey Techniques, Sampling Strategies, Modes of Observation, Interviewing , Focus Groups 1

Lecture outline: Overview of material covered in chapters 6, 8 -12 in text Modules created for each chapter in text Ch 6: Measuring Crime Ch. 8: Sampling Ch. 9: Survey Research Ch. 10: Qualitative Interviewing Ch. 11 Field Observation Ch 12: Agency Records, Content Analysis, and secondary Data 2

Module 1: Surveys( see Chapter 6, 9 in text) 1. Survey data collection methods: A look at UCR, Self Report , and Victimization Surveys 2. The Basics of Survey Design 3. Types of Surveys 3

Topics appropriate to survey research �Counting crime – asking people about victimization counters problems of data collected by police �Self-reports – dominant method for studying the etiology of crime �Frequency/type of crimes committed �Prevalence (how many people commit crimes) committed by a broader population 4

Topics appropriate to survey research �Perceptions and attitudes – to learn how people feel about crime and CJ policy( e. g. Death Penalty) �Policy proposals – search for ways to respond to crime that are supported by the general public( e. g Downsizing Prisons) �Targeted victim surveys – used to evaluate policy innovations & program success( e. g. domestic violence) �Other evaluation uses – e. g. , measuring community attitudes, citizen responses, etc. ( e. g. criminal thinking scales) 5

The Basic Structure of a Survey Instrument �Introduction: state the general purpose of survey and emphasize voluntary nature of survey, confidentiality and protection of human subjects. �Section 1: Factual non-threatening questions( demographics here) �Section 2: Core questions( ordering questions; contingency questions) �Section 3: Scales( Likert, Gutmann) �Example of Matrix questions--Likert Scale: TCU Criminal Thinking Scale used by Taxman, et. al. in recent study of criminal thinking( class handout, Nov. 26 th) 6

Types of questions: Open- and close-ended questions Open-ended – respondent is asked to provide his or her own answer � What were your initial expectations of the program? ________________________ Closed-ended – respondent selects an answer from a list or scale � On a scale of 1 to 5, where 1 is low and 5 is high, rate your initial expectations of the program. 1 2 Low 3 4 5 High 7

Open versus closed? �Choice between these question types will depend on the aim of your research, the study design, the population being study, etc. �Advantages and disadvantages to both. �Possible gains in reliability (closed-ended) balanced with possible losses in breadth of information and validity. �One strategy is to combine open- and close-ended questions. 8

Designing questions… �Make items clear – avoid ambiguous questions; do not ask “double-barreled” questions �Short items are best – respondents like to read answer a question quickly �Avoid negative items – leads to misinterpretation �Avoid biased items and terms – do not ask questions that encourage a certain answer �Closed question response categories must be exhaustive and mutually exclusive. 9

Survey data collection methods 1. Self-administered mail surveys, computer based surveys, and response rates 2. In-person interview: best for complex questions ( role of interviewer is key) 3. Telephone interviews; fast, low cost; CATI- computer assisted telephone interviewing 4. A comparison of three methods Focus: Racial Profiling Research( see text pp 256, 257) 10

Focus: Australia’s National Crime Victimization Survey Methodology � The MPHS was conducted as a supplement to the monthly LFS. � Each month one eighth of the dwellings in the LFS sample were rotated out of the survey. � In 2012 -13, all of these dwellings were selected to respond to the MPHS each month �. In these dwellings, after the LFS had been fully completed for each person in scope and coverage, a person aged 15 years and over was selected at random (based on a computer algorithm) and asked the various MPHS topic questions in a personal interview. � If the randomly selected person was aged 15– 17 years, permission was sought from a parent or guardian before conducting the interview. If permission was not given, the parent or guardian was asked the crime questions on behalf of the 15– 17 year old. � Questions relating to sexual assault, alcohol or substances contributing to the most recent physical or face-to-face threatened assault were not asked of proxy respondents. Only those persons aged 18 years and over were asked questions on sexual assault. � Data was collected using Computer Assisted Interviewing, whereby responses were recorded directly onto an electronic questionnaire in a notebook computer, usually during a telephone interview. 11

A Link to the USA National Crime Victimization Survey �http: //www. bjs. gov/index. cfm? ty=dcdetail&iid=245 12

In Review: Comparing UCR and NCVS—See Text Chapter �UCR data are based on reported criminal acts (offender characteristics) �NCVS data based on individuals actually victimized (characteristics of victims) �Whenever possible, review both data sources and compare findings

Assessment of NCVS �Document a massive amount of crime that goes unreported �Underestimate crime rate �Insignificant crimes tend to be forgotten �Victims of several crimes may also forget about all the crimes �Females do not report victimization if her abuser live in the same household �Whites and college graduates are more likely to report being victimized

Assessment of NCVS �NCVS respondents are interviewed every six months (7 interviews) �Reported victimization rates usually decease with each interview (awareness of victimization) �Overestimation of some crimes �Respondents might mistakenly interpret some noncriminal events as crimes �“Telescoping “ effect

Self-reports data �Created to complement UCR and NCVS �Interviews or questionnaires �Demonstrate the prevalence of offending (the proportion of respondents who have committed a particular offence) �Incidence of offending (the average number of offences person in the study)

Samples for self-reports �Target: Adult inmates of jails and prisons: PREA findings on sexual assault �Target: Adolescents, usually high school students �The most important finding: delinquency is very common �Middle-class youth commit as much crime as working-class youth �Testing criminological theories: Travis Hirschi’s Research Study testing Control Theory( compared to subculural & Strain)

Assessment of self-report studies �Focus on minor and trivial offenses (truancy, running away from home, minor drug and alcohol use) �Although recent studies (NYS) asked subjects about rape and robbery �Respondents might not to tell the truth (reliability issues)

If respondents lie…. �Self-report data can be checked against police records, school records, interviews with teachers and parents �The use of, or threat of , polygraph validation (20% change their initial responses when threatened with a “lie detector”) �Subsequent interviewing of subjects permits probing regarding the details and context of acts �Use of “lie scales”

Focus: Self Reported Substance Use Among Detainees in Australia �http: //www. aic. gov. au/media_library/conferences/eva luation/mcgregor. pdf �http: //192. 190. 66. 70/documents/7/E/8/%7 B 7 E 8 D 4 A 8 E -A 5 AF-4 D 3 B-8821 -ED 8 A 1 BA 489 B 6%7 Drpp 93. pdf 21

Focus: Research on Extent of Substance Use by High school students in USA �Question: Can we collect reliable estimates from survey questionnaires distributed to high school students across the country? �The Survey: Lets take a close look: �http: //www. casacolumbia. org/addictionresearch/reports/national-survey-american-attitudessubstance-abuse-teens-2012 �http: //www. drugabuse. gov/publications/drugfacts/hi gh-school-youth-trends 22

UCR, NCVS, and self-reports �None of the three is perfect �For the best estimates of the actual number of crimes, NCVS data are preferable � For the best estimates of offender characteristics, selfreports and NCVS are preferable �UCR are superior for understanding the geographical distribution of crime

Self-administered questionnaires �Can be home-delivered �Researcher delivers questionnaire to home of sample respondent, explains the study, and then comes back later �Mailed (sent and returned) survey is most common �Researchers must reduce the trouble it takes to return a questionnaire �Goal is High Response Rate 24

25 Computer-based self-administration § Via email, website § Issues: » representativeness » mixed in with, or mistaken for, spam » requires access to Web » sampling frame?

Module 2: More Survey Research What we will Cover: 1. Interviewing techniques and Focus Groups 2. A Comparison of Survey Methods: phone/ computer( survey monkey), mail, and in-person 3. Field Research: The good, the bad, and the ugly 4. Agency Records, Content Analysis and Secondary Data 26

27 In-person interview survey: See text, ch 9 § Typically achieve higher response rates than mail surveys (80 -85% is considered good) § Demeanor and appearance of interviewer should be appropriate; interviewer should be familiar with questionnaire and ask questions precisely § When more than one interviewer administers, efforts must be coordinated and controlled § Practice interviewing

28 Specialized interviewing • Two variations: » General interview guide – less structured, lists issues, topics, questions you wish to cover; no standardized order » Standardized open-ended interview – more structured, specific questions in specific order; useful in case studies, retrieves rich detail in responses

29 Telephone surveys § 94% of all households now have telephones § Random-Digit Dialing » Obviates unlisted number problem » Often results in business, pay phones, fax lines § Saves money and time, provides safety to interviewers, more convenient § may be interpreted as bogus sales calls; ease of hang-up

30 Comparison of the three methods § Self-administered questionnaires are generally cheaper, better for sensitive issues than interview surveys § Using mail – local and national surveys are same cost § In person or phone Interviews – more appropriate when respondent literacy may be a problem, produce fewer incompletes, achieve higher completion rates § Validity low in survey research; reliability high § Surveys are also inflexible, superficial in coverage

31 Tips on self-report items § Convince subjects you will guarantee confidentiality and anonymity § Minimize possible social undesirability you are asking respondents to admit § Phrase questions in non-judgmental manner § Bear in mind “fading memory” when setting time frame

32 Focus groups: See Putting it all together, pp 256, 257 in text § 12 -15 people brought together to engage in guided group discussion of some topic( e. g. addicts& recovery) § Members are selected to represent a target population, but cannot make statistical estimates about population § Most useful when precise generalization to larger group is not necessary § May be used to guide interpretation of questionnaires following survey administration § Examples: Drug addicts and reentry, homeless sex offenders, teenagers and sexting

Getting Out Module 3: Field Research 33

Field Research: See Text Chapter 11 • Field research encompasses two different methods of obtaining data: • Direct observation( example-bar room behavior; seat belt use, community disorder indices) • Asking questions; modes of recording observations • May yield qualitative and quantitative data • Often no precisely defined hypotheses to be tested

Topics Appropriate to Field Research • Gives comprehensive perspective – enhances validity • Go directly to phenomenon, observe it as completely as possible • Especially appropriate for topics best understood in their natural setting • How street-level drug dealers distinguish customers • Studies of ‘vice’, e. g. , prostitution and drug-use. • Aspects of physical settings, Disney World, social control

Various Roles of the Observer (Gold, 1969) • Complete participant – participates fully; true identity and purpose are not known to subjects • E. g. , posing as a bar patron; becoming a police officer; or corrections worker • Participant-as-observer – make known your position as researcher and participate with the group • E. g. , study of active drug users; Julie Mueler and the Guardian Angels • Observer-as-participant – make known your position as a researcher; do not actually participate • E. g. , Observational study of police patrol—”ride-along” ; research on gangs • Complete observer – observes without becoming a participant • E. g. , court observation, Chicago neighbourhood study( windshield studies)

Observer status �Be aware of, and document role of researcher (extent of participation) �Be aware that all observation is subjective. �Be aware of the possible “effect of participation”. �Be aware of often competing ethical and ‘scientific’ values related to all observational studies. 37

Asking Questions • Field research is often a matter of going where the action is and simply watching and listening • Also a matter of asking questions & recording answers • Field research interviews are much less structured than survey interviews • Ideally set up and conducted just like a normal, casual conversation

Preparing for the Field • Access to formal organizations • Find a sponsor, write a letter to executive director, arrange a phone call, arrange a meeting • Access to subcultures • Find an informant (e. g. , person who works with offenders), use that person as your “in” • Snowball sampling is useful as informant identifies others, who identify others, etc.

Module 4: Sampling

An Overview of Sampling Techniques: See Ch. 8 in text �Sampling Distributions �The Normal Curve �Sampling Error �Probability Sampling �Non-Probability Sampling �Illustrations: NCVS, British Crime Survey, Australian Crime Survey 41

Key Terms to Remember: Sampling �Simple random sampling �Table of random numbers �Confidence intervals �Stratified sampling �Population �Probability vs. non-probability sampling �Equal probability of selection method( EPSEM) �Focus: random sampling from probation case files 42

Module 6: Recording Observations in the Field • Note taking, tape recording when interviewing and when making observations • Videotaping or photographs can make records of “before” and “after” some physical design change • Field notes – observations are recorded as written notes, often in a field journal; first take sketchy notes and then rewrite your notes in detail • Structured observations – observers mark closed-ended forms, which produce numeric measures

Linking Field Observations and Other Data • Useful to combine field research with surveys or data from official records • Baltimore study of the effects of neighborhood physical characteristics on residents’ perceptions of crime problems (Taylor, Shumaker, & Gottfredson, 1985) • Perceptions – surveys; • Physical problems – (1) observations, (2) actual population and crime information - census data & crime reports from police records

Strengths and Weaknesses of Field Research • Provides great depth of understanding • Flexibility (no need to prepare much in advance) • More appropriate to measure behavior than surveys • High validity; quant. measures – incomplete picture • Low reliability – often very personal • Generalizability – personal nature may produce findings that may not be replicated by another • Precise probability samples can’t normally be drawn

Module 7: Secondary Data, Content Analysis Other Sources to Consider 46

Secondary Data: See text, ch. 12 • Data from agency records – agencies collect a vast amount of crime and CJ data • Secondary analysis – analyzing data previously collected • Content analysis – researchers examine a class of social artifacts (typically written documents) See Text example; p 329 Terrorist Recruitment

Topics Appropriate for Agency Records • Most commonly used in descriptive or exploratory studies • Content analysis often center on links between communication, perceptions of crime problems, individual behavior, CJ policy

Types of Agency Records • Published Statistics – gov’t organizations routinely collect and publish compilations of data (e. g. , ABS, BOCSAR, AIC) often available in libraries and online • Nonpublic Agency Records – agencies produce data not routinely released (e. g. , police departments, courthouses, correctional facilities) • New Data Collected by Agency Staff – collected for specific research purposes; less costly + more control

Units of Analysis in Criminal Justice Data Criminal Activity • Incidents • Crimes violated • Victims • Offenders Court Activity • Defendants • Filings • Charges and Counts • Cases • Appearances • Dispositions • Sentences Apprehension • Arrests • Offenders • Charges • Counts Corrections • Offenders • Admissions • Returns • Discharges

Sources of Reliability and Validity Problems • Virtually all CJ record keeping is a social process – “social production of data” • Records reflect decisions made by CJ personnel as well as actual behavior by juveniles and adults • Discretion factors in to recordkeeping • CJ organizations are more interested in keeping track of individual cases than in examining patterns • Potential for clerical errors due to volume of data

Content Analysis • Systematic study of messages – can be applied to virtually any form of communication • Decide on operational definitions of key variables • Decide what to watch, read, listen to & time frame • Analyze collected data • Well suited to answer “who says what, to whom, why, how, and with what effect? ”

Aspects of Sampling and Coding in Content Analysis 2 • Reminders: • Remember operational definition of variables, and their mutually exclusive & exhaustive attributes • Pretest coding scheme • Assess coding reliability via intercoder reliability method and test-retest method

Secondary Analysis • Sources – websites, libraries • AIC, http: //www. aic. gov. au/ • BOCSAR, http: //www. bocsar. nsw. gov. au/ • BJS, NCVS, ICPSR, NACJD • Advantages – cheaper, faster, benefit from work of skilled researchers • Disadvantages – data may not be appropriate to your research question; least useful for evaluation studies (which are designed to answer specific questions about specific programs)

Review: Key Terms in Chapters 6 -12 in text