Other Issues When Planning and Evaluating Behavior For

Other Issues When Planning and Evaluating Behavior

For ABA Importance of Individual Subjects Enables applied behavior analysts to discover and refine effective interventions for socially significant behaviors Contrasted with groups-comparison approach Lots of variability within the group

Absence of Intrasubject Replication In group designs, power of replicating effects with individuals is lost Many applied situations in which overall performance of group is socially significant When group results don’t represent individuals, should supplement the data with individual results Need to determine on the front end of the study

Design Issues KEY: An effective researcher must actively design each experiment so answers the question being asked. May want a between groups design.

Good Experimental Designs Often designs entail a combination of analytic tactics Many possible designs with different combinations can be effective Most effective use ongoing evaluation of data from individuals to employ baseline logic of prediction, verification, and replication

More Design Issues Maturation: changes in subject over course of experiment Repeated measurement controls and detects uncontrolled variables

Measurement Confounds Observer drift or bias Is a big problem Solutions Keeping observers naïve to expected outcomes can reduce observer bias Maintain baseline conditions long enough to reduce reactance Could use intermittent probes except when practice effects would be expected

Subject’s Expectations Can be an IRB issue with informed consent May not be as relevant in a treatment setting Can use a placebo control Separates effects produced by subject’s perceived expectations Can use a double-blind control Eliminates confounding by subject, teacher and parent expectations, differential treatment by others, and observer bias

Treatment Integrity Extent to which the independent variable is implemented or carried out as planned Treatment drift: when application of independent variable in later phases differs from original application

Solution: Precise Operational Definitions Keeps you on task Defined in 4 dimensions: Verbal Physical Spatial Temporal

More Solutions Simplify, Standardize, and Automate Simple, precise treatments are more likely to be consistently delivered Simple, easy-to-implement techniques are more likely to be used and socially validated Standardize as many aspects as possible If possible, can use an automated devices

Training and Practice Is critical Must provide practice for individual who will conduct the experimental sessions Can be detailed scripts, verbal instructions, modeling, or performance feedback Once done, practice it before using on the client.

Data Collection Measure how the actual implementation of the conditions matches the written methods Gives the researcher the ability to use retraining and practice to ensure high treatment integrity Reduce, eliminate, or identify the influence of as many potentially confounding variables as possible

Social Validity Is an important concept What is the social significance of the target behavior appropriateness of the procedures social importance of the results Usually assessed by asking direct and indirect consumers Consumer satisfaction

How to Determine Assess the performance of persons considered competent Experimentally manipulate different levels of performance Allows you to determine which produces optimal results

Social Importance of Interventions Can use rating scales and questionnaires for obtaining consumers’ opinions on acceptability of interventions Examples: Treatment Acceptability Rating Form

Methods for Assessing Outcomes: Many Methods Compare subject’s performance to a normative sample Use standardized assessment instruments Ask consumers to rate social validity of performance Ask experts to evaluate subject’s performance Test subject’s new performance in natural environments

Normative Sample Not limited to posttreatment comparisons Compare subject’s behavior to ongoing probes of behavior of normative sample to provide ongoing measure of improvement and how much is still needed

Consumers and Experts Most frequently used method for assessing social validity is to ask consumers Experts can be called upon to judge the behavior changes

Standardized and Real-World Tests Example of standardized test: Self-Injury Trauma Scale (SITS) Real-world test in the natural environment provides direct assessment of social validity Also exposes subject to naturally occurring reinforcement May promote maintenance and generalization

Direct Replication Duplicates exactly the conditions of an earlier experiment Two ways Intrasubject direct replication: Uses same subject to establish reliability of functional relation Intersubject direct replication: Uses different but similar subjects to determine generality

Systematic Replication Researcher purposefully varies one or more aspects of earlier experiment Can demonstrate reliability and external validity of earlier findings Allows you to alter any aspect Subjects Setting Administration of independent variable Target behaviors

Evaluating the Data Define and measure the dependent variable Use graphic displays What is the meaningfulness of baseline conditions compared to the treatment conditions Visual analysis and interpretation can give you a lot of creditability

Visual Analysis and Interpretation Factors that favor visual analysis over tests of statistical significance in ABA: Want to see socially significant behavior change, not statistically significant Good for identifying variables that produce strong, large, and reliable effects Accepting statistical analysis as evidence of functional relation may cause researcher not to experiment further Tests of statistical significance may cause data sets to conform, losing flexibility in design

Errors Type I error: when researcher concludes that independent variable had effect on dependent variable, when it did not Type II error: when researcher concludes that independent variable did not have effect on dependent variable, when it did Visual analysis leads to less Type I and more Type II errors Statistical analysis leads to more Type I and less Type II errors

Best Use both when possible Makes for a stronger argument.

Final Issues Independent variable should be assessed in terms of its effects on dependent variable, May need to address social acceptability, complexity, practicality MUST BE COST EFFECTIVE Cannot be underestimated May need to do a cost/benefit analysis When costs are to high, may want an alternative May need to consider maintenance and generalization of behavior outside experimental control