AAC Intervention with Visual Scene Displays and JustinTime

AAC Intervention with Visual Scene Displays and Just-in-Time Technologies with Pre. Adolescents, and Adults with Developmental Disabilities Kathryn Drager, Christine Holyfield, Jessica Caron, Nimisha Muttiah

Adolescents and Adults • Limited research exists in the area of interventions to improve social interactions and communication with adolescents and adults who are beginning communicators and have severe developmental disabilities. • Many of these individuals may benefit from the use of augmentative and alternative communication (AAC) to meet their needs.

Adolescents and Adults • However, there is a lack of appropriate intervention strategies documented in the literature that address a range of communicative functions (e. g. , few studies examine AAC intervention for purposes other than simple requesting).

Early language acquisition • It may be useful to consider how typically developing children acquire language in developing interventions for this population

Early language acquisition • Early spoken language acquisition centers largely on the development of a receptive and expressive lexicon representing single concepts (e. g. , mom, bath, car)

Early language acquisition • This acquisition largely occurs within the context of events in which concepts occur • Language representing a concept is extracted from the environment (i. e. , from more sophisticated communicators’ input), used within that environment, and then subjected to feedback • “Dog!” • “You’re right! That’s a dog! A brown dog!”

Early language acquisition • Language learners, then, rely heavily on episodic memory to retrieve the concepts they have learned within previous contexts/events for use at a later date

Early Language / Early AAC Programming • When mapping early language development onto AAC intervention for early language learners (or beginning communicators), there has sometimes been a disconnect • AAC programming often happens away from the early learner and the communication context • Concepts (words) are often chosen and represented using adult-driven conceptualizations • Word representations often do not capitalize on contextual knowledge or episodic memory

Visual Scene Displays • Visual scene displays, (photographic or pictorial scenes of events with concepts embedded), address one aspect of disconnect: representation • They represent concepts (words) within the contexts in which they frequently occur; this may make them: • More recognizable • More accessible via episodic memory

Limitations • Unfortunately, there are two major limitations to current AAC technologies and apps: • It is time consuming to program new VSDs and vocabulary • As a result, partners do not add vocabulary frequently • It is not possible for partners to dynamically capture new experiences / vocabulary and add them to AAC technologies on the fly during interactions • As a result, it is difficult for partners to respond to students’ interests • It is difficult to capitalize on “teachable moments”

Just-in-Time Programming • One potential solution to this problem is the implementation of AAC technologies that support “just in time” (JIT) programming. • JIT programming • Allows the quick and easy import of photos as VSDs • Allows the quick and easy programming of vocabulary as hotspots within the VSDs • Allows partners to respond to the students’ interests by adding new communicative contexts and vocabulary “on the fly” during daily interactions

What is the potential of JIT programming? • Simplifying the complexity of AAC programming is one way to lesson the time and attention required for programming to allow for it to occur JIT • BUT, can we make it SO straight-forward and lessen demands SO much that it is accessible to all people, including people with beginning linguistic, motor, and cognitive profiles?

Steps in Programming Select Photo Icon Take a Photo Select Hotspot Icon Create Hotspot Select Record Icon Record Output Optional: Select Draw Icon Optional: Draw on VSD

Study 1: Typically developing toddlers • A descriptive study involving 10 toddlers who were typically developing ranging in age from 10 months to 22 months (mean=16 months) • Described the toddlers’ successful participation in programming storybook pages into Easy. VSD during a shared storybook interaction, defined by their completion of a step in the programming process with or without visual/verbal prompting or modeling from the investigator

The Toddlers • All toddlers demonstrated an isolated point • All toddlers coordinated their gaze with their movement • All toddlers demonstrated evidence of joint attention skills • All toddlers demonstrated use of symbolic expressive communication • 8 of the 10 toddlers used spoken words

Study 1: Results • All 10 toddlers were able to participate in programming in some capacity, with overall rates of successful participation at 41%, out of an average of 80 opportunities • However, there was a great deal of variation in participation success across the toddlers who participated and the steps of the programming process

Study 1: Results, Continued Select Take Photo Icon Photo Select Hotspot Icon Create Hotspot Select Record Icon Record Message Select Draw Icon Draw on Photo 43% 29% 27% 20% 3% 41% 82% 79%

Study 1: Results, Continued

Implications • Although the study was not completed by individuals who are beginning communicators who require AAC, it suggests that programming can be simplified to allow more people to participate in the process • However, some aspects of programming may be more demanding than others

Study 2: Pre-adolescents and Adolescents • 5 adolescents, ages 9 -18 • All adolescents were early language learners, or beginning communicators (i. e. , they used less than 50 concepts expressively) • All of the adolescents had intellectual and developmental disabilities such as autism spectrum disorder, cerebral palsy, and fetal alcohol syndrome; 4 of the 5 adolescents had multiple disabilities

Study 3: Adults • 4 adults, ages 28 -56 • All adults were early language learners, or beginning communicators (i. e. , they used less than 50 concepts expressively) • All of the adults had intellectual and developmental disabilities such as autism spectrum disorder and cerebral palsy

Study Design • The studies both used a multiple baseline, across participants design to address the following question: • What is the effect of a communication application on mobile technology featuring visual scene displays and just-in-time programming on the number of communication turns taken by pre-adolescents and adolescents/adults who are beginning communicators during 15 -minute interactions?

Implications • Just in time programming of VSDs benefits not only young kids who are learning language, but also pre-adolescents, and adults who are also early language learners

Implications • Although they all have limited expressive vocabularies and other limitations (e. g. , motor, working memory), young kids and older beginning communicators are very different populations • These studies suggests that JIT programming benefits early language learners across ages

Just-in-Time • The following elements were part of programming “just-in-time”: • taking a picture • drawing a hotspot • recording a message • Some of the participants were able to participate in these programming elements (most often taking a picture).

Study 2: Pre-adolescents and Adolescents Completion of Programming Steps by Adolescents with JIT Technology Independent Programming Completed in Intervention Session 1 Session 2 Session 3 Session 4 Session 5 Adolescent Carver 5 4 0 3 0 Jason 2 0 2 1 0 Nathan 0 1 1 0 1 Amanda 0 0 0 Jose 0 0 0

Participation in Programming • An increase in participation in programming has important implications relative to supporting selfdetermination in beginning communicators and providing beginning communicators with control over the vocabulary available to them – an option that is rarely available.

Just-in-Time Technology • The introduction of the device with JIT technology allowed for partner responsiveness to the students’ interests through the quick and easy import of photos as VSDs and quick and easy programming of vocabulary as hotspots, resulting in an immediate and sustaining jump in communication turns for all participants.

Final summary • Consider the demands AAC technology imparts relative to: • Communication • Programming

Final Summary • Recognize that less demanding options are available for beginning communicators across ages that may: • Allow for JIT (i. e. , in the moment) programming of concepts • Allow AAC concepts to be made available in response to interest demonstrated by language learners (paralleling typical language development) • Allow beginning communicators to participate in some capacity in the programming process, regardless of motor, language, or cognitive profiles

Funded through Grant #1 R 43 HD 059231 -01 A 1 (PI: Jackobs), SBIR Phase II: "Play. Talk" Communication Software for Children with Complex Communication Needs, National Institute of Child Health and Human Development, National Institutes of Health