MEASURING IMPLICIT AND EXPLICIT KNOWLEDGE OF A SECOND

MEASURING IMPLICIT AND EXPLICIT KNOWLEDGE OF A SECOND LANGUAGE A PSYCHOMETRIC STUDY Rod Ellis, University of Auckland, SSLA, 27, 141– 172+

PROBLEM • Lack of valid measures of second language implicit and explicit knowledge • We can elicit performance, but how to elicit and measure true knowledge? This study attempts both to establish definitions of implicit knowledge and explicit knowledge, and to design and apply a battery of tests that will provide measures of them.

GOALS OF L 2 RESEARCH • Define and describe L 2 linguistic knowledge • Explain how this knowledge develops over time by specifying the internal and external variables involved

WHAT IS LINGUISTIC KNOWLEDGE? Two competing positions: • Chomsky (1976) posited that linguistic competence consists of a biological capacity for acquiring language (Universal Grammar). • Largely restricted to grammar • “Mentalist in orientation” – environment alone not capable of accounting for language learning • Cognitive psychologists, such as Rumelhart and Mc. Clelland (1986) say that language learning is not cognitively different from other forms of learning. • Linguistic knowledge emerges gradually • Learners acquire new sentences and restructure old sequences until underlying rules makes themselves apparent

WHAT IS LINGUISTIC KNOWLEDGE? CONT’D These two theories are in opposition. They agree only on one thing: Linguistic knowledge is implicit. We always know “that” without knowing “how. ”

NONINTERFACE POSITION How do implicit and explicit knowledge interface? Not as much consensus here. Noninterface position: • Implicit and explicit L 2 knowledge involve different acquisitional mechanisms • Stored in different parts of the brain • Accessed by different processes

STRONG INTERFACE POSITION Explicit knowledge can be derived from implicit knowledge. Explicit knowledge can also be converted into implicit knowledge through practice. Researchers disagree on a specific type of practice that is necessary (communicative versus mechanical) to convert explicit to implicit knowledge.

WEAK INTERFACE POSITION Adopts some ideas from both perspectives. Exists in three variations. These disagree in most regards, but agree on the fact that explicit knowledge through some mechanism can be converted to implicit knowledge. • If the learner is developmentally ready • The conversion is aided by the fact that learning explicitly triggers some cognitive processes needed for implicit knowledge

PAST STUDIES While theories of linguistic knowledge abound, they have not been tested empirically. Studies explored the relationship between explicit and implicit knowledge: • Green and Hecht (1992): German ESL learners could correct grammatical errors 78% of the time but could only name the applicable rule 46% of the time. • Macrory and Stone (2000): British learners’ knowledge of the French perfect tense (selfreport, on a test, and in free production). High scores on the first two, but in free production was rarely used correctly. • Hu (2002): Chinese ESL learners were given two free writing tasks, interspersed by error correction and rule verbalization. In the second task, several grammatical structures were used more accurately.

CONCLUSIONS FROM PAST STUDIES • Green and Hecht (1992): The learners’ explicit knowledge is only a subset of their greater implicit knowledge. • Macrory and Stone (2000): “language-as-knowledge” and “language-for-use” might derive from different sources. • Hu (2002): (inconclusive) There is a relationship between metalinguistic knowledge and writing production.

FLAWS IN THESE STUDIES They are primarily correlational in nature Demonstrating a relationship does not equate to establishing that implicit knowledge follows from explicit knowledge

ADDRESSING THESE FLAWS De. Keyser (1995): Performed a direct test of the interface position • Tested effects of explicit-deductive and implicit-inductive instruction on two constructs in an artificial grammar • Learners were then asked to complete a computerized judgment test (does this sentence match this picture? ) and a computerized production test (write a sentence to describe this picture. ) • Suggests that production is facilitated when learners are taught explicit knowledge and then practice it.

CURRENT STATE OF RESEARCH • Even De. Keyser does not discuss the construction and validity of his method. • In SLA research, this is widespread. • What is needed: a scientific, empirical method to test causation. • What this study will give us: hopefully the same

EXPANDING ON IMPLICIT VS. EXPLICIT KNOWLEDGE Characteristics Implicit Knowledge Explicit Knowledge Awareness Intuitive awareness of linguistic norms Conscious awareness of linguistic norms Type of knowledge Procedural knowledge of rules and fragments Declarative knowledge of grammatical rules and fragments Systematicity Variable but systematic knowledge Anomalous and inconsistent knowledge Accessibility Access to knowledge by means of automatic processing controlled processing Use of L 2 knowledge Access to knowledge during fluent Access to knowledge during performance planning difficulty Self-report Nonverbalizable Verbalizable Learnability Potentially only within critical period Any age

CRITERIA TO DESIGN ACCURATE MEASUREMENTS (ELLIS, 2004) Criterion Method Degree of awareness Ask learners to report retrospectively about whether they made use of feel or rule in responding to a task Time available Distinguish tasks that make significant demands on learners’ short-term memories and those that lie comfortably within their L 2 processing capacity Focus of attention Does the task prioritize fluency or accuracy? Systematicity Examine whether learners are consistent or variable in their response to a task (consistent = implicit; variable = explicit) Certainty How certain are learners that the linguistic forms they have produced conform to target language norms? (Treat with caution) Metalanguage Knowledge of metalingual terms will be related to their explicit (analyzed), not implicit, knowledge. Learnability Early L 2 learners more likely to display high implicit knowledge; those who began as adolescents or adults more likely to display high levels of explicit knowledge.

WITHOUT FURTHER ADO, THE ACTUAL STUDY!

BACKGROUND Continuing from a 1998 study which analyzed scores from a battery of tests (oral production, timed and untimed grammaticality judgment tests, metalinguistic comments). Limitation: Focused on only one grammatical structure (verb complementarity) despite obtaining statistically significant results Current study builds on this by adding grammatical structures and measures of explicit vs. implicit knowledge.

PURPOSE To develop a battery of tests that would provide relatively separate methods of implicit and explicit knowledge. CAVEAT: Entirely separating these types of knowledge is impossible, because learners will always draw upon the resources they have available regardless of their suitability to the task.

RESEARCH QUESTION To what extent is it possible to develop tests that provide separate measures of implicit and explicit L 2 knowledge?

HYPOTHESES 1. Tests of Explicit Knowledge Will Encourage the Use of Rule, Whereas Tests of Implicit Knowledge Will Favor Feel. 2. Time-Pressured Tests Will Require Learners to Rely on Their Implicit Knowledge, Whereas Tests Without Time Constraints Will Permit Learners to Draw on Their Explicit as Well as Their Implicit Knowledge. 3. Tests That Require Learners to Focus on Meaning Will Elicit Implicit Knowledge, Whereas Tests That Encourage Learners to Focus on Form Will Elicit Explicit Knowledge 4. Tests of Implicit Knowledge Will Elicit More Systematic (Less Variable) Responses Than Tests of Explicit Knowledge. 5. Tests of Implicit Knowledge Will Elicit More Certain Responses from Learners Than Tests of Explicit Knowledge 6. Tests of Explicit Knowledge Will Make Fuller Use of Metalinguistic Knowledge Than Tests of Implicit Knowledge 7. Scores on Tests of Implicit Knowledge Will Relate More Strongly to the Age Learners Started Learning the L 2 Than to Years of Classroom Instruction, Whereas the Opposite Will Be the Case for Scores on Tests of Explicit Knowledge

PARTICIPANTS • 111 participants • 20 native speakers • 91 learners of L 2 English • NS breakdown: • 13 male, 7 female • 15 previously studied a foreign language, 11 studied 2+ languages • L 2 breakdown: • 36 male, 58 female • Mixed proficiency • 21 in low-level English courses; 30 in higher-level courses, 44 had been tested on the IELTS with a mean of 6. 24/9. 00 • Average of 10 years’ English study • Average of 1. 9 years living in an English-speaking country

TESTS • 17 English grammatical structures • Grammatical structures known to be problematic to learners • Selected for both early and late acquired grammatical features • Selected for when features are usually introduced in ESL classes (early, intermediate, late) • Both morphological and syntactic features

GRAMMATICAL STRUCTURES

TEST BATTERY Imitation Test Subjects presented with one grammatical and one ungrammatical sentence out loud. They indicate whether each sentence is grammatical, and then repeat the grammatical form out loud Oral Narrative Test Subjects read a story twice and were asked to repeat it orally within three minutes Timed GJT Computerized, 4 sentences for each of 17 grammatical structures. Time allowed ranged from 1. 8 to 6. 24 seconds Untimed GJT Same content as timed GJT, adding self-reporting re: certainty and whether they used “rule” or “feel” Metalinguistic Knowledge Test Untimed, computerized: subjects are presented with 17 ungrammatical sentences and must select the rule that they are flouting. Second task: read a short text and find examples of 21 grammatical features therein

DESIGN FEATURES OF THE TESTS Designed to Measure: implicit explicit

PROCEDURE • Tests were completed in this order: imitation test, oral narrative test, timed GJT, untimed GJT, and metalinguistic knowledge test • Imitation test conducted in 1 -on-1 meetings with researchers • Other four tests conducted with participant alone with a computer • All tests completed in one 2. 5 -hour session per subject • Non-native speakers were given a background questionnaire that asked their L 1, age at beginning to learn English, number of years in an English-speaking country, other languages they studied, and type of instruction they have received

ANALYSIS • Descriptive statistics developed for each test • Reliability of these calculated using Cronbach’s alpha (statistical measure of internal consistency) • Pearson product moment coefficients • Computed to examine the interrelationships between the various test measures • Measured by r • Principal component factor analysis using SPSS Version 11. 5 • Investigate the predictions about the type of knowledge each test measures

TEST RELIABILITY

RESULTS Interesting findings: NSs achieved scores close to 100% on all measures except metalinguistic knowledge and sentences in timed GJT ungrammatical L 2 learners scored highest on untimed GJT measures Both NSs and L 2 learners scored much higher on grammatical sentences in timed GJT (than ungrammatical) L 2 learners’ results were much more variable than NSs’ results

VISUALIZATION OF RESULTS

CORRELATION BETWEEN TESTS (FOR L 2 LEARNERS)

PRINCIPLE COMPONENT FACTOR ANALYSIS Explaining Eigenvalues and Eigenvectors

RESULTS: HYPOTHESIS 1 Tests of Explicit Knowledge Will Encourage the Use of Rule, Whereas Tests of Implicit Knowledge Will Favor Feel Pearson product moment coefficients of correlation were computed between the measure of the learners’ application of rule in the untimed GJT and all the other measures. Prediction: rule will correlate more strongly with accuracy of judgment in the untimed (ungrammatical) GJT and also with scores on the metalinguistic judgment test. Hypothesis 1: Supported!

RESULTS: HYPOTHESIS 2 Time-Pressured Tests Will Require Learners to Rely on Their Implicit Knowledge, Whereas Tests Without Time Constraints Will Permit Learners to Draw on Their Explicit as Well as Their Implicit Knowledge PREDICTION: A difference in mean scores on the two groups of tests (unpressured vs. pressured) can be expected. Mean score for pressured tests: 57. 3% Mean score for all tests: 65. 9% Hypothesis 2: Supported! P-value: <. 01 Greatest difference can be seen between the timed and untimed GJTs: p-value drops to <. 001.

RESULTS: HYPOTHESIS 3 Tests That Require Learners to Focus on Meaning Will Elicit Implicit Knowledge, Whereas Tests That Encourage Learners to Focus on Form Will Elicit Explicit Knowledge Tests that require a focus on meaning: Oral narrative Imitation Hypothesis 3: Untestable Focus and time pressure variables were confounded in the design of the tests

RESULTS: HYPOTHESIS 4 Tests of Implicit Knowledge Will Elicit More Systematic (Less Variable) Responses Than Tests of Explicit Knowledge PREDICTION: Tests of implicit knowledge will result in lower standard deviations than the tests of explicit knowledge. Standard deviations were in fact higher in tests of explicit knowledge, HOWEVER, contrary to expectations, standard deviations for the timed GJT were higher than in the untimed GJT. Hypothesis 4: Unsupported

RESULTS: HYPOTHESIS 5 Tests of Implicit Knowledge Will Elicit More Certain Responses from Learners Than Tests of Explicit Knowledge Comparing certainty judgments for grammatical sentences (implicit) and ungrammatical sentences (explicit): r =. 32 (grammatical) Hypothesis 5: Unsupported r =. 31 (ungrammatical) Both correlations were statistically significant, and therefore this hypothesis is unsupported.

RESULTS: HYPOTHESIS 6 Tests of Explicit Knowledge Will Make Fuller Use of Metalinguistic Knowledge Than Tests of Implicit Knowledge Scores on the metalinguistic knowledge test correlated to: • untimed GJT (grammatical and ungrammatical items), r =. 60 • untimed GJT (ungrammatical items), r =. 64 • imitation test, r =. 28 • oral narrative test, r =. 27 • timed GJT, r =. 24 Hypothesis 6: Weakly supported

RESULTS: HYPOTHESIS 7 Scores on Tests of Implicit Knowledge Will Relate More Strongly to the Age Learners Started Learning the L 2 Than to Years of Classroom Instruction, Whereas the Opposite Will Be the Case for Scores on Tests of Explicit Knowledge Prediction: age of learning English will be a stronger predictor of implicit knowledge than years of classroom instruction, and explicit knowledge will conversely favor years of classroom instruction. Hypothesis 7: Supported

CONCLUSIONS The results of this study are of importance to the field of SLA, as they demonstrate that providing relatively separate measures of explicit and implicit learning might be possible. There is a need to replicate the results of this study to provide a stronger foundation Direction for future inquiry: determine the extent to which the kinds of tests of grammatical proficiency used here are predictive of general language proficiency

CRITICAL QUESTIONS • Ellis claims that no past study has tested the relationship between implicit and explicit knowledge empirically, and that his is the first (building on his previous work). • Is this true? I do not see where De. Keyser’s study did not do the following: • “Empirical evidence (the record of one's direct observations or experiences) can be analyzed quantitatively or qualitatively. ” • External criteria for L 2 learners not strict enough or evenly controlled

READER QUESTIONS