Its good things and bad things to both

‘It’s good things and bad things to both systems’: AAE MORPHOSYNTACTIC FEATURES AND EDUCATIONAL EVALUATION Danny E. Flynn, Jr. & Dennis R. Preston Michigan State University

Background How occurrences of nonstandard English influence the evaluation of written work in schools is surprisingly understudied. This study seeks to discover the relative weights assigned to two linguistic features associated with good writing and to occurrences of nonstandard English, in this case, morphosyntactic features of African American English (AAE).

The Essay We selected an essay from a website, (http: //www. emsc. nysed. gov/ciai/dbq/assmtexp. html), prepared by the Social Studies Division of the New York State Education Department. It was given the highest possible ranking for a 12 th grade essay. The topic was democratic versus totalitarian forms of government. We then modified this “good essay” (GE) lexically and syntactically, resulting in a “bad essay” (BE) (as in Freedman 1978).

Lexical Modification We substituted less sophisticated vocabulary or paraphrases. Typical substitutions were as follows:

Syntactic Modification Using average words per “T-unit” (roughly, independent clauses with associated subordinate clauses) as a measure of sophistication, based on Hunt (1965), we did away with many complex grammatical constructions, reducing words per T-unit from 14. 6 to 9. 3. (Hunt’s work showed that average words per T-unit for 12 th graders is 14. 4, 8 th graders 11. 5, and 4 th graders 8. 6).

Typical syntactic modifications

For example, in the change from “In a totalitarian form of government, the citizens have no say in the decisions that are made. ” to “There are forms of government with only one person in charge. The citizens have no say in the decisions that are made. ” there is a replacement of the lexical item “totalitarian” with a paraphrase and a reduction of a complex sentence with a word per T-unit score of 17 to two with scores of 11.

AAE Modification The following 8 AAE morphosyntactic features were added to both the GE and the BE

AAE Modification (cont. )

Essays Submitted These same AAE constructions were then doubled in both texts (which we called “heavy” AAE use), resulting in six different essays: 1. GE-No. AAE (Good essay; no AAE) 2. GE-LAAE (Good essay; light AAE) 3. GE-HAAE (Good essay; heavy AAE) 4. BE-No. AAE (Bad essay; no AAE) 5. BE-LAAE (Bad essay; light AAE) 6. BE-HAAE (Bad essay; heavy AAE)

Methodology 1 These six essays were presented to undergraduate students at Michigan State University at the beginning of Fall Semester 2005 (N=173). The majority (N=93) are English majors and have secondary English teaching as their goal. The remaining respondents are mostly English and History majors. Demographic information (age, sex, ethnicity, region) and career goals and relevant experience (e. g. , tutoring) was sought for each respondent.

Methodology 2 Each student who participated saw only one form of the essay and was asked to grade this essay as if it were the work of a high school senior, using the traditional A-B-C-D-F scale. The participants were also encouraged to write comments on the essay itself or on the back of the op-scan sheet used to collect the data.

Methodology 3 The data were analyzed statistically after converting the A-F scale into a 0. 0 - 4. 0 scale, one more often associated with university-level work. The 0. 0 -4. 0 scale is used in the reporting of the data that follows, but 4. 0 may be equated with an “A” and so on down the scale. Before we look at the results, let’s imagine some of the possible outcomes:

POSSIBLE OUTCOME #1 Nothing counts for nothing. Everybody gets a “C, ” or you can imagine an “A” or an “F” if you like.

POSSIBLE OUTCOME #2 A good essay is rated high and a bad one low, regardless of AAE amount or presence.

POSSIBLE OUTCOME #3 Any AAE presence is rated low, regardless of amount or of essay quality.

POSSIBLE OUTCOME #4 Amount of AAE influences ratings, regardless of essay quality.

POSSIBLE OUTCOME #5 Amount of AAE and essay quality influence ratings.

ACTUAL OUTCOME This is most like #1 (“nothing”) and #5 (“interaction”); the red boxes show that 1 and 2 are significantly different from 6 (. 007).

Results 1 Although good essays were rated slightly better and presence and greater number of AAE structures resulted in slightly lower ratings, neither factor was robust. AAE presence by itself was not particularly important, as shown by the fact that Essay 2 (“good essay” with “light” AAE) was highest rated.

Results 2 But good essays were not very highlyranked in general. There was no statistical difference between the three “good essays (1, 2, and 3) and two of the “bad essays” (4 and 5). Only heavy AAE and “bad essay” combined to produce a statistically significant lower evaluation (. 007), and then only in contrast to essays 1 and 2.

Results 3 Respondent sex played no role in these ratings, and there were insufficient numbers of minority students to calculate. Student major was also not significant. Region, however, showed an interestingly different distribution of grades (. 05). 1) Detroit urban area 2) Other MI urban areas (Flint, Lansing, Grand Rapids, etc. ) 3) Rural Michigan 4) Other

Overall assignment of grades by region

Results 4 Rural respondents are much harsher in their ratings overall (. 05), but small numbers in each cell do not allow us to relate that to the quality of the essay or the presence or amount of AAE. These students, many English teachers in training, are in fact very harsh overall. The “good essay” (with no AAE) got a mark just barely above a “C. ” Experienced professionals obviously thought much better of it.

Some Qualitative Results We hope to shed some light on these quantitative results by surveying some of the qualitative data collected. For the moment we limit ourselves to some examples of these comments. We plan a more thorough investigation based on content classification.

Here are 3 comments on the worst-rated Bad. Essay/Heavy AAE This first respondent refused to assign a grade: “I stopped at this point [i. e. , end of paragraph 2]. There is no thesis, organization, transition or grammar. Please re-write, starting w/ a clear thesis & outline. The thesis needs some serious work! It needs to be more detailed, clearer, arguable etc…” [The respondent liberally annotated the essay with comments and corrections up to that point. ]

Grade: F “Your paper did seem to get better as I read on but it needs a lot of work. I would love to meet with you after school & discuss the corrections I have made. –Mrs. Teacher” [She corrected all AAE and offered several other suggestions which would have made the grammatical structure more complex. ] Grade: Grammar: D, Overall: C“Overall, first para. is to [sic] wordy, needs better thesis. Overusing the word this & they. PS, Is this a real essay? ” [This rater also wanted the essay double spaced and the first paragraph indented. ]

And here is a comment made on a Good Essay/Light. AAE GRADE: B “If there weren’t so many grammatical errors the essay would have recieved [sic] an “A”, but the errors were sufficient enough to lower the grade by a whole letter. ”

Finally, here are 3 comments on the Good. Essay/No. AAE Grade: B The respondent made the notation “first name” by “Mussolini. ” Grade: C “avoid ‘good’ and ‘bad’ use high school vocabulary” Grade: D “What a steaming pile of crap! This is one of the most boring papers I have ever read. ” [Other expurgated comments. ]

Some comments found in several essays included 1) Misidentification of many short sentences as “sentence fragments” (e. g. , “This causes them to be unhappy. ”). 2) Complaints that the essay contained “incorrect information, ” although the essay was written by a student who had been given a reading on these facts.

In conclusion, English teachers-to-be (at least at Michigan State University) are very fussy graders, but attention to AAAE grammatical features does not overwhelm their evaluation of these essays in general, although an interaction between heavy AAE use and a less sophisticated essay (with regard to vocabulary and structure) results in a statistically lower evaluation. What next?

1) What would real teachers (not teachers in training) do? Would those in other areas (e. g. , history, social studies) have rated these essays differently. What is the source of the rural-urban difference? 2) Would it have made any difference if we had told the respondents that this was “on-the-spot” work? We noted only that it was 12 th grade written work; some comments suggested that extensive proof-reading and revision would have helped.

3) What other characteristics of good and bad essays or other nonstandard features might be studied in this format? How important would they prove? 4) Would further demographic differences of raters (whether trainees or teachers) make any difference? Our respondent group was almost uniquely European American.

We hope to have more to report to you along the lines of these improvements and others which we hope to get from you. Thank you for your attention. You can see this Power. Point and the complete text of all six essays at www. msu. edu/~preston