Crosscultural comparability of subjective ratings Ren Mttus University

Cross-cultural comparability of subjective ratings René Mõttus University of Tartu / University of Edinburgh Jüri Allik, Anu Realo, Helle Pullmann University of Tartu Jerome Rossier, Gregory Zecca, J. Ah-Kion, D. Amoussou-Yéyé, Rasa Barkauskiene, Martin Bäckström, Barry Oumar, U. Bhowon, Fredrik Björklund, Aleksandra Bochaver, Konstantin Bochaver, Deon de Bruin, Donatien Dahourou, Daouda Dougoumalé Cissé, Helena Cabrera, Sylvia Chen, Marcia Church, Timothy Church, Xiaohang Feng, Hyi-Sung Hwang, Peter Kuppens, Anna Kwiatkowska, Alfredas Laurinavicius, Khairul Anwar Mastor, David Matsumoto, Rainer Riemann, Joanna Schug, Brian Simpson, C. Ng Tseung

The problem When subjective ratings are used for cross-cultural comparisons, researchers sometimes do not like the results Chinese have more political freedom than Mexicans? People live longer where they describe themselves being more ill?

The personality example: Conscientiousness = being organized, dependable, purposeful, controlled The highest scoring countries – Senegal, Benin The lowest scoring countries – Japan, South-Korea High Conscientiousness predicts low GDP and poor health

“I am a reliable person”, “I am often late” The same responses to personality test items may refer to different trait levels in different cultures (Heine et al. , 2008) Culture-sepcific standards for the behaviours that characterize conscientiousness: – Being “on time” means a 25 min time-window in US but 39 min in Morocco (White, Valk & Dialmy, in press) As a result, in some cultures, self-ratings tend to be low simply because everyone is rated according to harsh standards

Is there anything we can do? We should make the differences in standards measurable Having measured the standards, we could correct selfratings accordingly Rarely done in cross-cultural personality studies

A ridiculously simple and cheap solution The technique of anchoring vignettes (King et al. , 2004) We have to ask everyone to rate something (relevant) that is the same for everyone: Short descriptions of hypothetical persons displaying various levels of the trait (called anchoring vignettes) The vignettes are the same for everyone, thus variance in their ratings can only reflect biases and error If the ratings differ systematically across cultures, this probably shows culture-specific standards for the trait

Moreover, the vignettes can do the magic Self-ratings can be “fixed” by anchoring them to vignette -ratings: Vignettes are the common norm/baseline for everyone Self-ratings can be recoded as differences from the common norm Anchored self-ratings are free from culture-specific standards, being thus more easily comparable

The present study. . .

Country Australia Benin Burkina. Faso China (Changchun) China (Beijing) Estonia Germany Hong-Kong Japan Lihtuania Malaysia Mali Mauritius Philippines Poland Russia Senegal South-Africa South-Korea Sweden Switzerland USA Language English French Chinese * Estonian German Chinese Japanese Lihtuanian Malay French Filipino Polish Russian French English Korean Swedish French English N 463 107 96 110 150 110 70 158 107 125 211 93 100 133 100 115 109 142 100 101 165

I am. . . Capable, efficient, competent __ __ __ Inept, unprepared Disorganized, sloppy __ __ __ Organized, neat, methodical Dutiful, scrupulous __ __ __ Unreliable, undependable Lazy, unambitious, aimless __ __ __ Ambitious, workaholic Disciplined, persistent, strong-willed __ __ __ Procrastinating, quitting, weak Spontaneous, careless, thoughtless __ __ __ Cautious, reflective, careful The person is. . . [Marc] often feels incapable of deciding and finding solutions to his problems. He always turns to his relatives and acquaintances for help and sometimes they indeed help him. However, at times the opinions of other people disagree, which makes it even more difficult for [Marc] to work out what he should do. [Thirty vignettes displaying various levels of conscientiousness]

Having the same trait-related information, people end up with fairly universal personality judgement Profile correlations ranged from 0. 96 to 1. 00

If self-ratings scattered around culture-specific standards. . . we would see as much culture-related variance in vignette-ratings than in self-ratings In self-ratings, differences in sample means explained 7 to 13% of total variance, depending on facet (median 9%) In vignettes, differences in sample means explained 2 to 10% of total variance, depending on vignette (median 4%)

The critical question If in some cultures people rate themselves highly because of lenient standards, they should also rate the vignettes highly (and the other way around) As a result, there should be a positive correlation between mean self-ratings and mean vignette-ratings But there is NO systematic trend

Mean self-ratings (solid lines) and vignette-ratings (dashed lines)

No need for fixing self-ratings in this case Otherwise, self-ratings could be “fixed” by anchoring them to vignette-ratings: Recoding Standardizing

Conclusions: Two good news, one not so good news In cases there are culture-specific standards, methods for identifying and correcting for them are available — NB! The use of the method is not restricted to crosscultural comparisons These data do not support the idea of culture-specific standards for subjective personality ratings The puzzling ranking of cultures on conscientiousness still needs to be explained

My thanks go to. . . Jüri Allik, Anu Realo, Helle Pullmann, Jerome Rossier, Gregory Zecca, J. Ah. Kion, D. Amoussou-Yéyé, Rasa Barkauskiene, Martin Bäckström, Barry Oumar, U. Bhowon, Fredrik Björklund, Aleksandra Bochaver, Konstantin Bochaver, Deon de Bruin, Donatien Dahourou, Daouda Dougoumalé Cissé, Helena Cabrera, Sylvia Chen, Marcia Church, Timothy Church, Xiaohang Feng, Hyi-Sung Hwang, Peter Kuppens, Anna Kwiatkowska, Alfredas Laurinavicius, Khairul Anwar Mastor, David Matsumoto, Rainer Riemann, Joanna Schug, Brian Simpson, C. Ng Tseung