Smalln Card Sort Jonah Loeb ENP 164 Final

Small-n Card Sort Jonah Loeb ENP – 164 Final Project

Description

Purpose of Card sort This card sort was conducted in conjunction with the user research phase of a senior capstone. The focus of the capstone was to create an improved headlamp for running and jogging. To accomplish this, it became necessary to understand the users’ current mental model for how headlamps work in order to see how any additional features and modes would be treated. The card sort was used as a method to understand the order in which users expect their headlamp’s modes to be and to see where they would add an additional mode.

What the data are The cards contained the following modes: Red Low High Auto Off Auto mode was the added mode. It represented a reactive lighting mode.

Methodology

Population Total n was equal to 16, n=8 male, n=8 female. The population tested were boy and girl scouts ranging from 4 th through 9 th grade. Testing was conducted during their weekly scout meetings with the permission of the child’s parents and the supervision of a scout leader. Tests were conducted in small focus groups with multiple children sorting separate decks of cards.

Data collection procedures The cards were randomized each time and handed to either an individual or a small group of 2 -3 based on the total size of the group. Children were allowed as much time as necessary to sort the cards. When finished, the tester would confirm that was the order they wanted and take a picture of the order. The data from the pictures was then consolidated. During consolidation, the off card was removed as the modes were cyclical and the off mode was included only to aid the understanding of the participants.

Analysis

Questions for analysis Q 1: Was the high-low order preferred over the low-high order in sorts? M 1: Two tailed Exact test of goodness-of-fit (Binomial test). The null hypothesis is that the number of observations for each order will be equal to the expected number. The alternative hypothesis is that the number of observations will differ from the expected number.

Modes sorted by Rank First Low High Red Auto Red Low Low Red Auto Low High Second Red High Low Low Auto High High Low High Auto Low Third High Red Auto Red High Low Red Auto High Red High Auto Fourth Auto Red Auto n/a Low Red Auto Red n/a Auto Red Red

Test for high/low order Sample High-Low Low-High Population High-Low Low-High 0. 375 0. 625 0. 5 Exact binomial test P value One tailed test: Two tailed 0. 227249 0. 454498 Binomial test

Q 1 Analysis Because the two tailed P-value is greater than 0. 05 we must retain the null hypothesis. This means that there is not enough evidence to suggest that there is a difference in the rates of either order.

Questions for analysis Q 2: Is average rank of each of the modes differ from the expected rank. M 2: Chi-square test with William’s Correction. Williams correction is necessary as chi-square tests give inaccurate results when the expected values are small (i. e. when the expected value for any of the factors is less than 5) The null hypothesis is that the average rank of each mode will equal the expected rank. The alternative hypothesis is that the average and expected ranks will differ.

Low Ranks sorted by Mode High 1 1 1 1 2 2 2 3 3 4 Auto 1 1 1 2 2 2 2 3 3 3 Red 1 1 2 2 3 3 4 4 4 4 1 1 2 3 3 4 4 4

Chi-Squared test Mode Average Rank Low High Auto Red 1. 8125 2. 1875 3. 066667 2. 8 Expected Rank 2. 5 N 16 16 15 15 Degrees of freedom 3 P value 0. 941773 William's correction 1. 013441 Adjusted P value 0. 929283 Corrected Chi -Square test

Q 2 Analysis Because the adjusted P-value is greater than 0. 05, we must retain the null hypothesis. This means that there is not enough evidence to suggest that the average rank of each mode is different from the expected rank.

Conclusions

Take-aways While neither of the statistical tests in this report yielded significant results, the data collected is still relevant. The data can be used to shape initial prototypes of the headlamp that can later be tested. The user’s mental model may change with an actual prototype compared to a more abstract card sort. The lack of significance means that this data would not be enough to justify a final decision for the production run of the headlamp; however, it can be a starting point for additional rounds of testing.