How many subjects to run Depends Depends on

How many subjects to run? • Depends! • Depends on: – What you are measuring – What question you are asking – How certain you want to be – Ability to get subjects

Examples • Vision – Ss don’t vary much • Use more when then do, e. g. , with specticles, color blind – Measures may be noisy, so take more measures – understanding does not influence measures • Learning – Ss vary – Measures are noisy, so take more measures – Can’t be reused because of learning • Social aspects – Ss really vary – Measures are noisy, so take more iff possible – Can’t use yourself, theory says knowledge

Examples • Navigation of web site – If interested in learning, can be run once – If interested in leanring, use about 5 to 20 to 30 – Theory says, knoweldge of experiment may influence use, so can’t run yourself • Choice of icons, do you like them? – Can only ask once – Theory of experiement influences behavior (you like me, you might like my choices and experiment) • Choice of icons, can you ID them? • Chice of icons, how fast can you read them?

More formal answers • Provide context • See what others do • User Cohen’s Power calculations – Based on effect size (related to d’ !) • • If you knew, you would not be running experiment! See similar published papers Guess pilot – Likelihood of finding results • If you care, make prob to. 99, but these numbers are typically > 100 • If you can live with small effects lost, then use medium to large effect sizes