Types of Validity Criterion Validity Predictive Validity Concurrent

• Slides: 17

Types of Validity Criterion Validity Predictive Validity Concurrent Validity Adapted from Sekaran, 2004 Content Validity Construct Validity Convergent Discriminant Validity

Types of Validity • Construct Validity • Extent to which hypotheses about construct are supported by data 1. Define construct, generate hypotheses about construct’s relation to other constructs 2. Develop comprehensive measure of construct & assess its reliability 3. Examine relationship of measure of construct to other, similar and dissimilar constructs • E. g. Height & Weight; Networking & career outcomes study; Learning Style Orientation measure

Establishing Construct Validity • Multi-trait multi-method matrix • Convergent validity coefficient • Absolute size of correlation between different measures of the same construct should be large & significantly different from zero • Discriminant validity coefficient • Correlations between the same construct measured by different methods should be significantly different from correlations of • Different constructs measured by different methods (across methods & across constructs) • Different constructs measured by same method (method bias)

Convergent & Discriminant Validity Corr b/w Objective (O) & Self-Reports (SR) of Height & Weight O-H SR-H O-W O-H 1. 00 SR-H . 98 1. 00 O-W . 55 . 56 1. 00 SR-W . 68 . 69 . 92 SR-W 1. 00

Convergent & Discriminant Validity of Objective & Self-Reports of Height & Weight • Convergent validity (across methods) • Objective and subjective measures of height are correlated. 98 • Objective and subjective measures of weight are correlated. 92

Discriminant Validity of Objective & Self-Reports of Height & Weight • Discriminant validity (across constructs) • Objective measures of height & weight are corr. 55 • Subjective measures of height & weight are corr. 69 • STRONG CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by same methods • i. e. , Are. 92 &. 98 significantly different from . 55 &. 69? • Convert rs to z scores and compare

Discriminant Validity of Objective & Self-Reports of Height & Weight • Discriminant validity (across constructs) • Objective height & subjective weight are corr . 68 • Subjective height & objective weight are corr. 56 • WEAK CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by different methods • i. e. , Are. 92 &. 98 significantly different from . 56 &. 68

Establishing Construct Validity in Networking Study • For convergent validity, different measures of the same construct should be highly correlated • Note: In networking study, diff measures= diff subscales – Corr b/w diff measures of career success (promotion, salary, perceived career success) range from. 20 to. 36 (moderate support) • . 20 &. 36 should not be sig diff from each other – Corr b/w diff measures of networking (increasing internal visibility, socializing etc. ) range from. 03 to. 43 (weak support) • . 03 &. 43 should not be sig diff from each other

Establishing Construct Validity in Networking Study • For discriminant validity, different measures of the same construct should be more highly correlated than different measures of different constructs – Correlations b/w career success & networking (. 01 to. 35) should be sig different from • Corr b/w diff measures of career success (. 20 to. 36) • Corr b/w diff measures of networking (. 03 to. 43)

Learning Style Study • Developed items by generating critical incidents (Study 1) – N=67 – Yes/no responses to statements – Recall of learning events • Two types of learning: theoretical, practical (2) • Two types of outcomes=success, failure (2) • 2 x 2 events per participant • 112 items constructed in total

Learning Style Study • Study 1 Part 2: • Created 112 items from critical incidents – Administered to 154 participants – 5 -point likert scale (agree/disagree) • Extracted 5 factor solution w/factor analyses • 54 items loaded highly on the 5 factors • Content validity: item sorting by 8 grad students – Also administered personality scale

Learning Style Study • Item Development Study (study 1) • Convergent Validity • High reliabilities of subscales of Learning Style (. 81 -. 91) • Corr b/w different measures (subscales) of learning style =. 01 to. 32 but 1 only corr is significant – Weak support for convergent validity of new learning style measure • Discriminant validity • Corr b/w different measures of different constructs (Learning Style & personality). 42 to. 01 should be lower than and sig diff from corr b/w different measures of same construct (subscales of learning style). 01 to. 32

Learning Style Orientation Measure • Validation Study (study 2) – N=350 -193 – New learning style, Personality, old Learning style, preferences for instructional & assessment methods – Construct validity • Confirmatory factor analysis confirms 5 dimensions • Reliability of new learning style subscales=. 74 to . 87 compared to… – Reliability of old learning style subscales=. 83 to. 86 – Reliability of personality subscales=. 86 to. 95

Learning Style Orientation Measure • Validation Study (study 2) – Convergent validity • Corr b/w similar measures of key construct =Corr b/w diff subscales of new learning style 01 to. 23 should be comparable to… – Corr b/w similar measures of other constructs in the study • Diff subscales of old learning style. 23 to. 40 • Diff subscales of personality. 01 to. 27

Learning Style Orientation Measure • Validation Study (study 2) – Discriminant validity • Corr b/w measures of similar constructs= Corr b/w new learning style subscales & old learning style =. 01 to. 31 • Corr b/w measures of different constructs – Corr b/w new learning style & personality subscales is . 01 to. 55 – Corr b/w old learning style & personality subscales= . 02 to. 38

Learning Style Orientation Measure • Validation Study – Incremental validity (aka construct validity) • Additional variance explained by old vs. new learning style measures in preferences for assessment & instruction DV Subjective assessment LSOM LSI (new) (old). 15. 01 Interactional instruction . 21 . 04 Informational instruction . 06 . 00

Types of Validity Criterion Validity Predictive Validity Concurrent Validity Adapted from Sekaran, 2004 Content Validity Construct Validity Convergent Discriminant Validity