Learning Analytics Process Theory March 24 2014 Generalizability

Learning Analytics: Process & Theory March 24, 2014

Generalizability • Does your model remain predictive when used in a new data set? • Underlies the cross-validation paradigm that is common in data mining • Knowing the context the model will be used in drives what kinds of generalization you should study

What kind(s) of generalizability • Did Baker et al. 2008 look for?

What kind(s) of generalizability • Did Baker et al. 2013 look for?

What kinds of generalizability • Could one test for?

For each kind of generalizability • Has anyone done it? • What are the advantages to doing it? • What are the practical challenges/barriers/costs?

Given… • Given the massive cost of conducting every possible type of generalizability analysis • How do we know when to stop?

Let’s go back to the class on validity… • Since we didn’t quite finish our discussion…

Types of validity we discussed • • • Ecological Construct Predictive Substantive Content • Can anyone define and give a quick example of each?

Exercise • In groups of 3 • Write the abstract of the worst EDM paper ever

Any group want to share?

Exercise #2 • In different groups of 3 • Now write the abstract of the best EDM paper ever

Any group want to share?

The End