Collect Store and ReUse Paradata in DDI we

Collect, Store, and Re-Use Paradata in DDI – we need a working group EDDI 2018, Berlin (Germany) David H. Schiller, M. A. (TBA 21 Germany) The TBA 21 Group Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland

Outline • • • TBA 21 Group What is Paradata? Why is it important for research? Why is there a need to cover it by a standard? I feel alone. Lets have a working group! The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 2

The TBA 21 Group • Origin: department TBA (Technology Based Assessment) at DIPF in Frankfurt/Main, TBA 21 established as a spin-off in 2012 Group Structure: • Nürnberg, Germany: Management, concepts, coordination, expertise, sales office • Szeged, Hungary: Programming, expertise, sales office • Chur, Switzerland: Sales office, expertise • Freelancer/subcontractor network: Programming, expertise The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 3

The TBA 21 Group Customers: • EDK (Switzerland), IAB, Bundesbank, OECD, IAL, Statistic Austria, GFK, DIPF, TAO. Products: • Data Analysis, Data Documentation, Data Management, Data Portals, Assessment, Questionnaires, Surveys. Associate member of the DDI Alliance The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 4

What is Paradata? • If metadata describes the already collected research data, describes paradata the process of data collection? (David, abstract EDDI 2018) • The paradata of a data set or survey are data about the process by which the data were collected. (Wikipedia, Groves/Heeringa, 2006) • Paradata is data on study processes and the collection of study data. (The Paradata Information Model, Greenfield/Carpenter, 2013, presented at NADDI). • Paradata – Big Data of Survey Research (Kreuter, 2015) The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 5

Why is it important for research? • Paradata is important to assess the quality of research data. It can help to evaluate and improve the data collection process. (David, abstract EDDI 2018) • Example paradata topics include the times of day interviews, how long the interviews took, how many times there were contacts with each interviewee. • Paradata about each observation. [Helping to evaluate] the costs of a survey, the findings of a survey, interviewers, and non-responds. (Taylor, 2008) • Paradata information can be used to help achieve the goals of a survey. For example, early responses may be mainly from one type of respondent, and the collectors knowing this can focus on reaching the other types so the survey has good coverage of the intended population. (Kreuter, 2013) The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 6

Why is it important for research? Paradata is important for two main topics: • Monitoring and controlling the fieldwork processes. • Evaluating the research data quality. The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 7

Real life example… The ÜGK study in Switzerland (Survey on basic skills of pupils). • Analysis units are: pupils in classes in schools. • They conduct tests and complete questinaires. • Paradata about the test environment and the test realization are important for further data management. • E. g. the fire alarm during the test is responsible for poorer results afterwards. The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 8

Why is there a need for a standard • Currently and in many cases paradata is collected: – – as written notes, not standardized, if the interviewer had time to take notes, or not at all. • Notes may: – get lost, – need to be scanned, – are not machine readable. The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 9

I feel alone. Lets have a working group. • Why not improving research by collecting paradata using software (within a tool chain) and store it in a useful and common standard, let’s say: DDI? • Working Groups play a vital role in the development of DDI and its semantic products. They also advise the Executive Board or the Scientific Board. (www. ddialliance. org) The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 10

I feel alone. Lets have a working group. So: • What do you think? Is there a need? • Are you or your organisation interested in participating? • The group should consist of experts from different areas and different regions. • Meetings will be virtual; while face to face meetings can be organized alongeside EDDI and NADDI conferences. The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland 11

TBA 21 Germany Gmb. H, Nürnberg Technology Based Assessment for the 21 Century David H. Schiller, M. A. schiller@tba 21. net The TBA 21 Group - Nürnberg, Germany – Szeged, Hungary – Chur, Switzerland