Mapping DDI 2 to DDI 4 Larry Hoyle

Mapping DDI 2 to DDI 4, Larry Hoyle and Joachim Wackerow • This poster describes a mapping of leaf DDI 2 elements into corresponding DDI 4 properties. The mapping can be described as a multi-column, machine actionable, table. • DDI 2 and the later versions differ in the underlying reliance on reusable objects. This makes mapping from DDI 2 to DDI 4 complicated. Software using this table can also collapse content that is repeated in DDI 2 (like Code. Lists) into single reused DDI 4 objects. • The table is derived from a spreadsheet listing DDI 2 elements important to CESSDA. That table was further refined at the DDI 4 Norway Sprint. The table below right explains the role of each column in the mapping table. Those columns contain the information needed to move information from one model to the other. Examples in the table below are color coded to corresponding cells in the table above. The example below is taken from the European Social Survey file ESS 1 e 06. 6. xml Maps var. Grp to an object DDI 2 var. Grp “Weights” Variable. Collection object Id = 84 f 810 fb-7555 -4 bb 8 -af 88 -2148 baca 7 d 85 <var. Grp ID="WEIGHTS" type="other" var="V 564 V 565 V 566"> <labl> Weights </labl> (Contains ) </var. Grp> URN: DDI: example. org: b 17688 c 1 -97 a 9 -41 f 0 -8 ff 1 -4 b 7 fe 367 c 3 f 3: 1 DDI 25 ject <labl> Design weight </labl> sl ap o lt Identifiables. Mapping Abstract. Substitution Study/ has. Instance. Variable/ uses. Unit. Type/ Definition /code. Book: Study, /code. Book/ data. Dscr/ var: Instance. Variable, /code. Book/data. Dscr/ var/anlys. Unit: Unit. Type /code. Book/ data. Dscr/ var/ qstn/ pre. QTxt Study/ has. Instance. Variable/ source. Capture/ has. Instruction/ instruction. Text/ text. Content/ text/content /code. Book: Study, /code. Book/data. Dscr/ var: Instance. Variable, /code. Book/data. Dscr/var/ qstn: Represented. Question, /code. Book/data. Dscr/var/ qstn: Instruction Study/ has. Instance. Variable/ local. Id. Value /code. Book: Study, /code. Book/ data. Dscr/ var: Instance. Variable /code. Book/ data. Dscr/ var/@ID /code. Book/ Study/has. Variable. Collection/ data. Dscr/ has. Annotation/ var. Grp/@var summary/ language. Specific. String/ Content Capture: Represented. Question, Dynamic. Text. Content: Literal. Text Parameter. Values Is. Id. Ref DDI 4 Id. Ref. Path DDI 4 Id. Ref. Append Represented. Question / has. Instruction/ instruction. Text/ text. Content/ purpose/ language. Specific. Structured. String/ content='Pre. Question. Text ' FALSE Study/ has. Variable. Collection / has. Annotation/ summary/ language. Specific. String / scope='DDI 2 variable IDs ' /code. Book: Study, /code. Book/ data. Dscr/ var. Grp: Variable. Collection Notes TRUE contains/ member Program will need to match DDI 2 variable ID to DDI 4 Ddi. Urn for references to the variable like var. Grp TRUE, FALSE requires matching DDI 2 ID with DDI 4 Ddi. Urn and inserting DDI 4 URN references into the Variable. Collection Study/ /code. Book: Study, has. Instance. Variable/ display. Label/ /code. Book/data. Dscr/ language. Specific. Structured. String var: Instance. Variable / content ty er p ro DDI 4 Property. Name /code. Book/ data. Dscr/ var/ anlys. Unit /code. Book/ data. Dscr/ var/ Labl Instance. Variable object Id = b 17688 c 1 -97 a 9 -41 f 0 -8 ff 1 -4 b 7 fe 367 c 3 f 3 Name = “dweight” display. Label = Maps var to an ob <var ID="V 564" name="dweight" wgt="wgt" files="F 1" dcml="2" intrvl="contin"> <location width="4"/> ap ab M <qstn> <qstn. Lit> R 34 Design weight </qstn. Lit> </qstn> <valrng> <range UNITS="REAL" min="0. 0044"max="4. 34"/> </valrng> <sum. Stat type="vald"> 42359 </sum. Stat> <sum. Stat type="invd"> 0 </sum. Stat> <var. Format type="numeric“ schema="other"/> </var> Label. For. Display Structured. Datatype language. Specific. Structured. String = location. Variant = valid. Dates = max. Length = Maps var to an object <var ID="V 565" name="pspwght" wgt="wgt" files="F 1" dcml="2" intrvl="contin"> <location width="4"/> <labl> Post-stratification weight including design weight </labl> <qstn. Lit> R 35 Post-stratification weight including design weight </qstn. Lit> </qstn> <valrng> ct bje <range UNITS="REAL" no a t to Sta m min="0. 000750077458907839" su ps Ma max="6. 85496650535486"/> </valrng> <sum. Stat type="vald"> 42359 </sum. Stat> <sum. Stat type="invd"> 0 </sum. Stat> <var. Format type="numeric" schema="other"/> </var> (Contains ) URN: DDI: example. org: b 10 f 7 a 22 -97 f 6 -4971 -99 cf-481 e 2 cc 12367: 1 Language. Specific. Structured. String. Type Structured. Datatype Content = “Design weight” language = scope = … is. Plain. Text = structure. Used = other. Defined = DDI 2 var “V 565” Post Processing This table can grow as new element types are found in DDI 2 instances. DDI 4 Objects DDI 2 XML DDI 2 var “V 564” Sample rows from the mapping table. The full table currently lists 181 DDI 25 XPaths. Column Descriptions for the Table Above Column DDI 25 Function The unpredicated XPath of a DDI 2 text or attribute node /code. Book/data. Dscr/var/anlys. Unit The corresponding path to a leaf in DDI 4 Study/has. Instance. Variable/ uses. Unit. Type/definition DDI 4 Property. Name Instance. Variable object Id = b 10 f 7 a 22 -97 f 6 -4971 -99 cf-481 e 2 cc 12367 Identifiables. Mapping (for Instance. Variable ) URN: DDI: example. org: b 10 f 7 a 22 -97 f 6 -4971 -99 cf-481 e 2 cc 12367: 1 Abstract. Substitution Variable. Statistics object Id = 00091102 -8197 -4387 -b 738 a 1 bd 8 eb 9 c 736 has. Summary. Statistic = Parameter. Values Summary. Statistc Structured. Datatype Content = “Design weight” type. Of. Statistic = has. Statistic = Is. Id. Ref Statistic Structured. Datatype Content = “Design weight” content = “ 42359” is. Weighted = computation. Base = type. Of. Numeric. Value = • As DDI 2 instances are processed, XPaths of leaf elements with content that do not have an entry in the mapping table can be recorded for possible inclusion in future versions of the mapping table. • Some elements in DDI 2, like var. Grp, will make reference to elements that appear later in the document. This means that assigning Instance. Variables to a Variable. Collection or Instance. Variables to Variable. Statistics should be done after a first pass through the DDI 2 document. • DDI 4 allows reuse. Another type of post-processing can collapse duplicated content like repeated codelists into references to just one instance. This process might require parameters to dictate how aggressive the collapsing is. For example should all identical categories be collapsed into a single one, or only those together in duplicate Category. Sets? • We’re working on adding some of these features to the DDI 4 R R package. DDI 4 Id. Ref. Path DDI 4 Id. Ref. Append Notes This maps a DDI 2 sub-path to a DDI 4 Identifiable class. An object of that class will need to be created for each unique instance of that DDI 2 sub-path. Some references in the DDI 4 model are to abstract classes. In these cases it is necessary to specify which extension of the abstract class should be used in the mapping. Some values in DDI 4 can use additional explanatory metadata. This column lists the path and the value for that information. Is this value a reference to an ID in the DDI 2 XML (an xs: IDREF)? , if so this will ultimately need to be transformed into a proper reference in DDI 4 through a DDI URN. This is the sub-path within the last DDI 4 identifiable object for the DDI URN of the referenced object. This describes whether to append or replace values in the DDI 4 path. Used to describe any notes for the mapping Example Details Each piece of information to be imported from DDI 2 should have a corresponding XPath listed. The first node in this path is a DDI 4 class. The remaining nodes are properties in a chain down to a leaf value. The value of some properties are references to other objects, that object may need to be created. Other values are "structured datatypes". /code. Book: Study, In the example to the left, for each unique var /code. Book/data. Dscr/var: Instance. Variable, element in a DDI 2 instance, the same DDI 4 Instance. Variable needs to be used. A predicated /code. Book/data. Dscr/var/anlys. Unit: Unit. Ty XPath identifies a specific DDI 4 object, e. g. pe /code. Book[1]/data. Dscr[2]/var[7] indicates a sepcific Instance. Variable. In this example the 7 th variable in the 2 nd data. Dscr element of the codebook always maps to the same reusable Instance. Variable. Capture: Represented. Question, In this example a source. Capture associates with Dynamic. Text. Content: Literal. Text the abstract class Capture. The mapping will use the Represented. Question extension of the Capture. Represented. Question/has. Instruction/ Instruction. Text/text. Content/purpose/ language. Specific. Structured. String/ content='Pre. Question. Text' The Literal. Text above is further described as "Pre. Question. Text" TRUE An example is the @var attribute of the DDI 2 var. Grp. This will need to be implemented as a reference to an Instance. Variable in a Variable. Collection in DDI 4. contains/member In the case of a var. Grp, the Variable. Collection has a contains/member value that is the URN of the DDI 4 Instance. Variable created to match the DDI 2 var referenced by the @var IDREF. In the case above, there may be more than one member property under contains, but there can only be one value for member. In this example, it describes what is needed in the matching process of DDI 2 IDs and DDI 4 URNS. TRUE, FALSE requires matching DDI 2 ID with DDI 4 Ddi. Urn and inserting DDI 4 URN references into the Variable. Collection DDI 25, DDI 4 Property. Name, and Identifiables. Mapping columns from DDI 2 anlys. Unit Abstract. Substitution and Parameter. Values from DDI 2 pre. QTxt Is. Id. Ref, DDI 4 Id. Ref. Path, DDI 4 Id. Ref. Append, and Notes from DDI 2 var. Grp/@var North American Data Documentation Initiative Conference (NADDI 2019), Ottawa, Canada, April 2019. Larry Hoyle IPSR, University of Kansas; Joachim Wackerow, GESIS - Leibniz Institute for the Social Sciences