Link Plus Probabilistic Record Linkage Software Application Overview

Link Plus Probabilistic Record Linkage Software Application Overview. Net Conference with IMPAC August 1, 2006

CDC–NPCR Link Plus Contacts Kathleen K. Thoburn, CDC/NPCR Contractor E-mail: kthoburn@cdc. gov David Gu, CDC/NPCR Contractor E-mail: dgu@cdc. gov Tom Rawson, CDC Computer Programmer

Outline • Link Plus Software Overview • Link Plus Linkage Overview • Link Plus Demonstration

Overview of Link Plus Application

Link Plus Software • Stand-alone probabilistic record linkage program • Combines ease of use and statistical sophistication • Detects duplicates within a cancer registry, or links cancer registry files to external files • Supports North American Association of Central Cancer Registries files, fixed width files, delimited files, and CRS Plus database

Link Plus Software • Computes probabilistic record linkage scores based on theoretical frame work developed by Fellegi and Sunter – Fellegi, I. P. , and A. B. Sunter (1969), "A Theory for Record Linkage, " Journal of the American Statistical Association, 64, pp. 1183 -1210 • Can handle missing values of matching variables – automatically treats null or empty values as missing data and allows user to indicate additional values to be treated as missing data

Link Plus Software • Facilitates a simple and efficient blocking ("OR blocking"): indexes blocking variables and compares pairs with identical values on at least one blocking variable • Provides powerful support for manual review of uncertain matches

Link Plus Is Free $0. 00

Link Plus Is Easy To Use Link Plus gets you from HERE: Cancer Registry data for John Smith: Last name First Name Site SSN DOB SMITH JOHN C 619 123654789 02111934 Sex 1 Date. Dx 06152004 Vital Statistics data for John Smith: Last name First Name DOB Date of Death COD Death Cert # SMITH JON 02011934 03202006 123654789 01234

Link Plus Is Easy To Use To HERE: Linked data for John Smith: Last name First Name Site SSN DOB SMITH JOHN C 619 123654789 02011934 Sex 1 Date. Dx Death Date 06152004 03202006 COD Death Cert # C 100 01234

Link Plus Is Easy To Use Without having to go HERE:

Link Plus Is Easy To Use • Designed especially for cancer registry work – HOWEVER, can be used with any data • Mathematics largely hidden from user • Practical default values supplied for many tasks • Familiar Windows interface • Includes Help and test examples

Link Plus Is Robust • Program written by a mathematical statistician • Specifications based on research into the published literature • Tested by researchers experienced in record-linkage • Results are clear and accessible to novice users

Overview of Link Plus Linkage Process

Probabilistic Matching • Find the records in File 2 that seem to match records in File 1 • Calculate a score that indicates, for any pair of records, how likely it is that they both refer to the same person • Sort the likely and possible matched pairs in order of their scores • Define a threshold (Cut Off value) for automatically accepting and rejecting a potential link – Discard unlikely matched pairs (scores below Cut Off) – Gray area: range of scores considered as uncertain matches • Manually review uncertain matches

Link Plus Linkage Overview Two main types of linkage: • External Linkage – Probabilistically link one file to another file • Deduplication – Special case of record linkage – Records in the same file are blocked, compared, and scored against each other – Result is a ranked list of record pairs – High-scoring pairs may be duplicates

Link Plus Linkage Overview

Link Plus Linkage Overview External Linkage Steps: 1. Select Data Type for File 1 9. 2. Locate/Identify File 1 10. Define Missing Values 3. Data Import for File 1 11. Select Direct/EM Method 4. Select Data Type for File 2 12. Enter Cut-off Value 5. Locate/Identify File 2 6. Data Import for File 2 13. Specify Linkage File Name and Location 7. Select Blocking Variables & Phonetic System 8. Select Matching Variables & Matching Methods Select ID Variables 14. Perform Manual Review of Uncertain Matches 15. Export Merged File

File Import File 1 verses File 2 • File 1 and File 2 have a one to many relationship – File 1 (ONE); File 2 (MANY) • Cancer Registry is File 1; External file is File 2: – Non_Match. Report. txt contains records from External file not matched to records in CCR (death clearance for unreported tumors) • External File is File 1; Cancer Registry is File 2 – Non_Match. Report. txt contains records from CCR file not matched to records in External file (follow up for date of death)

Link Plus Linkage Overview Deduplication Linkage Steps: 1. Select Data Type for File 7. Define Missing Values 2. Locate/Identify File 8. Select Direct/EM Method 3. Data Import for File 9. Enter Cut-off Value 4. Select Blocking Variables & Phonetic System 10. Specify Linkage File Name and Location 5. Select Matching Variables & Matching Methods 11. Perform Manual Review of Uncertain Matches 6. Select ID Variables 12. Export Merged File

Manual Review of Uncertain Matches

Manual Review of Uncertain Matches • Manual review screen provides special sorting that always keeps the comparison pairs together • Fields that are not on the linkage report can be included for manual review • Users can switch between the two viewing modes: the Datasheet View and the Pair View • Match status may be assigned manually, or may be assigned automatically by match score

Manual Review of Uncertain Matches • Includes the option to restrict view to only uncertain matches • Individual columns may be sorted, hidden and unhidden from view, and re-sized • Order of the columns on the review screen may be modified • Review sessions may be saved and re-opened at a later time • Allows two reviews to be compared so that the difference can be resolved into a final review file

Helpful Link Plus Tips • With real data Link Plus may take awhile to read data files • Be patient - Linkage times vary – IHS Linkage (2. 4 million records) With VS Data: • Oregon: 2 hrs/400, 000 recs • Michigan: 6. 5 hrs/1, 207, 000 recs With Cancer Registry Data: • Oregon: 36 min/94, 172 recs • California: 13 hrs/1, 935, 255 recs • Wait (but not days) – something went wrong – Can run out of virtual memory • Takes up CPU – Shutdown and restart computer right before linkage to clear up as much space as possible prior to linkage – Turn off screen saver/Close all other programs

Obtaining Beta Version* • Go to NPCR Home Page: http: //www. cdc. gov/cancer/npcr • In the ‘Tools’ Section – • Under ‘Registry Plus Components’ – • click on Registry Plus click on Link Plus Under ‘On this page’ – click on Installing and Upgrading Link Plus *Release of Link Plus Version 2. 0 scheduled for Fall 2006

Link Plus Linkage Demonstration