CREATING PRIMARY DIGITAL ARCHIVES VIA WEB ARCHIVING TECHNIQUES

CREATING PRIMARY DIGITAL ARCHIVES VIA WEB ARCHIVING TECHNIQUES Alexander Duryee SAA Research Forum 2013 -08 -13

THE PROBLEM Studies of the archives profession rarely span more than 3 -5 years I wanted to examine the profession with as broad a view as possible Surely, as archivists. .

So, we have to find extant primary sources on the archival profession Such as – the Archives and Archivists listserv! Unfortunately, its current form isn't very research-friendly

THE SOLUTION The A&A listserv archives are split between Miami University (L-SOFT), ibiblio (flat HTML), and SAA (Lyris) L-SOFT (1993 -2002, 2004 -2006) and ibiblio (2002) build their collections in predictable manners Lyris (2006 - ) is trickier – metadata and content are never on the same page

THE RESULT This gave me ~77, 000 of pages with lots of unwanted noise L-SOFT and ibiblio messages came with a full HTML/Javascript page Lyrasis pages had to be stitched back together It wasn't difficult to remove the web frontend markup, resulting in 'cleaned' messages

EXCEPT. . . I assumed that, by parsing out message metadata and preserving content, I'd have a corpus ready to start working on algorithmically I couldn't have been more wrong!

When we began working with the data, we realized it wasn't nearly clean enough for algorithmic methodologies (e. g. natural language processing) Yet, cleaning it would remove information that was immeasurably valuable to the right researcher (e. g. textual scholars) No matter what we do, we make the data unusable for certain research fields!

GLIMPSING THE FUTURE I suspect that this will be a common theme in archives as research uses and primary sources shift An object can be a number of things to a number of users – was the listserv a record of communications, style, or content? More saliently – how does this apply to archivists? What will that role be in the future?