Taxonomy Strategies LLC Frequently Asked Questions about Taxonomies

Taxonomy Strategies LLC Frequently Asked Questions about Taxonomies and Metadata Ron Daniel Taxonomy Strategies LLC rdaniel@taxonomystrategies. com Sept. 28, 2005 Copyright 2005 Taxonomy Strategies LLC. All rights reserved.

Pop Quiz On a blank piece of paper: 1. What questions did you want to have answered by coming to today’s talks? 2. What new questions do you have, based on what you’ve learned from the previous presentations? Flag one question to be answered later. You do NOT have to provide your name. Please DO provide your job title, division, and either company or company type. Taxonomy Strategies LLC The business of organized information 2

Agenda § Pop Quiz § FAQs – Frequently Asked Questions § SAQs – Seldom Asked Questions § Today’s Questions Taxonomy Strategies LLC The business of organized information 3

What is a taxonomy – just a folder structure or something else? n Irony in action – there is no agreed definition of what a “taxonomy” is. § When talking with someone about taxonomy, make sure you are talking about the same things. n We look at taxonomies and metadata together. § The metadata specification will call for several fields that take pre-defined lists of values. § Those lists, flat or hierarchical, are “facets” within the overall taxonomy. Taxonomy Strategies LLC The business of organized information 4

Other things sometimes called taxonomy Type Remarks Synonym Ring 4 Connects a series of terms together 4 Treats them as equivalent for search purposes e. g (Dog, Canine, Pooch, Mutt) (Cat, Feline, Kitty), … Authority File 4 Used to control variant names with a preferred term 4 Typically used for names of countries, individuals, organizations e. g. (IBM, Big Blue, International Business Machines Inc. ) Classification Scheme 4 A hierarchical arrangement of terms 4 May or may not follow strict “is-a” hierarchy rules 4 Usually enumerated; ie, LC or Dewey Thesaurus 4 Expresses semantic relationships of: • Hierarchy (broader & narrower terms) • Equivalence (synonyms) • Associative (related terms) 4 May include definitions 4 Resembles faceted taxonomy but uses richer semantic relationships among terms and attributes and strict specification rules 4 A model of reality Ontology Taxonomy Strategies LLC The business of organized information 5

How do taxonomies actually improve search? Input (Query) Side § “Search” using a small set of pre-defined values instead of trying to guess what word or words might have been used in the content. § Have synonyms mapped together so searches for “car” and “automobile” return the same things. Output (Results) Side § Organize search results into groups of related items. § Sorting and filtering § Refinement Taxonomy Strategies LLC The business of organized information 6

Taxonomy in action on the results side n Position Category n Company n City n State n Salary Taxonomy Strategies LLC The business of organized information 7

Who should build the taxonomy? n The taxonomy (and metadata specification) should be produced by a cross-functional team which includes business, technical, information management, and content creation stakeholders. n The team should plan on maintaining the taxonomy as well as building it. § Maintenance will not (usually) be anyone’s full-time job. § Exact mix of people on team will change. n It should be built in an iterative fashion, with more content and broader review for each iteration. Taxonomy Strategies LLC The business of organized information 8

How big should the taxonomy be? n Consultant’s answer – “It depends” § How much content do you need to organize? § How fine-grained does the categorization need to be? n Overly-simplistic method: § Nterms = # items / desired bucket size § (1 M documents, 100 documents / bucket = > 10 k buckets) § Bad method – documents don’t distribute evenly n Second method: § # facets ≈ Log(# items) ± 2 § (1 M items => 5. . 7 facets) § Sum of terms across all facets < 1200 in most cases Taxonomy Strategies LLC The business of organized information 9

How do we know we have a good taxonomy? Method Process Who Requires Validation Walk-thru Show & explain 4 Taxonomist 4 SME 4 Team 4 Rough taxonomy Walk-thru Check conformance to editorial rules 4 Taxonomist 4 Draft 4 Consistent look and feel taxonomy 4 Editorial Rules Usability Testing Contextual analysis (card sorting, scenario testing, etc. ) 4 Users 4 Rough taxonomy 4 Tasks & Answers 4 Tasks are completed successfully 4 Time to complete task is reduced User Satisfaction Survey 4 Users 4 Rough Taxonomy 4 UI Mockup 4 Search prototype 4 Reaction to taxonomy 4 Reaction to new interface 4 Reaction to search results Tagging Samples Tag sample content with taxonomy 4 Taxonomist 4 Team 4 Indexers 4 Sample content 4 Rough taxonomy (or better) 4 Content ‘fit’ 4 Fills out content inventory 4 Training materials for people & algorithms 4 Basis for quantitative methods Taxonomy Strategies LLC The business of organized information 4 Approach 4 Appropriateness to task 10

Taxonomy validation: Tagging content How many items? Goal Number of Items Illustrate metadata schema 1 -3 Criteria Random (excluding junk) Develop training documentation 10 -20 Show typical & unusual cases Qualitative test of small vocabulary (<100 categories) 25 -50 Random (excluding junk) Quantitative test of vocabularies 3 -10 X number of categories Use computer-assisted methods when more than 10 -20 categories. Pre-existing metadata is the most meaningful. v The best way to validate a taxonomy is to use it to tag some content. Taxonomy Strategies LLC The business of organized information 11

Taxonomy validation: Closed card sorting n Useful to validate whether the terms in a taxonomy are organized in a way that is commonly understood. § Ask people to sort narrower terms in a taxonomy into the broad categories or facets. § The card sort is considered closed if you provide the names of those broad categories. § Ask people if there are facets that they think should be added and why. § 15 -20 users are sufficient to get useful feedback. Taxonomy Strategies LLC The business of organized information 12

Taxonomy validation: Quantitative Method How evenly does it divide the content? § Documents will not distribute uniformly across categories § Zipf (1/x) distribution is expected behavior § 80/20 rule in action (actually 70/20 rule) Leading candidate for splitting Above the curve is better than expected Leading candidates for merging Taxonomy Strategies LLC The business of organized information 13

What if I have to do it solo? n. Realize: § Its not totally solo – IT help, Graphics & UI help, Business Goals help, Funding help, Review & QA help… § You are the general contractor § It needs to be part of your objectives § Limit the objectives to what can be achieved by you, and by your organization n. Concentrate: § Resource allocation § (i. e. Manage your time) § Fundamental processes § Query log examination § Error correction procedure § Communications!!! Taxonomy Strategies LLC The business of organized information n Cherry-pick from Roles on a larger team: § Business Lead – align with organization goals, get needed resources, make cost/benefit decisions, report upstairs § IT Liaison – Work with IT specialists to get software installed, logs gathered, content harvested, etc. Consider impact of changes on tools and data § Taxonomy / Search Specialist – analyze behavior and suggest changes. Implement changes which pass cost/benefit muster § Website/User Representative – consider impact of changes on users and job performance 14

Where do the benefits come from? Common taxonomy ROI scenarios n Catalog site - ROI based on increased sales through improved: § Product findability § Product cross-sells and up-sells § Customer loyalty n Call center - ROI based on cutting costs through: § Fewer customer calls due to improved website self-service § Faster, more accurate CSR responses through better information access n Compliance – ROI based on: § Avoiding penalties for breaching regulations § Following required procedures (e. g. Medical claims) n Knowledge worker productivity - ROI based on cutting costs through: § Less time searching for things § Less time recreating existing materials, with knock-on benefits of less confusion and reduced storage and backup costs n Executive mandate § No ROI at the start, just someone with a vision and the budget to make it happen Taxonomy Strategies LLC The business of organized information 15

Agenda § Pop Quiz § FAQs – Frequently Asked Questions § SAQs – Seldom Asked Questions § Your Questions Taxonomy Strategies LLC The business of organized information 16

What should I be thinking about at the start of a taxonomy project? Taxonomy development and maintenance is the LEAST of three problems: § The Taxonomy Problem: How are we going to build and maintain the lists of pre-defined values that can go into some of the metadata elements? § The Tagging Problem: How are we going to populate metadata elements with complete and consistent values? § What can we expect to get from automatic classifiers? What kind of error detection and error correction procedures do we need? What fields do we need? § The ROI (Return On Investment) Problem: How are we going to use content, metadata, and vocabularies in applications to obtain business benefits? § More sales? Lower support costs? Greater productivity? Risk avoidance? § How much content? How big an operating budget? How to expose to users? Business Goals and Cultural Factors are major influences on tagging and taxonomy. These must be acknowledged at the start to avoid rework. Taxonomy Strategies LLC The business of organized information 17

What must change when the Taxonomy changes? There’s more to maintaining the Taxonomy than maintaining just the taxonomy. þ The master copy of the taxonomy. þ Announcements for stakeholders! þ The information sent to downstream users of the taxonomy. þThe versions and formats of the taxonomy distributed to others. þThe list of changes. þ The data tagged with the taxonomy? þ The user interface which uses the taxonomy? þ Backend system software which uses the taxonomy? þ The training set for automatic classifiers? þ The educational material for users, catalogers, programmers, etc. ? Taxonomy Strategies LLC The business of organized information 18

Agenda § Pop Quiz § FAQs – Frequently Asked Questions § SAQs – Seldom Asked Questions § Your Questions Taxonomy Strategies LLC The business of organized information 19

Backup Slides Taxonomy Strategies LLC The business of organized information 20

Why do we usually recommend faceted taxonomies? n Categorize in multiple, independent, categories. n Allow combinations of categories to narrow the choice of items. n 4 independent categories of 10 nodes each have the same discriminatory power as one hierarchy of 10, 000 nodes (104) Main Ingredients • • • Chocolate Dairy Fruits Grains Meat & Seafood Nuts Olives Pasta Spices & Seasonings Vegetables Meal Type • • • Breakfast Brunch Lunch Supper Dinner Snack Cooking Methods Cuisines • • • African American Asian Caribbean Continental Eclectic/ Fusion/ International Jewish Latin American Mediterranean Middle Eastern Vegetarian • • • • Advanced Bake Broil Fry Grill Marinade Microwave No Cooking Poach Quick Roast Sauté Slow Cooking • Steam • Stir-fry § Easier to maintain § Easier to reusue existing material § Can be easier to navigate, if software supports it Taxonomy Strategies LLC The business of organized information 42 values to maintain (10+6+11+15) 9900 combinations (10 x 6 x 11 x 15) 21

What could possibly go wrong with a little edit? n ERP (Enterprise Resource Planning) team made a change to the product line data element in the product hierarchy. n They did not know this data was used by downstream applications outside of ERP. n An item data standards council discovered the error. n If the error had not been identified and fixed, the company’s sales force would not be correctly compensated. “Lack of the enterprise data standards process in the item subject area has cost us at least 30 person days of just ‘category’ rework. ” Source: Danette Mc. Gilvray, Granite Falls Consulting, Inc. 22 Taxonomy Strategies LLC The business of organized information 22

When should we NOT use facets? n When you have to work with software that can’t handle them. § Remember, software is replaced but data is migrated. n When you need to use an existing standard taxonomy. Taxonomy Strategies LLC The business of organized information … By Content Type Calendars & Events Top Links… Holidays Upcoming Events Federal Reserve System… Beige Book Board of Governors FOMC More Calendars & Events… ERAC Officer Availability Staff Conference Toastmasters Tours Directories Documentation Forms News Policies & Procedures By Organization Federal Reserve System FRB Atlanta Board of Directors Executive Office Management Committee Research Division S&R Division Facets can help you build a useful hierarchy. This one is a mix of content type and organization. 23

What are facets I might think about? Wells Disciplines Facilities Lease Mgmt Other Orgs Countries & Regions Production Content Types Reserves Human Resources E&P Lifecycle Hydro carbon System Geologic Age Basins, Reservoirs & Fields Locations Org Chart Taxonomy Strategies LLC The business of organized information Process Mgmt 24

Taxonomy Strategies LLC Questions? Ron Daniel 925 -368 -8371 rdaniel@taxonomystrategies. com Sept. 28, 2005 Copyright 2005 Taxonomy Strategies LLC. All rights reserved.