Using HTML Metadata to Retrieve Relevant Images from
- Slides: 62
Using HTML Metadata to Retrieve Relevant Images from the World Wide Web Ethan V. Munson University of Wisconsin-Milwaukee
Why is image search important? • The Web is becoming the world’s primary information source • Images are one of the Web’s key features • Few WWW image search engines exist currently • Using textual search engines to find images manually is laborious
A Requirement for Web Image Search • We need an efficient method of discovering and indexing image content. • Two main sources of information about image content: – image processing – associated text • text content • markup
Related work • QBIC (the IBM Almaden Research Center) – indexes and retrieves images according to: – – shape color texture object layout – queries are formulated through visual examples – a sample image – user provided sketches
Related work QBIC system
Related work QBIC system
Related work QBIC system
QBIC: Advantages and Disadvantages • Advantages – well-developed visual query language – interesting GUI – queries are based on image appearance • Disadvantages – works only at the primitive feature level (color, texture, shape) – doesn’t recognize semantics of image • very sensitive to camera viewpoint – doesn’t scale up to the Web
Related work • Web. Seek (J. Smith & S. Chang, Columbia University) – performs a semi-automated classification of the images • • automatically extracts keywords from image file names computes the keyword histogram manually creates a subject hierarchy manually maps the images into the subject hierarchy – User can • browse the categories • search the categories by keyword • search the database using image features – color content
Webseek: Advantages/Disadvantages • Advantages – Large index of Web images – Supports both text and image search • Disadvantages – Not clear that database can scale up • Manual categorization is very expensive – Relevance feedback mechanism is computationally expensive
Related work • Web. Seer (M. Swain et al. , The University of Chicago) – uses associated text and markup to supplement information derived from analyzing image content – uses multiple kinds of metadata • image file names • alternate text • text of a hyperlink – decides which images are photographs, portraits, or computer generated drawing – research emphasized categorization, not metadata-based search
Why seek new image retrieval methods? • The number of WWW documents is growing rapidly and constantly changing • We need fast and efficient methods for finding images • Image processing is – – complex computationally expensive limited (misses true image semantics) unnecessary
Research Goals • Show that images can be found using HTML “metadata” – textual content – HTML tag structure – attribute values • Determine which metadata features are the best clues to image content
The URL Filter • assembles a list of URLs from the results returned by Alta Vista – parses the first page returned by Alta Vista – follows the URLs of results pages, retrieves these pages, and parses them – extracts list of URLs from the results pages
The Crawler • retrieves the pages • saves each page’s HTML source code in a separate file
“Tidy” • converts arbitrary and probably ill-formed HTML into XHTML
XHTML Parser • parses an XHTML document • builds an XHTML parse tree
The Document Analyzer • scans the parse tree for image URLs – an image URL appears in either an image or anchor element • converts relative URLs into absolute URLs • uses various heuristics to determine which URLs point to relevant images
Search Strategies • • Image’s file name Textual content of the TITLE element Value of the ALT attribute of IMG elements Textual content of anchor elements Value of the title attribute of anchor elements Textual content of the paragraph surrounding an image Textual content of any paragraph located within the same center element as the image • Textual content of heading elements
Image Retrieval Experiment
Experimental Questions • Which HTML features reveal the most information about image? – Do particular patterns of HTML structure carry useful information? • Do image search results depend on the type of query?
Informal Experiments • Conducted extensive informal testing – to check software correctness – to investigate possible metadata clues – to determine rules for filtering out images based on size • images smaller than 65 pixels in either dimension almost never contained useful content • reduced the number of images we had to classify
Metadata Clues 1 2 3 4 5 6 7 Image’s file name Textual content of the TITLE element Value of the ALT attribute of IMG elements Textual content of anchor elements Value of the title attribute of anchor elements Textual content of the paragraph surrounding an image Textual content of any paragraph located within the same center element as the image 8 Textual content of heading elements
Query Categories • Famous people “Gorbachev”, “Yeltsin”, and “Streisand” • Non-famous people “Yelena” and “Ekaterina” • Famous places “Paris” and “London” • Less-famous places “Bremen” and “Spokane” • Phenomena “Explosion”, “Sunset”, and “Hurricane”
Experimental Procedure • For each of the 12 queries – Alta Vista returned 200 URLs (20 groups of 10) – We used first, middle, and last groups (30 URLs) – Downloaded pages and all images on pages • excluding small images (< 65 pixels in either dimension) • 276 pages and 1578 images were accessible – Manually determined relevance of each image – Used our system to determine the effectiveness of each of the 8 metadata clue • standard information retrieval measures: precision and recall
Information Retrieval Measures Relevant, not retrieved A Relevant, retrieved B Nonrelevant, not retrieved Nonrelevant, retrieved C D • Recall = B/(A + B) – Warning: our study does not really test recall • We need a controlled sample of the Web, but instead, we are using Alta Vista’s biased sample • Precision = B/(B + D)
Recall Table
Precision Table
Key Results Overall percent of recall Overall percent of precision Image file name Textual content of TITLE Value of ALT 43. 5 % 62. 1 % 13. 7 % 70. 7 % 58. 2 % 87. 5 % • Image file name has poor recall for people’s names and excellent recall for less-famous cities • Famous names have poorer precision than non-famous and place names
Problems with this study • This is a single, small study – results must be replicated • No standard corpus for testing Web image search – our “recall” results are not reliable or truly sound • Our choice of tools may bias our results – Title tag may be important only because Alta Vista considers it important – Tidy may remove some clues • What is the structure of “<P> Text <IMG>”? – Analysis of “header” clue is questionable
Body P Body IMG P IMG
Conclusion • Existing content-based image retrieval systems are not good models for Web image search • HTML metadata is useful for Web image search – Image file name and document title are most useful – Alternate text is extremely precise, when present • HTML metadata should provide faster image search than image processing approaches – no need to download analyze images – can take advantage of existing search engines
Using HTML Metadata to Retrieve Relevant Images from the Web Ethan V. Munson Dept. of Electrical Engineering & Computer Science University of Wisconsin - Milwaukee munson@cs. uwm. edu http: //www. cs. uwm. edu/~multimedia
- Pengertian dan prinsip teknologi ramah lingkungan
- 5 year old tammy mistakenly believes
- Reading vipers inference
- Infer viper
- Structuralism vs functionalism
- How to enable enterprise vault in outlook
- Retrieve
- Doctype html html head
- Doctype html html head
- Head body html
- Doctype html html head
- Doctype html html head
- Real vs virtual image
- Hi ho di do
- Images.search.yahoo.com
- How to save images on google images
- Https://tw.images.search.yahoo.com/images/view
- Irrelevant sentences examples
- Decision making and relevant information
- Relevant cost adalah
- 5 step decision making process
- Consider all relevant criteria
- Broad problem area in research examples
- Culturally responsive vs culturally relevant
- Select significant and relevant information
- Anggaran biaya variabel
- Ad hominem
- Contribution margin
- Relevant cost for decision making exercises
- Relevant cost for decision making solution chapter 13
- Decision making and relevant information
- What is font
- Why is the bible still relevant today
- Relevant circumstances adalah
- Chapter 11 decision making and relevant information
- Prefetching relevant priors
- Relevant contextual factors
- Other personality traits relevant to ob
- Relevant range managerial accounting
- Decision making and relevant information
- Vague questions examples
- Relevant cash flows definition
- Cash flow estimation
- Relevant cost in management accounting
- Iman sharif md
- Culturally relevant pedagogy
- Relevant theory
- Relevant costing
- Falling action of mulan
- What is litreature review
- Is the constitution still relevant
- Decision making and relevant information
- Decision making and relevant information
- Incremental cash flow example
- Relevant ones
- Relevant information for decision making
- Relevant evidence in writing
- Pengertian relevant cost
- Mohan lal grero
- Comparing images using the hausdorff distance
- Logos definition in literature
- Metadata lifecycle
- Web service metadata