Ryerson University Library and Archives Searching the Deep

Ryerson University Library and Archives Searching the Deep Web Winter 2012

Virtual Parking Lot If you should have questions that are either too time consuming, theoretical or technical in nature to be addressed in this introductory session, then e-mail your question to Jay Wolofsky jwolofsk@ryerson. ca … the answer to your question(s) will be shared with the group.

The Deep Web is currently 400 to 500 times larger than the commonly defined Surface Web or WWW (7, 500 terabytes of information compared to 19 terabytes of information in the Surface Web and is growing exponentially

Deep Web/Surface Web The Deep Web (a. k. a. ) the Invisible Web contains high quality information not accessible from conventional search engines such as Google

Deep Web/Surface Web Structured information contained in research databases cannot be accessed from the Surface Web

Deep Web/Surface Web The real problem is the spidering and crawling technology used by conventional search engines that return links based on popularity, not content Surface Web search results are ranked by the Frequency documents link to each other (page rank) The first results are those that have had the most references by other documents, and not necessarily the most relevant or recent Information or content

Federated Search Engines Federated search engines execute simultaneous real time search of the Deep Web using sophisticated software “connectors” The results are collated and presented back to the user in a unified format

Federated Search Engines One type, a ‘web spider variant’ crawls information from as many databases as possible creating a giant uniform index, e. g. Google Scholar A more advanced type searches across each database’s own indexing AND crawls information, e. g. , Biznar, Mednar, Deep. Dyve

Federated Search Engines There are 3 general types: The first type searches across each database using its own indexing The second type ‘web spider’ crawls Information from as many databases as possible creating a giant uniform index, e. g. Google Scholar, Open. DOAR The third type searches across each database’s own indexing AND crawls information, e. g. Biznar, Mednar, Deep. Dyve …

Accessing Deep Web Content Biznar (Business) Deep. Dyve (Multidisciplinary) E-Print Network (Science and Technology) Google Scholar (Multidisciplinary) Highbeam (Multidisciplinary) High. Wire (Multidisciplinary) Mednar (Medicine) Meta. Press (Multidisciplinary) Open. DOAR (Multidisciplinary) Science. gov (Science and Technology) Scirus (Science and Technology) Social Science Research Network (Social Sciences) World Wide Science (Science and Technology)