Search Engine Optimization L Grewe 1 Why Search

Search Engine Optimization L. Grewe 1

Why Search Engines? n n An estimated 77 percent+ of Internet users employ search engines to find Web sites If you are not listed on Google or Yahoo!, you are invisible • It doesn’t matter how well known your brand is! • Visitors will not guess your site name but will go through Google or Yahoo! just to be safe 2

Objectives 1. 2. How search engines handle your content How to improve ranking of web pages on search results 3

Major Search Engines and Directories Search Engine Producers n Google n Yahoo! n Bing n Ask. com 4

Modified from Bruce Clay’s Search Engine Chart: http: //www. bruceclay. com/searchenginechart. pdf 5

Search Engine Optimization n Search Engine Optimization or SEO is the task of designing (or redesigning) you site (pages) to appear higher in the list of results from Search Engines. 6

Basics of Search Friendly Design 7

Basic SEO Concepts n n Keywords: Use text that targets your audience; and attracts search engines and links from other sites Navigation: Give users and crawlers easy access to content Design Elements: Make sure bells and whistles don’t undermine SEO efforts Page Rank: Develop a linking strategy to improve your site’s “popularity” 8

Things Search Engines Like n Text (Keyword) • Words and phrases that match what your target audience types into search engines n Site Navigation • Site navigation and URL structure that search crawlers can easily follow n Links to Your Site • How “popular” are you? 9

Basic Idea of how Web Discovered n n How does a Search Engine discover the web……. . Much is proprietary Actual algorithms can be complex but, need to be fast! Here is a peek into the concept…. 10

Web crawlers (and robots) n n n Software that travel through a website gathering information. Search engines use these automated SW robots to index a website. Web crawlers are mainly used to copy all the visited pages for later processing by a search engine that will index the downloaded pages to provide fast searches. Crawlers can also be used for automating maintenance tasks on a Web site, such as checking links or validating HTML code. Also, crawlers can be used to 11 gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for spam).

Other names for Web Crawlers n n Web crawlers are ants, automatic indexers, bots, [1] or Web spiders, [2] Web robots, [2] or— especially in the FOAF community—Web scutters[3]. How it works: • it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier 12 are recursively visited according to a set of policies.

What Information Do Search Engines Gather n n n No one know exactly –except each Search Engine Company People fear too much. Some known things: keywords, links to and from, content /word counts, images, contact info (email, etc. ), media and ? ? 13

Managing Page Content 14

Keyword Prominence o Search engines place “weight” on terms according to where they are used Primary Text n Title tag n Body text n Anchor text n Text near the top of the page n Headings and Emphasized Text Secondary Text o Alt text o Description tag o Domain name and URL elements 15

What About Metadata? n n n Metadata is not a factor in most web search engines Except for Title and META description tags, crawlers ignore other META tags Use metadata if important for your agency’s mission or enterprise systems (e. g. , CMS, site search engine) 16

Keywords in Text n n Choose one or two keywords or phases to optimize for each page Use language of your audience Do not over use - avoid keyword stuffing Keywords in links influence relevancy of the pages your linking to • No “click here” links 17

Keyword Selection Tools n Site’s Search Metrics n Online Advertising Tools • Yahoo!/Overture: http: //advertising. yahoo. com/ n (Yahoo! White paper , Case Study , Advertising Case Study ) • Google: n n https: //adwords. google. com/o/Targeting/Explorer? __u=10000 0&__c=100000&idea. Request. Type=KEYWORD_IDEAS#search. n one Keyword Density • http: //www. webjectives. com/keyword. htm • http: //www. keyworddensity. com/ n Google Webmaster Tools 18

Other Keyword Selection tools n n http: //www. wordtracker. com/ Not directly related to web and search engines • http: //thesaurus. reference. com/thesaur us/ 19

Keyword Selection n Must relate to your business / site n Use Tools to help n DO you choose popular keywords or more specific less popular (but, very targeted keywords)? n n Gardening or Butterfly Gardening ? ? ? Dig Deep and then select 20

Chose your Keyword now what? n n n Put keywords in your site prominently (think primary text) Link to other pages that feature this keyword / related information. Have annotated media related to this keyword(s). 21

Site Navigation 22

Linking Basics n Plan how pages link to each other • Pages will not rank well if your site does not have a navigation scheme n n n Navigation scheme must please users and search engines Create a crawler-friendly site map Avoid dangling pages 23

Creating Links n n n Keywords in links tell crawlers about the pages to which you are linking Keywords in links influence relevancy of the page to which you are linking Avoid “click here” links, instead create links like: Bad: Click here for more info on famous admirals. Good: Visit our naval history site for more info on famous admirals. 24

Site Navigation Schemes n Make a site with a clear hierarchy n Text Links • The most search engine friendly • Use for primary or secondary navigation • Every page should be reachable from a text link n n If using graphic navigation, use alt text Avoid Java. Script, unless you provide a secondary text navigation scheme 25

Problem Navigation Schemes n n n Poor HTML coding Image maps Frames Java. Script Dynamic Pages Flash 26

Help Crawlers Navigate n Create a Site Map • HTML page on your site with text links n Create a Sitemap • An XML file that lists your site’s URLs • Allows crawlers to analyzes information about your site’s architecture to improve crawling • Google: http: //www. google. com/webmasters/ • Yahoo: http: //siteexplorer. search. yahoo. com/ • Site Maps Protocol: http: //www. sitemaps. org/ 27

Site. Maps n n n The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. is (usually) an XML file that lists the URLs for a site. This allows search engines to crawl the site more intelligently. Sitemaps are a URL inclusion protocol and complement robots. txt, a URL exclusion protocol. Sitemaps are particularly beneficial on websites where: • some areas of the website are not available through the browsable interface, or • webmasters use rich Ajax, Silverlight, or Flash content that is not normally processed by search engines. 28

Sitemaps n Google Sitemaps • http: //www. electionsonline. us/sitemap. xml for example tells Google where to begin crawling even without a site <urlset> <url> <loc>http: //www. electionsonline. us/</loc> <changefreq>monthly</changefreq> </url> <loc>http: //www. electionsonline. us/sitemap. htm</loc> <changefreq>monthly</changefreq> <priority>1. 0</priority> </url> <loc> http: //www. electionsonline. us/products-and-prices/election. Services. htm </loc> <changefreq>monthly</changefreq> </url> …. . MORE <url> ‘s </urlset> 29

Sitemap XML protocol n Same for popular Search Engines: Google, Yahoo!, Bing 30

Robots Exclusion n n robots. txt File • Place in server’s root directory • Two elements: User-agent, Disallow • Example: is a convention to prevent cooperating web spiders/crawlers User-agent: * and other web robots from Disallow: /cgi-bin/ accessing all or part of a website which is otherwise publicly viewable. The standard Disallow: /scripts/ is unrelated to, but can be used in conjunction with, Sitemaps, a Disallow: /images/ robot inclusion standard for websites. Meta-Tag Robots Exclusion <head><meta name=“robots” content=“no index, nofollow”></head> 31

Managing Your Popularity 32

What is Popularity? n n A calculation assigned to a page based on the number of links into the page The more popular pages that link to you, the higher your popularity rank • • n An An inbound link is a vote for a page outbound link is a vote for a page you’re linking to All search engines have different popularity algorithms • Google’s algorithm is called Page Rank • Based on a scale from 0 -10 33

Popularity Factors n n Number and Popularity of Inbound Links Anchor Text • Others’ use of keywords in a link to your site n Popularity is assigned per page, not for the entire site • Popularity is not inherited • Need to deliberately link internal pages to pass on PR 34

Getting Links to Your Site n Get Listed in Yahoo! and DMOZ (www. dmoz. org – open directory project) • Impact on popularity may vary • Helps to get noticed by crawlers n n Establish reciprocal link arrangements with agencies covering similar topics Reach out to state and local agencies Syndicate your content Reach out to professional communities of interest 35

Links That Aren’t Links n n n Link has been created in a manner that search engines can’t understand Link actually points elsewhere, and not directly to your site Example: http: //www. site. gov/external. jsp? url=http: //www. noaa. gov/ • Link passes through a JSP program that logs the click and re-directs the browser to noaa. gov • Search engines read this as an internal link to Site. gov, not an outbound link to NOAA • Links from these sites will not help boost your search ranking 36

Design Considerations 37

General Considerations n Implement bells and whistles carefully • Use external Javascript and CSS to decrease page load time • Avoid using Frames • Flash only pages – include skip link and title and META description tag • Omit Session IDs when requestor is a search engine crawler n Ensure design team understands SEO concepts 38

Dynamic Pages n n Database-driven, created on the fly by asp, cfm, php, jsp or cgi scripts Dynamic sites are comprised of templates, but usually without original content When a page is viewed, the template loads the content from the database Parameters are added to the URL, which tells the template to load specific content 39

Dynamic Pages Example: http: //smithsonianstore. com/catalog/product. jsp? product. Id=1 4273&parent. Category. Id=3151&category. Id=3152 n n n URLs such as this are difficult for search engines to index because they do not know the parameters that define a unique page The more parameters, the less likely pages will be indexed A database may continually feed data, crashing your server and scaring off the crawler 40

Search Friendly Dynamic Pages n n n Modify URLs so they don’t look like dynamic pages, fewer parameters Use URL re-write trick using mod_rewrite (Apache) Create static HTML pages 41

Optimizing PDFs n n n Make sure PDFs contain actual text, not images of text Same rules for use of keywords and phrases apply Put the most important text in the title, headlines Minimize document size (> 100 K) Create optimized HTML pages for PDFs 42

Site Maintenance 43

Re-Designing Your Site n n n Plan BEFORE you embark on a re-design • Will the site architecture change? • Static to Dynamic? • How is the content changing? Add SEO review as an activity to your project plan Make sure your contractor has SEO skills Poor planning and execution can kill your search rankings Do not wait until after you re-designed! 44

Changes to Site Architecture n n n Try to keep the same filenames and directory structure when redesigning Recommended: Set up HTTP 301 redirects that point to the new site or pages Not Recommended: Add a meta-refresh tag to your page header. This won’t remove your original page from the search index 45

Removing Content n n Avoid using default 404 pages Custom 404 pages are more userfriendly Submitting all 404 URLs to the search engine helpful, but not essential Remember to change the HTTP status code on custom error pages from 200 to 404 46

Updating Content n n n Usually when you update a page on your web site it helps your site get a higher ranking Not only does frequent updating get you a higher ranking it will also assist search agents with finding your page as new or newly updated. So be sure to update your site as frequently as possible (but, only appropriately). 47

Case Studies 48

49

Case Study: Business. gov o Page created March ‘ 07 o Submitted to crawlers, listed in DMOZ (www. dmoz. org – open directory project) o 0 -78 inbound links Term: Advertising Regulations n #6 on Google n #8 on Yahoo n #2 on MSN n #1 on USA. gov 50

Case Study: Business. gov n n n Text Component: page is welloptimized for “advertising” and “regulations” Link Component: All text links Popularity Component: Inbound links are using title submitted to DMOZ – “advertising” and “regulations” 51

Case Study: NOAA Fisheries Term: Fisheries n #1 on USA. gov n #2 on MSN n #2 on Yahoo n #3 on Google Term: Fish n Doesn’t show up on the first page or subsequent pages 52

53

Case Study: NOAA Fisheries n n n Text Component: page is welloptimized for “fisheries” but not “fish” Link Component: “pull-down” menu may be a problem for some crawlers. Add search friendly navigation (e. g. , site map) Popularity Component: Lots of inbound links using “fisheries” (not “fish”) in anchor text 54

Next Steps 1. Visit Web. Content. gov Guide to Search Engines: http: //www. usa. gov/webcontent/technology/search. shtml 2. 3. 4. 5. Conduct an SEO best practices review Conduct keyword analysis and determine your rank Set success measures Use Webmaster Tools offered by the search engines, (e. g. Google Webmaster Tools, etc. ) Make SEO a routine in your web content management program 55

Resources and Recommendations 56

Recommended Reading n n n Battelle, John. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed our Culture. New York. Penguin, 2005. Kent, Peter. Search Engine Optimization for Dummies. Hoboken, NJ. Wiley, 2004. Moran, Mike, Hunt, Bill. Search Engine Marketing, Inc. Upper Saddle River, NJ. IBM Press, 2006. Thurow, Shari. Search Engine Visibility. Indianapolis, IN: New Riders, 2003. FIND YOU OWN MORE recent publications – search in library – ACM/ IEEE portals. 57

SEO Forums & Events o Industry Forums Webmaster World: http: //www. webmasterworld. com/ Search Engine Watch: http: //www. searchenginewatch. com/ Digital Point Forums: http: //forums. digitalpoint. com/ High Rankings: http: //www. highrankings. com/forum/ o Industry Events Search Engine Strategies: http: //www. searchenginestrategies. com/ 58

Now… n Using tools monitor your site Study your traffic reports to determine those keywords which drive the most traffic to your site. Then edit your site to ensure they are near the top of the target page and the target page is not overwhelmed by other, less relevant, content. n 59

Some Additional SOE tools/info n n n http: //www. seotoolset. com/tools/free_tools. html http: //www. seomoz. org/tools http: //seobook. com 60

The Next Step n Pay for it - Search Engine Marketing (SEM) n n Can be a “sponsored link” Data Asset Optimization (DAO) – manage your media /data (images, video for Search Engines n Youtube links or ? ? n n Social Media n n Twitter, Facebook, My. Space and more Mobile 61

The Next Step Continued n Press Releases (paid for on external sites) PR web sites that specialize in online press releases can n help in taking the press releases outside of your own site and n onto various web sites of relevance. n i. e. http: //prweb. com n 62