Skip to content
Gravity Search Marketing
  • Why Gravity
  • Services
  • Blog
  • Contact Us

Our Favorite Spider Emulators for SEO (2015)

  • March 6, 2015September 17, 2017
  • by Gradiva Couzin

Q:  In your book you recommended using a spider emulator to see what a webpage looks like to search engine robots. What tool do you use when you’re doing this?

A: These days, we use three different methods for reviewing a site with the “spiders-eye view.” Here they are in order of simplest to most complicated:

* Rex Swains HTTP viewer at http://www.rexswain.com/httpview.html.  Type in any URL and see what a typical client (browser or spider will see).  This is a great tool for a quick and easy look at a page.

* Fetch as Googlebot. This only works for sites that you have a Google Webmaster Tools account for (so it can’t be used for competitive review, only for your own site).  Sign up for Google Webmaster Tools by following the instructions here: https://www.google.com/webmasters/tools.  Once you have your website verified, click Crawl > Fetch as Google and enter the URL of the page that you want to see.  This method has the advantage that it is an accurate rendition of what Google sees, according to Google itself.

* Microsoft’s IIS SEO Toolkit. This is useful when you want to crawl an entire site, and this is what we use for our site technical audits. It is a free crawler, requiring a PC platform to run: http://www.iis.net/downloads/microsoft/search-engine-optimization-toolkit

Share this:

  • Click to share on Twitter (Opens in new window)
  • Click to share on Facebook (Opens in new window)

Related

Why You Should Add Captions and Transcripts to Your YouTube Videos
Should I Pay for a Press Release Distribution Service?
Who We Are
Gravity Search Marketing was formed in 2006 as a partnership between Jennifer Grappone and Gradiva Couzin. Gravity’s clients include Fortune 500 companies, global entertainment brands, niche B2Bs, large and small retailers, and non-profits.
As SEO industry veterans, Couzin and Grappone co-wrote Search Engine Optimization: An Hour a Day (Wiley, 2006, 2008, 2011) and Five Stars: Putting Online Reviews to Work for Your Business (Wiley, 2014), and enjoy sharing their expertise in speaking engagements and press interviews. 
Gravity's small, talented team has expertise in analytics, paid search, social media and technical SEO. Gravity team members are located throughout California, in Los Angeles, Sacramento, and the San Francisco Bay Area.

Get in Touch

Get An Effective SEO Strategy

Most of our business comes through word of mouth from happy customers. We work with clients who have what it takes to make the project a success: intelligence, openness to new ideas, a commitment to communicating with us regularly, and a workflow that allows us to work with you effectively.

Contact Us
  • Home
  • Why Gravity is Different
  • Services
  • Blog
  • Books We’ve Written
  • Contact Us

Women-Owned Small Business (WOSB)

Gravity is an SBA-Certified WOSB

NAICS Codes:

  • 541613 Marketing Consulting Services
  • 541810 Advertising Agencies
  • 541820 Public relations agencies
  • 541990 Other Professional Services
  • 611430 Professional and Management Training
Gravity Search Marketing LLC - A Full-Service SEO Company
Los Angeles • San Francisco
Copyright © 2020 All Rights Reserved
Theme by Colorlib Powered by WordPress