robots Archives - Page 3 of 4 - Gravity Search Marketing

Ask the Experts: How Can I Improve my Google…

November 6, 2008
by Jennifer Grappone

Q: The listings for my website don’t look very good. The titles and descriptions are weak in the summaries captured by these search engines. I have tried everything: writing new descriptive text on the page, correcting the meta description and page title, but the SERPS are not responding to the changes. The titles and summaries always stay the same.

A: Here are two possible explanations for the problem you’re having:

(1) Your listings may be showing titles & descriptions culled from directory listings rather than your page HTML Title & Meta Description. To find out your site has a listing on Yahoo! and Open Directory, try searching within those directories. Open Directory can be found at www.dmoz.org, and Yahoo! directory can be found at http://search.yahoo.com/dir . If the listings you see in those directories match the titles & descriptions that have been bothering you, then you’re in luck – this is an easy fix. In order to stop Google & Yahoo! from showing directory titles & descriptions in search results, you simply add a tag to the <head> section of the page instructing them not to do so. The format of the tag is this:

<meta name=”robots” content=”noydir, noodp”>

The next time the robot visits your page, this change should go into effect and you’ll see your own titles & descriptions (or snippets from page content). For a full list of tags that you can use in the “robots” meta tag, see our page: How to Use the Robots Meta Tag.

(2) Another possibility is that the robots are coming to your site so rarely that you aren’t seeing your changes in place in a timely manner. This is possible if your site has no – or few – links pointing to it. To find out if this is your problem, you can do two things: look at the cached version of the page listing, by clicking on the small “cached” link that shows up in your Google listings. Here, you will see a date when Google last gathered the page. You can also find this information by signing up for Google Webmaster Tools and looking at the indexing statistics from inside the tool. If it turns out that your problem is a lack of visits, your best bet is to increase the number of links pointing to your site.

seo resources

How to Use the Robots Meta Tag

February 20, 2008
by Gradiva Couzin

The Robots meta tag has been around for ages, but some new features introduced in 2006 and 2007 have made it a lot more useful than it used to be. Here’s the 4-1-1.

Robots meta tag basics

The robots meta tag is a part of a web page that communicates information to search engine robots. The robots meta tag is written within the <head> of an HTML file and looks something like this:

<meta name=”robots” content=”noindex, nofollow, noodp, noydir, nosnippet”>

Here is what you can put inside the “content” area:

“noindex”– this will prevent search engines from indexing the page
“nofollow” — this will prevent search engines from following any link on the page
“noarchive” — this will prevent search engines from keeping a cached version of the page in search results
“nosnippet” — this will prevent a description from appearing on your site’s listing on search engine results pages
“noodp” — this will prevent search engines from displaying the Open Directory title or description on your site’s listing
“noydir” — this will prevent Yahoo! from displaying the Yahoo! Directory title or description on your sites listing
“none” – this is the equivalent of “noindex, nofollow”
“noimageindex” – this will prevent search engines from indexing images on the page

Excluding a page from indexing with NOINDEX

To exclude a page from indexing, use the following format: <meta name=”robots” content=”noindex”> .

This type of exclusion is an alternative to excluding with the robots.txt file. Differences between the two methods of exclusion can be seen in the following table: (note, this table applies to Google only).

Table: Google Robots Exclusion with the Robots Meta Tag

Feature	Robots.txt exclusion	Robots meta tag exclusion
Page accumulates PageRank?	yes	yes
Page listed in Google?	the URL may be displayed in listings	no
Page passes PageRank through outbound links?	no	yes!

Improving Search Engine Listings with NOODP and NOYDIR

Some websites are saddled with outdated titles and descriptions from the Yahoo! Directory or the Open Directory. This is a very simple problem to solve, using the “noodp” and “noydir” meta tags. These tags will prevent the directory titles and descriptions from displaying for your site. Instead, your own HTML titles will display, along with snippets that are generated from your page content (or, sometimes, your Meta Descripton tag).