Q: We have some private materials on our site, but we’re not able to use robots.txt or a robots meta tag to disallow the pages from indexing.  As long as we don’t link to the pages, and only send them out as links in emails, will Google have any way of indexing these pages?

A: As we’ve said many times, the only way to keep materials truly private online is to password-protect them.  Even if you don’t link to your pages, here are a few ways that your private URLs might find their way into Google’s index:

• People have reported seeing links from within Gmail messages spidered by Google, although this isn’t something that we have experimented with first-hand.  We do know that Google spiders Gmail content emails and uses “content extraction” in order to match advertisements, but there is no documentation of other uses of the spidered content.
• Private URLs can be seen by Google in other odd ways.  For example, if somebody clicks from your private page to another website, then your private URL would show up as a referrer in the server logs.  Some server logs are public, or find their way into the public realm, and that could expose the private URL.
• If someone visits the private URL while having the Google toolbar activated, then the URL could get collected and find its way into Google’s index.
• If the link is included in a listserv email (seemingly private), then that could be scraped and republished on the web.
• One of your authorized visitors could post a link within a forum post, on Facebook or Twitter, or somewhere else that he/she believes is private or semi-private, and that link could eventually be followed and indexed.

As you can see there are a lot of possible sources for leaks!

The best safeguard would be to password protect the individual pages, and your second-best approach is to deindex using the robots meta tag.  Without password protection, robots.txt or meta robots to prevent indexing, your next best line of defense is to watch for indexing and then do one of two things:

• remove any files that have been indexed and put them in a different URL; or
• place all of the private content in the same folder on your server, and then use Google Webmaster Tools and Bing Webmaster Tools to remove the files or folder from the indexes if/when it gets indexed.

These should give minimize your exposure in Google, but if your materials are truly confidential, you need password protection.  And without a doubt, ixnay on the social security umbers-nay!

We recently noticed that Google Australia is showing much more information-rich search results for certain searches than we’ve ever seen. We’re not seeing this type of result in US searches. Could this be a peek at future, more widespread Google results?

Update: 5/16/12 Google has given the sidebar a name – Knowledge Graph — and has officially announced it in this YouTube video.  Universal roll-out is likely imminent at this time.

We don’t know how long this feature has been in testing mode, and not everyone on our staff was able to trigger these results. Here are some examples:

A search for “the walking dead” results in a detail-rich sidebar, with cast, episodes, and more.

A search for “apple fruit” results in a sidebar containing scientific details.

(Admittedly, we’re giving the search engine a lot of context here. A search for just “apple” results in both Apple stores as well as some details about the fruit.)

A search for “Iron and Wine,” the name of a singer/songwriter, results in a list of songs.

Why Is This Important?

We think it’s worth watching as a possible indicator of future semantic/information-rich results on Google.com and worldwide. It’s been reported in The Wall Street Journal that, over the next several months, Google will be adding more facts to prominent positions in search results in addition to standard links. Mashable discussed this evolution, too, describing a the development of a resource called Google’s “knowledge graph.”

This Isn’t New, Just Bigger

Google behaving as an “answer engine” is not new. Currently, for example, if you search Google.com for “what is mitosis” you will see a definition, provided by Google, above the regular search results.

Similarly, if you search for “men in black 3 release date” you will see Google’s best guess for the movie release date, based on several sources.

But there’s a big difference between these definitions and the big semantic sidebar we’re seeing in Australian results. This is the first we’ve seen something so extensive.

What’s Driving These Semantic Sidebars?

The results are coming from a mix of sites, and the links within the sidebars go to various websites as well as Google search results. For example, if you click on a movie poster, you may go to a site such as freemovieposters.net, but if you click on the cast image, you may be taken to a Google search result for the actor name.

What Google does to gather this information is crawl websites and look for data that it can understand. This doesn’t always involve semantic tagging (such as schema.org microformatting) but semantic tagging would certainly make it easier for Google.

We tried to find a clear connection among the sites chosen for inclusion in the sidebar, but so far we haven’t identified one. The image results in the sidebar are not the top ranking Google image results, and we see no evidence that the featured sites are collectively using semantic markup such as Schema microformatting.

What Does This Mean for Site Owners?

If what we’re seeing on Google.com.au is a harbinger of a more widespread Google search result, site owners may feel a pinch in their click-through-rates. We believe Google’s goal in adding more semantic answers in its results is primarily to keep searchers on Google; you might say that Google is eliminating the need for searchers to click away from Google.

Some site owners have expressed concern about this as a growing trend for Google to steer visitors to its own properties, and we share this concern. There’s not much we can do but roll with these changes. But now we may have a new featured spot to vie for: how to get sites featured in these semantic sidebars is likely to be a topic of much research and speculation.

UPDATE: Matt McGee of Small Business Search Marketing pointed out that Google was previously testing this change in 11/2011. You can read the story on Search Engine Land.  Thanks, Matt!

Q: I have an e-commerce site and my products have recently started getting “Pinned” by people on Pinterest.  Sometimes a price shows up in the corner banner.  Why do these price banners sometimes happen?  Can I control them?

A: Price banners in Pinterest can be a nice feature for product pages, but site owners have limited control over them.  The price banner is automatically generated when a price including dollar ($) or pound (£) is included in the pin’s description.  For example, in the screenshot shown here, the description reads “$5 Face Painting”.   Pinterest grabs the $5 in the description to generate the  $5.00 price banner seen on the pin.

Here are a few notes and caveats:

• If  you have a “Pin it” button on your product pages, you can pre-fill the description so that it includes a price, formatted with a dollar sign.  Of course, Pinterest users may choose to overwrite your description, but the starter text you insert will often be included.
• If multiple prices are listed in the description, the last one in the list is the one utilized in the banner.  For example, if the description reads: “We charge only $50 compared to other people’s $65 fee!” the price shown in the banner will be $65.
• Many retailers have prices that change due to promotions or sales.  This could be a concern because Pinterest pins will be susceptible to displaying outdated price information.  There is unfortunately no way for a site owner to update Pinterest pins after they have been created by site visitors.  If a price changes, the retailer may wish to post a comment on product pins letting viewers know about the updated price.
• If you have serious concerns about prices being displayed on Pinterest – for example, if you charge different amounts to different people and really don’t want them shared – you can add a tag to your pages that will disallow Pinterest pins.  The tag is placed in the <head> of your page and reads as follows: <meta name=”pinterest” content=”nopin” />

One last helpful note:  any site owner can see Pinterest pins from their site at this URL: http://www.pinterest.com/source/example.com — just replace “example.com” with your own domain. This is a great way to keep an eye on your site’s Pinterest presence.

One of the more common emails we get is from people who think their website has disappeared from Google. If you’re in a similar situation, we have some ideas for you in this article.

If your previously high-ranking website seems to have fallen out of Google altogether, you are probably teetering somewhere between panic and desperation. As friendly consultants who care about your mental health, we want you to know this first:

Don’t panic! You probably didn’t get banned.
Give this a few hours and see if it works itself out.

We say that because it’s the most common scenario that we’ve seen.  But if you can’t bear to wait around to see if the problem will resolve itself, read on.

Special note: If you’re experiencing a major drop in your website traffic or ranks that occurred in late April 2012, this is probably due to Google’s Penguin and Panda 3.5 updates.  The Penguin update was intended by Google to address spammy SEO techniques.  The Panda 3.5 update continued Google’s ongoing adjustments to reduce the prominence of low-quality content in its index.  Here are some of our thoughts on recovering from a Panda update.   You can read more about Google’s Penguin update on SearchEngineLand.com.

The first thing you need to do is clarify the problem. Specifically, has your entire site been removed from Google, or has it just been knocked down in the rankings?  To figure this out, you can do a site search, by typing this into Google:

site: www.myurl.com

(replacing “myurl” with your own URL, of course).  If Google doesn’t display any pages from your site, it’s likely that your entire site has been dropped.  Another way to check is to take a unique string of text from your website, one that is unlikely to be on anyone else’s site, put it in quotes, and search for it on Google. If Google displays the page from your site that contains this text, then your site hasn’t been dropped from Google.

If your site is still in Google, but your ranks have taken a sudden and dramatic nosedive, skip down to the “What Happened to my Ranks?” section, below. If, after several strategic searches, you still can’t find your site on Google, then yes indeed, your site has been removed from the index, and it’s time to figure out why.

Unintentional Spamming?
It’s possible that your site was found guilty of spamming in Google’s eyes, and got banned. Websites can be banned for SEO spam techniques such as “white on white” text, blatant keyword stuffing, or tricky redirects. J.C. Penney’s website was famously banned temporarily for using a link-buying SEO technique.  It’s not likely that your website would be banned if you’ve never done anything to intentionally trick Google into thinking that your site deserves a higher rank than is appropriate.

If you’re not sure about your status as spammer, SEO expert Jill Whalen has a forum that discusses “SEO No-nos.” Read it to get some indication of whether your site was spamming inadvertently.

You’ll also want to check all your messages in Google Webmaster Tools.  There, Google will sometimes (but not always) alert website owners that their site has been penalized for spam.

Victim of Foul Play?
Whether you’re into conspiracy theories or not, you should entertain the idea that someone else did the spamming for you, causing your site’s disappearance from Google. We’ve seen it more than once: a site’s domain expires for a short time, and during that period, the pages are taken over by a domain squatter that displays not-so-nice content. Or, your site may have been hacked and stuffed with nuggets of spammy badness (this is especially common with outdated WordPress versions, so keep your WordPress up-to-date, yo!).

The fallout from one of these scenarios is something we describe in our case study of Providence Restaurant. (It’s worth noting, by the way, that even though that site temporarily displayed spam content, it suffered with terrible ranks, but wasn’t entirely dropped from Google’s index.)

Again, Google Webmaster Tools can help you diagnose this problem.  Check your messages to see if there have been any alerts describing malware or other problems on your site.

Another form of devious doings on the web is content scraping, which is when other websites steal content from your site and then republish it, often multiple times.  If this has happened to you, it is possible that Google banned your site along with the “bad guys” who copied you. Try doing some searches on other search engines, looking for strings of text from your website.  You just might find an unexpected match.

Redirects or Canonical Tags Gone Wrong?
We once had a client who inadvertently added  a canonical tag on every page of his site pointing to a different domain.  Google followed this instruction diligently and removed every page of his site from its index.  Yeowch!  The good news: once the errant canonical tag was removed, ranks and indexing were back where they belonged within a couple of weeks.

We have also seen examples of sites that redirect their pages to another domain using a 301 redirect, and then are surprised when their domain is dropped from Google.   A 301 redirect is like a permanent change-of-address for search engines.  If you redirect your entire domain to somewhere else, Google is not going to continue to display your domain in its search results.

Robots.txt or Robots Meta Tag Deindexing?
Have you double-checked that you didn’t accidentally deindex your entire site with the robots.txt file or robots meta tags?

Here’s the line you really don’t want to have in your robots.txt file:

User-agent: *
Disallow: /

And here is a tag that really shouldn’t be on every page of your site:

<meta name=”robots” content=”noindex”>

Go on… check it now.  We won’t judge you.

Server Outage?
Your server might have hiccupped just when Googlebot came to visit your site recently, but that shouldn’t cause your site to be dropped from the index.  We believe that Google will check a site at least a couple of times before dropping it.   But, if the site went down for an extended period (days, weeks), this could be an explanation.

Most likely, the bot will come back and rediscover your site, and you’ll find your site back where it belongs after that.  But how long will you have to wait?  It depends on how many links there are to your site and how often Googlebot visits it. While there’s no sure-fire way to trigger a visit from Googlebot, getting some new inbound links, refreshing your content a bit, and submitting to the free URL submittal page probably won’t hurt.  You can submit an XML Sitemap via Google Webmaster Tools, which has the nice fringe benefit of telling you how many of your URLs are indexed.

In the future, if your site has anticipated downtime, set a 503 server status so that search engines will know it’s just a temporary glitch.

Reinclusion Request
If you are certain your site has been penalized for spam or malware, you can submit to Google’s request for reinclusion page. Matt Cutts of Google has posted some information on how to file a reinclusion request. But before you file a request, be very, very sure that there’s nothing spammy on your site.

What Happened To My Ranks?
If Google hasn’t actually removed your site, but you just noticed a sudden and substantial drop in rank for your favorite keywords, there are several possible explanations:

• Google could have made an intentional change to its results or its algorithm.  Panda 3.5 and Penguin (late April 2012) are a couple examples of algorithm shifts that blew some sites out of the water.  SearchEngineLand.com is a good place to look into industry chatter about algorithm updates.   See also our post about recovering from Panda. Once the dust settles, we’ll write a post about the Penguin update.

• Google could be experiencing some temporary weirdness that makes the results different from what you’re used to. As we mentioned at the top of this article, we’ve seen this so many times with so many clients that this is probably the most likely scenario. In this case, just check your ranks again in a day or so . You might once again see that good rank you were used to.

• Your competitors could have gotten their acts together. OK, if your rank dropped by a few hundred spots, it’s probably not your competitors, as that would mean a lot of competitors getting their acts together at the same time!  But if you’re just being outranked by legitimate competitive activity, then it’s time for you to work on your site. And this work includes optimizing for, and tracking, additional keywords.

• Is there something funky going on with your homepage, but not other pages?  We’ve seen sites that have suffered big rank drops because their homepage had been deindexed due to redirect or canonical issues, and no other page had as much power as the homepage, so no other page on the domain was making it into those previous ranking spots.  If you’re seeing only a portion of your site in the index, run through the diagnostic steps above to see if you can figure out why.
• Is your website new?  Sometimes, websites show up in search results briefly after being launched, but then seem to drop dramatically out of the rankings after a couple of days, and are stuck in the depths for months.  It’s a phenomenon that some have called the “Google Sandbox.”  How to get out the sandbox? Get some inbound links to your site, and be patient.

• Is it possible that something has dramatically changed on your website? Say, you intended to update a single page but accidentally changed the page titles on every page of your site?  Don’t forget to check your site and make sure everything is in order.
• Could you be seeing different results because your personalization has changed?  We’ve known people who thought their ranks were phenomenal – only to be brought down to earth when they realized that the ranks they were seeing were personalized just for them by Google.   Read more about Google search personalization.

Hopefully we’ve given you some ideas about how to track down the source of your site’s disappearance from Google. Regardless of what you find, it’s always a good idea to make sure you’re offering lots of great content that makes people want to link to your site, and do some link building. In this way, you’ll have other sources of traffic that makes a Google problem like this less devastating.

Upon bouncing back to Google from a search result, you may see a notice allowing you to block that domain from future Google results.  Or you may see a notice encouraging you to +1 that domain.  It all depends on the amount of time you spent on the page.

Google is well aware of what listings searchers click after performing a Google search.  But did you know Google continues to care what a user does after clicking on a listing?  The search engine monitors whether the user clicks right back to Google, and even pays attention to the length of time the user spent on a site.

Here’s an example.  After searching for “eco-friendly down comforters” I clicked a page on the domain www.beddingdowncomforters.com, reviewed it briefly, and clicked the Back button to return to Google.  This is known as a G-bounce. When I landed back on the Google search results page, Google had added the message ” – Block all www.beddingdowncomforters.com results” to the listing:

Click on the “Block…”  link, and you’ll never be bothered by this website’s listings again.

Later, I was searching for “how to decant wine” and clicked on wineintro.com.   I spent a long time on the page and eventually clicked the Back button to return to Google.  Upon landing back on my Google results page, I saw a popup suggesting that I +1 the site on Google:

This experience got me wondering:  How long do I need to spend on a website before Google stops suggesting I block it, but instead suggests that I +1 it?  What is that magic length of G-bounce time that Google deems to represent a successful website experience?

I tested it out by G-bouncing after various lengths of time, and here’s what I found:

• 15 seconds — block
• 30 seconds — block
• 45 seconds — block
• 1 minute — block
• 1 minute 15 seconds — block
• 1 minute 30 seconds — block
• 2 minutes — no message
• 3 minutes — no message
• 4 minutes — no message
• 7 minutes — no message
• 7 minutes + 30 seconds – no message
• 7 minutes + 40 seconds -  +1 message
• 8 minutes – +1 message

So apparently, according to Google, the 1-1/2 minute mark is where a site goes from hateworthy to just OK, and at the 7-1/2 minute G-bounce mark, the site has clearly served you well and deserves a +1 for its trouble.

I anyone else seeing these same times?  Different ones?

UPDATE:  Bing and Google reps discuss the “dwell time” signal about 5 minutes into this discussion at SXSW: http://searchengineland.com/too-much-seo-google%E2%80%99s-working-on-an-%E2%80%9Cover-optimization%E2%80%9D-penalty-for-that-115627

When you’re launching a new app, naturally you want everyone in the world to see it!  But many of the app review sites you’ll find in Google are spammy or require a payment for submittal.  Try this list for free app review submittals.

Before you submit, do a little prep work.  You’ll need a promo code if your app is not free.  And your app will have a much better chance of getting reviews if you create a fantastic demo video.  That way reviewers don’t have to download your app to see just how fun and fantastic it is!

When you’re ready to get started, here are your marching orders:

11 Great App Review Sites to Submit Your App For a Free Review

• http://www.macworld.com/info/appsubmit.html
• http://appadvice.com/appnn/contact-info-developers/
• http://www.appcraver.com/contact/
• http://www.appolicious.com/
• http://www.appstoreapps.com/contact/
• http://www.tuaw.com/contact/appreviews
• http://www.iusethisapp.com/submit-app/
• http://www.theiphoneappreview.com/submit-your-app/
• http://www.slapapp.com/request-a-review
• http://www.appchatter.com/submit/app-for-review/
• http://forms.whatsoniphone.com/forms/woip_submitanapp

Readers, do you have any more to suggest?

Q: I just hired a developer for my small-budget site development project. Is there a Top 10 list of things I should insist on SEO-wise to be built into the site?

A: As we all know by now, SEO consists of several different endeavors:

• Technical SEO: ensuring a search-friendly and social-friendly site from a structural/technical standpoint
• Content/Editorial SEO: creating content that aligns with your target audience’s search behavior and keywords
• Link appeal and link-building

It’s obvious that your web developer’s work influences the technical side of SEO.  But structural issues and CMS capabilities will also have long-term effects on your ability to maximize your content/editorial SEO and link power.   Here are my top 10 developer requirements for SEO:

• Each page on the site must exist on a separate, unique, permanent URL that is indexable by search engine robots
• All pages on the site must be accessible by clicking standard HTML links, preferably no more than 3 clicks from the home page
• Text content on the site must be search-engine-indexable, standard, visible HTML text
• Search engine robots are like site visitors that do not accept cookies, do not fill out forms, do not log-in, and have limited javascript and CSS capabilities.  These robots should be able to read versions of each page that match as closely as possible what a human visitor would see.
• Every page on the site must have a unique, descriptive HTML title.   The CMS should allow the site owner to edit the HTML title separately from editing other elements of the page, such as the page heading.
• We recommend that every page on the site include a Facebook Like, Google +, and Tweet button.
• The CMS should allow the site owner to edit OG tags on each page.
• If there is any chance that identical pages will be seen via more than one URL (creating potential duplicate content in search engines), the CMS should allow the site owner to include/edit a canonical tag on each page.
• URLs should be meaningful, human-readable and include the page title or relevant keywords
• Images that are clickable should have ALT and TITLE tags that provide a text description of the destination page being clicked to.

Granted, every one of these won’t apply to every project, but for the majority of small budget jobs, this list will form a strong foundation on which you can build your SEO efforts.

And one last piece of advice: communicate like crazy and stay on ‘em.  If you don’t have an outside SEO vendor involved in the project, you will need to serve the role of SEO watchdog during development.

Q: I have an ecommerce site containing a useful library of product information that the manufacturer allows me to use on my site.  My site was hammered in Google’s Panda update, and I think it was because of this “duplicate” content.  What should I do now?  I don’t want to remove this library because it is useful for my customers.

A: Even though this library of content is genuinely helpful for your site visitors, it could be similar to the low-quality, “stub”, or content farm materials that were punished in Google’s Panda update.  To make matters worse, the product information library you’re describing is probably also present on many other sites as well.  These two factors are both likely to be viewed negatively in Google’s eyes.

To play it safe, the easiest thing you can do is to deindex all these pages using either the robots.txt file or a robots meta tag on the concerning pages.  This way, the materials can still help your website visitors, without putting your site at any risk for Panda issues.

If deindexing the content is not an option, then you should work on improving the content, with an eye toward improving the user engagement on these pages.  What can you do to make these pages more unique, more interesting, and more likely to be a satisfying experience for searchers who land on these pages?

Not interested in rewriting hundreds or thousands of pages? Here’s a hybrid approach you might try:

• Deindex any pages that have a 90% or higher bounce rate
• For pages that have better (lower) than 90% bounce rate, work on improving content.  Possible ways to improve content:
• manually add your own links to related materials elsewhere on your site.  Make them intriguing or attractive enough to encourage visitors to click on them; remember your goal is to engage visitors and prevent bounces.
• manually add commentary/analysis that makes these materials more useful.  Be sure this is unique and well written.
• Proofread.
• add a prominent Google +1, Facebook Like, and Tweet button
• In addition to these steps, you should also pursue other SEO efforts, especially link-building, in order to gain stature in the Post-Panda search world.

Like any SEO effort, approach your Panda recovery with an iterative mindset.  Make some changes, watch for results over a several-week period, and repeat.  And a final word of wisdom: Keep the user experience a priority, and you can’t go wrong.

Tongue twister, anyone?  Pagination canonicalization is one of the trickier challenges for the technical SEO set today.  We’ve been closely monitoring the hints and tips that occasionally drop from our favorite Googlers’ lips (sure, we all adore Matt Cutts, but let’s save a little love for JohnMu and Maile Ohye, shall we?).  Here’s a roundup of good advice we’ve heard, along with our analysis and resulting recommendations.

Pagination is commonly seen on ecommerce sites when a category contains more products than can be listed on each page:

The question often arises:  How should an SEO handle all of these pages?  Allowing all of these pages to be indexed separately in search engines means indexing a lot of duplicate or near-duplicate content on a website, which dilutes SEO power and creates an undesirable user experience, since nobody particularly wants to click through from Google to a deep paginated page.

Here is the wisdom we’ve gathered from our favorite Googlers on the subject:

JohnMu on paginated pages and the canonical tag (Feb, 2010):

Pagination: this is complicated, I personally would be careful when using with rel=canonical with paginated lists. The important part is that we should be able to find all products listed, so at the very least those lists should provide a default sort order where we can access (and index) all pages. Since this is somewhat difficult unless you really, really know what you are doing, I would personally avoid adding rel=canonical for these pages. One possible solution could be to use JavaScript for paginated lists with different sort orders, for example, that way you would have a single URL which lists all products.

Our interpretation: Google needs to be able to see all of these paginated pages so that it can click through and get to all the individual links (product pages) that are listed on all the pages.  JohnMu suggests creating a default page that contains all of the links, so that Google can get to them.   Trouble is, what if you have a thousand of these links?  Or even more.

At SMX West (March 2011), Maile Ohye noted that you shouldn’t use canonical tags for paginated pages. (We’ve heard her say this since 2010) Here are notes from Barry Schwartz (SEORoundtable) on this session:

Maile explained that since the results on pages 2, 3, 4, and 5 are different from page 1, you should not use the canonical tag here.
Not only that, if you do, Google may ignore it because Google uses methods to determine if the canonical tag command is actually something valid for that case. So if you canonical page 2 to page 1 and page 2 is not similar enough to page 1, Google may ignore your canonical tag.

Our interpretation:  Since paginated pages aren’t identical to each other, and they aren’t subsets of each other, they shouldn’t be canonicalized to each other.  Google is pretty smart and can probably figure out when it’s dealing with page=1, page=2, page=3, and Google may ignore your canonical tags in this case.

The one exception to the no canonical tags for pagination rule is when you have a “view all” page.  As Brian Ussery of SEO firm Nine by Blue notes:

If paginated content uses “view all” page if it loads well you can put rel canonical on those URLs.
If you don’t have a “view all” page 2 isn’t a subset so you can’t use rel canonical.

At the end of the day, our two favored pagination approaches are the following:

• noindex,follow
  Place a robots meta tag on all deep paginated pages instructing the robots to “noindex, follow” the page.  This will allow Google to visit the deep pages, follow links on these pages, but not index these pages.
  
• “view all” + canonical
  Create a “view all” page showing all of the links on your list.  Then, on all other paginated pages (including page 1), add a canonical tag identifying the “view all” page as the canonical form.   We think this solution is best used when the “view all” page is also the version of the page that is linked from elsewhere on the site.

UPDATE:  Google got its act together and created a better way to deal with pagination canonicalization.  Read all about it at Google’s webmaster blog.

Q: I’m confused. What are Schema.org tags and are these the same as semantic tags for rich snippets? Which ones should I use?

A:  Have you ever wondered how some websites’ listings in Google and Bing get those great-looking review stars, price ranges, and other enhancements, like the ones shown here?

This is all achieved through the magic of semantic tagging.  Semantic tagging allows website owners to put tags around their content to help search engines understand the meaning of that content.  Say a website owner put a <price> tag around the price of a product:  <price>14.99</price>.  That would help a search engine understand that it’s looking at the price of the product.  Then, the search engine can use this information to improve its listings for your site.

Semantic tags for rich snippets have been supported by Google for years, but the new Schema.org tagging expands the type of information that can be tagged, and is also supported by Bing.  Here’s a quick overview:

• Semantic tagging per Schema.org.   This was recently announced by Google and Bing and is remarkable for being supported by both of the rival search engines.   Because Schema.org tagging is still very new, we don’t know exactly how it will be used by Google and Bing, but we expect that it will be used to improve search engine listings with rich snippets, and could help with positioning in Shopping search.

• Semantic tagging for rich snippets using microformats or RDFa.   This is an older form of semantic tagging that has been supported by Google for at least a couple of years.  It uses a different format than Schema.org tagging but provides some of the same information.  It is more limited than Schema.org tagging and only provides a few pieces of information about a product, including name of product, price, availability, and the number of review stars.

Which one should you use?  We recommend Schema.org because it is fully supported by both major search engines, Google and Bing.

Are there any possible downsides to using semantic tagging?  Yes.  Semantic tagging could help search engines create content pages that give visitors the information they’re looking for without clicking through to your site. Many website owners already feel that Google News, Google Places and Google Shopping have intercepted potential visitors to their site.  Some content creators think search engines are becoming no better than content scrapers that steal content from their websites. Weigh this concern against the benefits of having enhanced listings on search engines.  As SEOs, we continue to lean toward doing everything we can to feed quality information to search engines, but a dose of healthy skepticism never hurt anyone.

Readers: are you using rich snippets?  Have you seen more traffic, or are you concerned about search engines scraping your content?