Skip to content
Gravity Search Marketing
  • Why Gravity
  • Services
  • Blog
  • Contact Us

Ask the Experts: How can I keep private materials…

  • April 18, 2012
  • by Gradiva Couzin

Q: We have some private materials on our site, but we’re not able to use robots.txt or a robots meta tag to disallow the pages from indexing.  As long as we don’t link to the pages, and only send them out as links in emails, will Google have any way of indexing these pages?

A: As we’ve said many times, the only way to keep materials truly private online is to password-protect them.  Even if you don’t link to your pages, here are a few ways that your private URLs might find their way into Google’s index:

  • People have reported seeing links from within Gmail messages spidered by Google, although this isn’t something that we have experimented with first-hand.  We do know that Google spiders Gmail content emails and uses “content extraction” in order to match advertisements, but there is no documentation of other uses of the spidered content.
  • Private URLs can be seen by Google in other odd ways.  For example, if somebody clicks from your private page to another website, then your private URL would show up as a referrer in the server logs.  Some server logs are public, or find their way into the public realm, and that could expose the private URL.
  • If someone visits the private URL while having the Google toolbar activated, then the URL could get collected and find its way into Google’s index.
  • If the link is included in a listserv email (seemingly private), then that could be scraped and republished on the web.
  • One of your authorized visitors could post a link within a forum post, on Facebook or Twitter, or somewhere else that he/she believes is private or semi-private, and that link could eventually be followed and indexed.

As you can see there are a lot of possible sources for leaks!

The best safeguard would be to password protect the individual pages, and your second-best approach is to deindex using the robots meta tag.  Without password protection, robots.txt or meta robots to prevent indexing, your next best line of defense is to watch for indexing and then do one of two things:

  • remove any files that have been indexed and put them in a different URL; or
  • place all of the private content in the same folder on your server, and then use Google Webmaster Tools and Bing Webmaster Tools to remove the files or folder from the indexes if/when it gets indexed.

These should give minimize your exposure in Google, but if your materials are truly confidential, you need password protection.  And without a doubt, ixnay on the social security umbers-nay!

 

Share this:

  • Click to share on Twitter (Opens in new window)
  • Click to share on Facebook (Opens in new window)

Related

Google Semantic Sidebar [“Knowledge Graph”] Spied in Australian Results
Ask the Experts: How Can I Get Video Thumbnails in Google? [updated]
Who We Are
Gravity Search Marketing was formed in 2006 as a partnership between Jennifer Grappone and Gradiva Couzin. Gravity’s clients include Fortune 500 companies, global entertainment brands, niche B2Bs, large and small retailers, and non-profits.
As SEO industry veterans, Couzin and Grappone co-wrote Search Engine Optimization: An Hour a Day (Wiley, 2006, 2008, 2011) and Five Stars: Putting Online Reviews to Work for Your Business (Wiley, 2014), and enjoy sharing their expertise in speaking engagements and press interviews. 
Gravity's small, talented team has expertise in analytics, paid search, social media and technical SEO. Gravity team members are located throughout California, in Los Angeles, Sacramento, and the San Francisco Bay Area.

Get in Touch

Get An Effective SEO Strategy

Most of our business comes through word of mouth from happy customers. We work with clients who have what it takes to make the project a success: intelligence, openness to new ideas, a commitment to communicating with us regularly, and a workflow that allows us to work with you effectively.

Contact Us
  • Home
  • Why Gravity is Different
  • Services
  • Blog
  • Books We’ve Written
  • Contact Us

Women-Owned Small Business (WOSB)

Gravity is an SBA-Certified WOSB

NAICS Codes:

  • 541613 Marketing Consulting Services
  • 541810 Advertising Agencies
  • 541820 Public relations agencies
  • 541990 Other Professional Services
  • 611430 Professional and Management Training
Gravity Search Marketing LLC - A Full-Service SEO Company
Los Angeles • San Francisco
Copyright © 2020 All Rights Reserved
Theme by Colorlib Powered by WordPress