Ask the Experts: How Can I Count the Number of Pages On My Site? (Updated)

by Gradiva Couzin on December 17, 2013

Q: Hello! I am a consultant in web analytics, and I’m looking for a tool that simply counts how many pages the website has, so I can calculate the inclusion ratio, or the percentage of pages indexed in the search engine. It seems like such a basic thing, yet I’ve been literally searching for hours and can’t find what I’m looking for. Can you help?

A: What a great question.  Most people just do a site: search in Google to find the number of pages on their website (for example, you could search for <site:www.yourseoplan.com> and look at the number of pages returned in Google – currently 282 pages).  But if you want to find out what portion of your total pages have been indexed by Google, you’ll need another way of counting the total number of pages on your site so you can compare it against Google’s index count.

We suggest using XML sitemaps along with Google Webmaster Tools in order to calculate the inclusion ratio of pages indexed in Google.  If you cannot create an XML sitemap internally, you may want to look into using one of the sitemap tools that are available, below are some resources:

When you have a sitemap created, sign into Google Webmaster tools and follow the steps below (if you don’t have an account, create one!):

  • When logged into Webmaster Tools, go to the Sitemaps section (Crawl > Sitemaps).
  • From here, add the sitemap that you created for your website.
  • If all goes well, Google will show the URLs you have submitted (see below):

sitemap indexing

  • Google does not show Indexed count yet, but give it a day (or so) and come back and Google will now give you the pages it has included in the index.

With the Submitted and Index counts, you can then calculate an inclusion ratio. See image below:

Sitemap 2

For larger sites, you may want to break up sitemaps by category to provide better diagnosis for any potential indexing problems your site may have. These sitemap categories can then be included into a sitemap index file, and you can submit just that sitemap index file to Google Webmaster Tools. More information on a sitemap index file can be found here.

When you submit a sitemap index file to Google Webmaster Tools, you’ll get an overall number of Submitted and Indexed pages, and you can click down to the individual sitemaps to get a sense of the category index rates.

Share/Bookmark

We’ve been helping customers improve their search engine ranks and conversion rates for over a decade. We consult on SEO and social media for major brands, one-person shops, and everything in between. Get in touch to find out how we can help you!

{ 20 comments… read them below or add one }

luxdesign28 February 17, 2010 at 9:44 pm

I did not understand how to instal this tools – SiteCrawler. Can you help me ? Thank you !

gradiva February 17, 2010 at 10:35 pm

instructions for downloading & installing the software can be found on the creator’s website, here: http://www.lightheadsw.com/sitecrawler/

(note that we have no affiliation with this software).

Another spidering tool that is available as of earlier in 2009 is the Microsoft IIS SEO Toolkit. Read more about it here: http://weblogs.asp.net/scottgu/archive/2009/06/03/iis-search-engine-optimization-toolkit.aspx. Although this tool must be run on Microsoft Windows Server, your website does not need to be hosted on Windows for it to be spidered.

joe March 7, 2010 at 9:03 am

Many times i have SEO questions in my head, i found your blog on Google and nowhere else. And your articles have never disappointed me once like this one also. However, could you please explain a little more about “using a trend as your metric” Why is it better? And how do i gather the trend info. exactly? Please. (don’t worry i’m not your competitor, my SEO firm is in Thailand) Thanksss

Gradiva Couzin March 8, 2010 at 11:43 am

Hi Joe,
When an exact value is not available for a metric, you can watch how it’s trending, rather than the absolute value. In this example, you could monitor whether the number of pages on your website showing up in the search engines is increasing, decreasing, or flat. Watching this trend might allow you to catch indexing problems early, or understand when efforts you’re making are paying off!

nitin jain gurgaon March 9, 2010 at 11:32 am

I use Xenu Link Sleuth for small sites, I copy all results in excel file and sort/delete. Its a bit lengthy way but it works for me for small sites

vijendra singhal April 5, 2011 at 10:07 am

i want to know,how to count static pages of a website?Thanks in advance.

Fernando April 25, 2011 at 9:22 am

Hi there! Thanks for this Q/A forum, this was a particular interesting one. I was asking myself this same question when I realized that Google and Yahoo! indexes where substantially different. To understand how bad was the situation I needed to now the number of total pages of the site to extract an indexation rate and start working on optimization. The new problem is, what am I going to do identify the pages that are not indexed?

togotutor May 11, 2011 at 4:23 pm

I am great fan of xml-sitemaps and have been using for my site http://www.togotutor.com for almost a year now. The only problem I am facing here is with the sitemap for vbulletin forum, which is really complicated to generate and the calender.php takes all the primary url`s during the crawl. So it`s useless If you are not able to aford that seo tool which costs almost 100 bucks.

Vishal Bansal September 5, 2011 at 1:05 am

There is another free tool Gsitecrawler with which you can count the no. of live pages in a website.

M October 21, 2011 at 5:21 pm

Thank you for the crawler info. Google is particularly picky when it comes to site maps. Thanks.

Anoop December 27, 2011 at 3:32 am

Why can’t you give site: yoursitename.com in google search. Go till the last page and this give the no pof pages that google covered or approx the no of pages

Sujeet February 16, 2012 at 9:25 pm

Use the online sitemap generator and it will count your pages and broken links also. You can than fix the broken links easily than.

[name removed] April 10, 2012 at 12:39 am

Yes, The XML Sitemap has a very good tool for counting the web pages, Thanks for informative sharing to all.

trev November 14, 2012 at 6:56 am

The easiest way to check in google is to type

site:www.( domain of your site).com

example
site:www.msn.com

The results google returns are all the pages indexed by google under that domain. Take a look at the url of the displayed page and you will see they all start with www.( domain of your site).com…..etc

Gradiva Couzin November 14, 2012 at 9:35 am

hi Trev,
Using the site: search on Google is a common way that people count the number of pages indexed on their site. However, the person asking the question wanted to know how many total pages on their site exist so he could *compare* that number with Google’s indexed number. So, telling him to use Google’s indexed number wouldn’t have been a very helpful answer :)

Chandrashekar BCS December 9, 2012 at 9:55 pm

From past 3years, I am using http://www.xml-sitemaps.com to find number of pages in a website..

zahra February 5, 2013 at 1:16 am

hi
I want to find the number of pages in a website and I went to this site http://www.xml-sitemaps.com but I could not find what I need. please help me
thanks

Thomas March 26, 2013 at 9:30 am

One little trick I use to quickly count the number of pages from a competitor website. I go to their sitemap usually http://www.competitor.com/sitemap.xml then I use the search tool of my browser (Ctrl+F) and search for . It gives me the count of occurences (Chrome does at least).

Of course, the sitemap has to be up to date.

John Marshall November 27, 2013 at 11:00 am

Simpler idea is just to type:

site:www.website.com

into Google. This will show you the number of results, and thus number of pages that Google has indexed.
e.g. site:http://www.yourseoplan.com/ returns 305 results

Possibly not entirely accurate as you may have some broken links or unindexed pages, but I’d say it’s pretty close.

Gradiva Couzin November 27, 2013 at 12:04 pm

Hi John,

Using the site: search on Google is a common way that people count the number of pages indexed on their site. However, the person asking the question wanted to know how many total pages on their site exist so he could *compare* that number with Google’s indexed number. So, we looked for alternative ways to find this number.

Thanks,
Gradiva

Leave a Comment

Previous post:

Next post: