Q: Hello! I am a consultant in web analytics, and I’m looking for a tool that simply counts how many pages the website has, so I can calculate the inclusion ratio, or the percentage of pages indexed in the search engine. It seems like such a basic thing, yet I’ve been literally searching for hours and can’t find what I’m looking for. Can you help?
A: What a great question. Although we don’t know of a tool that is designed to do specifically what you’re describing, we suggest you try an XML Sitemap generator. These tools are designed to create, basically, a list of all the pages on a site, so you could get the total number as a fringe benefit.
For a small site under 500 pages, you could use the online version at http://www.xml-sitemaps.com/ . For larger sites, you can pay for the standalone software available on that site, or review other sitemap generators at: http://code.google.com/sm_thirdparty.html
Another option would be an application such as SiteCrawler, designed to download and crawl an entire website.