I posted this here because of previous threads asking about the xml sitemap creator or other sitemap generator from different vendors that work with extremely large sites. The sitemap generator uses your traffic from bots and visitors to develop a sitemap by reading your apache logs. It creates the sitemap index file and multiple sitemaps files up to 50,000 urls and keep each file size under 10MB. For those interested you can read more about this sitemap on the official Google code page. http://code.google.com/p/googlesitemapgenerator/ This generator have to be installed via putty. It also pings major searchengines once sitemaps are generated.