Best Way to Index Pages in a Site

yasin10

Junior Member
Joined
Jan 6, 2010
Messages
146
Reaction score
30
Hi

What the Best Way to Index Pages from a site.?

The Site is already Indexed in Google But some of the pages are not :eek:

How Can i Index my unindex pages in the site?

Thanks.
 
i have the same problem as you.. My privacy policy and about us page is indexed and some of my main pages are not!!
 
Submit your sitemap to search engines and ping them... You can use your webmaster section for it... Bookmarking is also another good option for fast indexing... You can also use in-linking to get it done...
 
Get some high PR links to those pages and it will be indexed. Ping that page you get a link on and it will be almost instant. I use one website and it took 8 minutes to index a new website.
 
Also you can try to post the links in some forums that the links are ****************
 
I don't think I can post links yet, so check out:
google webmaster tools
bing webmaster tools
yahoo site explorer

Add your sitemaps, check out if you have any issues, add your RSS feeds, etc.
 
Place all the internal links in some directories or blog or forums.

also submit sitemap in webmasters tool!
 
I give 75% odds that adding canonical tags to every page fixes this problem.

25% that your site is too slow and googlebot slows down to not be a denial of service.

Googlebot quits when the dupe to non-dupe ratio favors dupes. This makes googlebot think it has most of the content and its just in infinite loops on your site.

The other way googlebot quits is when the crawl efficiency becomes too low over a period of time. Page response times are vital... A really good performance benchmark for your server is the difference and % difference of a page's load time versus a static html page load time on the same server. This makes relative adjustments for your location's bandwidth as well as your server's bandwidth. This ratio is what googlebot sees when your server is not under load... if you can find a load testing tool configure it for 11 simultaneous connections and crawl the site... This is twice max googlebot crawl speed, but lets assume you have users other than googlebot... this load will simulate peak volumes of about 250K visitors per day. If your site gets too slow... googlebot removes crawl speed options from webmaster tools and doesn't give them back until you make your site faster.

With static HTML and canonical tags I was able to get googlebot to visit 300K pages in under 5 weeks. Google estimated I had 750K pages but their math is wrong. I also make reports when indexing a new site to show me which pages still haven't been visited by googlebot yet according to the referer logs. This helps me find and fix any bugs or quirks in site design that prevent googlebot from visiting everypage... example... once I found that having "items per page" at the top of my category pages short-circuited gbot's crawl of my categories... Making them nofollow doesn't stop googlebot. I moved the links to the bottom of the page and kept them visually in the same place with css and googlebot started crawling whole categories without interruption.


Note: the bastard stepchild MSNBot does not use compression and will infinite loop your site for hours at a time... If googlebot visits your site while MSNBot is DOS'ing it then googlebot will assume your site is fucked and leaves. If you can't fix page load times you will be far better off blocking MSNBot from robots.txt
 
Last edited:
bookmark your site or links with diggs...diggs can help your site to get index fast...some people said few minutes...my experience is less than one day...if lucky less than 1 hour...

the important thins is bookmark it...settle...
 
If you're site is not yet index by Google try to apply some link building strategy. You can do either social bookmarking submission, profile linking and other link building strategy. Also, if you want that your sub-pages will be index as well, during promotion be sure to include the external pages as well.
 
Use ScrapeBox and make 50-100 backlinks to each page
 
I can't remember the URL but there is a UK site that will ping your site to help the search engines find it. It was free when I used it.
 
Back
Top