googlebot crawl/index/cache

captchadreams

Power Member
Joined
Sep 19, 2008
Messages
509
Reaction score
133
I have a question. I got into an argument about the googlebot and what it can actually index and can't.

another person said googlebot could still crawl and index private pages on websites and indexes them.

to prove my point I posted a google cache which is basically just a screenshot of when the googlebot crawled that specific page and the cache said "only members can view this page". since googlebot isn't a member of that site.. it can't view all the threads in otherwords.

he said that googlebot could still crawl and index all the private pages it just didn't put them in their cache.. he knew this because he's had private sites where googlebot crawled.

I don't know as much about SEO, as I get my traffic from blackhatting social networks.. so I could be wrong.. if I am please explain how this works to me.

my understanding is basically if you can't view a page, thread, profile, whatever when you're logged out.. google isn't going to be able to see it either.

thanks.
 
Last edited:
Obviously google can't crawl a login require page,only public pages get crawl and indexed by googlebot .

"Google Site Search can't provide search results for pages that require passwords."
 
I thought it was pretty self explanatory as well. but wanted to make 100% sure.. so thanks for responses.
 
Back
Top