captchadreams
Power Member
- Sep 19, 2008
- 509
- 133
I have a question. I got into an argument about the googlebot and what it can actually index and can't.
another person said googlebot could still crawl and index private pages on websites and indexes them.
to prove my point I posted a google cache which is basically just a screenshot of when the googlebot crawled that specific page and the cache said "only members can view this page". since googlebot isn't a member of that site.. it can't view all the threads in otherwords.
he said that googlebot could still crawl and index all the private pages it just didn't put them in their cache.. he knew this because he's had private sites where googlebot crawled.
I don't know as much about SEO, as I get my traffic from blackhatting social networks.. so I could be wrong.. if I am please explain how this works to me.
my understanding is basically if you can't view a page, thread, profile, whatever when you're logged out.. google isn't going to be able to see it either.
thanks.
another person said googlebot could still crawl and index private pages on websites and indexes them.
to prove my point I posted a google cache which is basically just a screenshot of when the googlebot crawled that specific page and the cache said "only members can view this page". since googlebot isn't a member of that site.. it can't view all the threads in otherwords.
he said that googlebot could still crawl and index all the private pages it just didn't put them in their cache.. he knew this because he's had private sites where googlebot crawled.
I don't know as much about SEO, as I get my traffic from blackhatting social networks.. so I could be wrong.. if I am please explain how this works to me.
my understanding is basically if you can't view a page, thread, profile, whatever when you're logged out.. google isn't going to be able to see it either.
thanks.
Last edited: