I have a question. I got into an argument about the googlebot and what it can actually index and can't. another person said googlebot could still crawl and index private pages on websites and indexes them. to prove my point I posted a google cache which is basically just a screenshot of when the googlebot crawled that specific page and the cache said "only members can view this page". since googlebot isn't a member of that site.. it can't view all the threads in otherwords. he said that googlebot could still crawl and index all the private pages it just didn't put them in their cache.. he knew this because he's had private sites where googlebot crawled. I don't know as much about SEO, as I get my traffic from blackhatting social networks.. so I could be wrong.. if I am please explain how this works to me. my understanding is basically if you can't view a page, thread, profile, whatever when you're logged out.. google isn't going to be able to see it either. thanks.
No that's impossible, you friend is wrong. How can Google get into a page that requires a user name and password? And Google says it cant here: http://support.google.com/customsea...+answer&cbid=qmrlvyu3q82j&answer=72365&src=cb
Obviously google can't crawl a login require page,only public pages get crawl and indexed by googlebot . "Google Site Search can't provide search results for pages that require passwords."
oh ffs. seriously, this is what happens when people that haven't figured out how to use google webmaster tools try to do seo http://support.google.com/webmasters/bin/answer.py?hl=en&answer=158587 use the fetch as googlebot function and see what googlebot sees in realtime protip: use fetch as googlebot to force google to index a page right then and there
I thought it was pretty self explanatory as well. but wanted to make 100% sure.. so thanks for responses.
