Hide Pages from user, but show google (not cloaking)

[Bloghat]

I thought it would be nice, to have a forum which is crawled by google (many good working keywords in that niche) but is private if being browsed by a human.
So google knows all the private pages and has them in the serp, but when people click on it, they get a 'please register to see the page'.

Is this possible?

 

[dspiel]

Maybe having a sitemap with the urls you want google to index but have no way for the user to navigate to those pages unless they read the sitemap. I'm not 100% that would be sufficient though.

 

[Bloghat]

But if G indexes the patge (from sitemap) people will reach that site via the serp. So they would get in the same moment, they find the site on G.

 

[orbit]

I was just thinking about how to do this as well. Would like to know if there is a way.

 

[The Doctor]

what forum software are you using? I've seen lots of forums like what you're describing. Especially in the niche of providing expert help to people. There have been times when I visited a forum and the only way to see the content without paying or registering was to switch my user agent to Googlebot. Thats what you need. A plugin that can restrict certain areas to registered members or the googlebot useragent. Its also worth mentioning that it is possible to tell if someone is REALLY googlebot or not by not only looking at the useragent but also looking at the IP/dns of the visitor since googlebot only comes from certain IPs.

 

[ronegraT]

 

[Bloghat]

The forum is not there yet, I'm still at planning ahead. I guess I will be taking mybb - I fear simplePress (though easier to integrate into my wp-project) might have limits that get into my way later.
@Fumanchau, that's exactly what I am looking for, seen it myself and wondered how they're doing it.

 

[Bloghat]

@ronegrat: Since I will offer very useful (readable) stuff, I want to lock them out without registering. They don't need to pay to see it (though I've been told I could take money for it), but at least I want their email adress. It's not that important if they can post or not, it's about being able to read it...

 

[christoss1959]

I've seen forums like that. But note if you do this then the user can ask to see Google cached version and still see the page without registering (that is what I did!).
One of the forums that does this is (I am not sure how they do it but looking at their html code will help you):

www.hackforums.net

 

[Bloghat]

@christoss that would be enough, the info that I have would make them come back often and they would be soon annoyed to go back to search every minute instead of surfing the page naturally. Any insights, hwat I need to copy that behaviour?

 

[CyberSEO]

Maybe having a sitemap with the urls you want google to index but have no way for the user to navigate to those pages unless they read the sitemap. I'm not 100% that would be sufficient though.

If you want, I can write such a plugin for you. So only Google and other search engines you specify will be able to see the content, others will be redirected to the page you want (sign up or whatever).

 

[Bloghat]

@cyberseo, if people click on the serp link, they would get in (without navigating your page) what's the use of that? Ok, i guess for some pages that would work, but i have bunch of connected lists, so if you get into the page (via serp) you could surf the rst. So i need the permission management of a forum

 

[CyberSEO]

@cyberseo, if people click on the serp link, they would get in (without navigating your page) what's the use of that?

Nope, they wouldn't.

 

[Bloghat]

Ok, that would work then. Have yet to decide on a software. Found out simple press can only secure registering with math captchas, easy to bypass for a bot.
Would your solution work with any forum?

 

[Bloghat]

I asked on that mentioned forum for their solution and they said mybb would allow such behaviour right out of the box. You would have to set different permissions for guests and spiders.