question about deindexing

rap8557

Banned - Selling via PM
Joined
Sep 15, 2008
Messages
2,000
Reaction score
631
Im planning to set up a buy and sell site.. now after purchasing the product the thank you page will appear with the download link right?!

my question is how can i hide the thanks you page from search engine?
i mean i dont want this page to be indexed how can i do this?

thanks
 
Im planning to set up a buy and sell site.. now after purchasing the product the thank you page will appear with the download link right?!

my question is how can i hide the thanks you page from search engine?
i mean i dont want this page to be indexed how can i do this?

thanks

Create a robots.txt file. Put this in it:
Code:
User-agent: *
Disallow: /path/to/your/thankyou/page.html

Save it and upload it to your root web directory. If you want to double check and make sure, Google has a tool in their Webmaster Tools services which will parse robots.txt and let you know where they will and will not go.

For more info, go here:
Code:
http://www.robotstxt.org/robotstxt.html
 
Last edited:
how about yahoo, aol and other search engine?
 
i think there is a vulnerability that contents of robots.txt file can be read as normal txt file.
correct me if i m wrong. i m not expert in such things.
 
well, sorry for double post(can't edit the one above) but what i meant is this:

Code:
http://www.6figurefreelancer.com/robots.txt

;)
 
Using a robots.txt file is very weak
Many bots will bypass it
.htaccess password protect a folder
Then email out the password
Do this right and at the same time make yourself a nice long list of email addresses that spend online
 
If you put an entry in robots.txt saying don't access a page or directory, then well behaved robots will not spider it, BUT, you're telling the rest of the world here's something interesting please don't read it.

Use some form of password protection.
 
*zap* said:
Using a robots.txt file is very weak
Many bots will bypass it

If you put an entry in robots.txt saying don't access a page or directory, then well behaved robots will not spider it, BUT, you're telling the rest of the world here's something interesting please don't read it.

Use some form of password protection.
Very good points.

That said, there are a LOT Of reasons why you'd want to have public content on your site that you'd only be concerned about major search engine robots finding. I've *never* heard of an instance where a Google or Yahoo bot hasn't followed robots.txt. I'm not saying they don't fake UA and all of that good stuff, but a robots.txt file will keep content off the index and out of Archive.org.

I don't know why I didn't think of it before, rap8557, but if you aren't too technically adept, you might want to look at a company like E-junkie to handle your files. Very easy to setup and not expensive at all. I think their first service level is only $5.00 a month. It's worth it and you can use either Google Checkout or PayPal as your payment processor. It's really easy to setup affiliate systems through them too.
 
Do it like this...

1. Use a robot.txt , like Oldhat's suggested.

2. Use a login/membership management script for your site to password protect it with sessions and redirects.

3. On the page leading into the thank you page, use an individual " nofollow " code around download page code to protect it.


Doing all of this will protect your page from being hit by anything short of a professional hacker handing you your server on a silver platter. If they are that desperate to steal your ebook, they have too much time on their hands.

Hope that helps


Im planning to set up a buy and sell site.. now after purchasing the product the thank you page will appear with the download link right?!

my question is how can i hide the thanks you page from search engine?
i mean i dont want this page to be indexed how can i do this?

thanks
 
Back
Top