Using robots.txt to remove single pages is not always effective because a) the page could still be indexed from crawling links from other pages, and b) it makes your 'hidden' pages very visible to anyone viewing the robots.txt file.
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.