I have a site with 60K+ pages and am looking to EXclude URLs from my sitemap, or prevent Google indexing, if the URLs contain certain keywords. Using disallow on robots.txt doesn't work because the URLs I'm looking to exclude are not in a particular directory. Any thoughts much appreciated.
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.