bizroot
Supreme Member
- Feb 25, 2008
- 1,346
- 926
I am stuck with my robots.txt...
One of my site is facing problem... Google is indexing txt and print version of my articles also... I tried to block them from robots.txt but still same problem... I am unable to block bots from crawling and indexing them... Site got 1600+ unique article and now in webmaster I am seeing tons of duplication issue... It's affecting my overall ranking for this site...
Anybody here using Vivvo CMS can help me out... How you are blocking bots to index txt and print version of articles...
My robots.txt
One of my site is facing problem... Google is indexing txt and print version of my articles also... I tried to block them from robots.txt but still same problem... I am unable to block bots from crawling and indexing them... Site got 1600+ unique article and now in webmaster I am seeing tons of duplication issue... It's affecting my overall ranking for this site...
Anybody here using Vivvo CMS can help me out... How you are blocking bots to index txt and print version of articles...
My robots.txt
Code:
User-agent: *
Disallow: /cgi-bin/
Disallow: /*.txt$
Disallow: /*.html?print$
Disallow: /admin/
Disallow: /cache/
Disallow: /feed/