A question pings me many times If a website disallow bot for specific directory like (
Disallow: /user/* ) but I crawl a URL ( /user/someone) through submit tool so what will happen. Will this page crawl or not?
I forgot to tell that I am getting the do-follow links here. Will I get the link...
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.