Peter Manson
Newbie
- Joined
- Jan 3, 2018
- Messages
- 6
- Reaction score
- 0
A question pings me many times If a website disallow bot for specific directory like (
Disallow: /user/* ) but I crawl a URL ( /user/someone) through submit tool so what will happen. Will this page crawl or not?
I forgot to tell that I am getting the do-follow links here. Will I get the link juice or not?
Disallow: /user/* ) but I crawl a URL ( /user/someone) through submit tool so what will happen. Will this page crawl or not?
I forgot to tell that I am getting the do-follow links here. Will I get the link juice or not?