Headlines News
Newbie
- Jan 11, 2017
- 25
- 0
Hello friends- Is it necessary to use Robots.txt file? It can be affect on website ranking in search engine.
Is it necessary to use Robots.txt file?
- Thread starter Headlines News
- Start date
markmueller
Newbie
- Feb 5, 2018
- 42
- 16
It is important, because there you can define how search robots like google can "work" on your website. For example you can define there, that google only will crawl some specific pages of your website. Or no page, or only pictures, or whatever.
So you have some power there over search engines
So you have some power there over search engines
MuayThai
Elite Member
- Aug 25, 2015
- 3,944
- 1,137
Robots.txt is the door of a website. Now think what type of house you wanna build? With door or without?
It's absolutely NOT a door.
WPninja
Regular Member
- Apr 25, 2017
- 354
- 90
A house can be access without a door mate, but when there's a door, obviously this is the first place that bot come at first
- Aug 21, 2016
- 3,588
- 831
Not certainly, That's for your own preference, Everything have certain limit,
If you use too much, Definitely it's affect your search engine ranking.
If you use too much, Definitely it's affect your search engine ranking.
- Aug 21, 2016
- 3,588
- 831
Without using "Robots.txt" can't able to create the website, This your theory right?,
Definitely wrong, its help to hide your sensitive data in google eyes. But it's not necessary.
Definitely wrong, its help to hide your sensitive data in google eyes. But it's not necessary.
A house can be access without a door mate, but when there's a door, obviously this is the first place that bot come at first
Robots.txt is the door of a website. Now think what type of house you wanna build? With door or without?
WPninja
Regular Member
- Apr 25, 2017
- 354
- 90
You can't get my point then. I said you can build a house without a door, it already means that you can build a site without robots file, but when there's a robot file, it's the first place where google or even any other bot landed.
BMW™
Newbie
- Feb 6, 2018
- 23
- 2
just say in simple word guysRobots.txt is the door of a website. Now think what type of house you wanna build? With door or without?
It's up to you, you wana list/show your website on google search engine or not?
If not then they will not list your site on there search data and it's not necessary unless you have a secret website
WPninja
Regular Member
- Apr 25, 2017
- 354
- 90
Mate, I made a dramatic intro at first, but both of those people maybe don't have enough entertainment on their life and couldn't get my point of view at all and start arguing with me :-D \just say in simple word guys
It's up to you, you wana list/show your website on google search engine or not?
Nowhere I said that robots file is a mandatory one
BMW™
Newbie
- Feb 6, 2018
- 23
- 2
Yes you are right well you said indirectlyMate, I made a dramatic intro at first, but both of those people maybe don't have enough entertainment on their life and couldn't get my point of view at all and start arguing with me :-D \
Nowhere I said that robots file is a mandatory one
let me clear for them if there's door means they can see your contents on your website through that and if you don't wana show to the public like in previous case there was a door to see but in this case you got a invisible magic door that only certain people allowed to view
digimarket
Regular Member
- Sep 20, 2016
- 351
- 59
Yes, because Robosts.txt generator tells the search engines which pages to access and index on your website and which pages not to.
- Jun 10, 2016
- 4,523
- 567
Not so important, only need of robots.txt file is to disallow search engine to not index some pages.