Hello everybody , I have a problem concerning the robot file , actually this robot file is blocking my urls and it prevents the appearance of our page in Google research results !!! and when i try to unlock it it displayed that the URL is in redirection state !!!! What should i do To unlock this robot file ???
Robot file !!! It is urgent !!!!!!!!
- Thread starter Mehsal
- Start date
Heisenberg
Senior Member
- Sep 11, 2014
- 885
- 644
Make a blank text file with notepad and call it robots, upload it to your website folder. Or just edit current one, if you posted contents of your robots.txt we could give you more info.
What is the next step after uploading the blank text file ??
- Mar 21, 2013
- 24,786
- 31,565
A blank robots.txt file is not recommended. Just delete it if you want it blank. Better return 404 than a blank robots.txt
How i will delete it ? ?
SunnyLeon
Elite Member
- Oct 13, 2013
- 2,237
- 2,082
Do you have cpanel or ftp access to your webhosting account ? because that`s where you find the robots.txt file, inside your website's files
- Mar 21, 2013
- 24,786
- 31,565
Do you have cpanel or FTP? In either one it's just a click.
I didn't understand exactly the main idea !!!! Can you clarify more please ??
SunnyLeon
Elite Member
- Oct 13, 2013
- 2,237
- 2,082
Yes, your website is hosted in a hosting account with a hosting company. What type of hosting do you have? How did you build your website? where did you forward the domain ? You have a hosting account and if that hosting account has access the the website's files in either cpanel or ftp, you can modify, add and delete files from there.
naskootbg
Senior Member
- Nov 8, 2010
- 847
- 280
robots.txt tells search engines robots what to do with your website.
Strange issue. Very strange!
When you try to open it, it redirects correct?
Check the redirects in the host panel.
If have not redirection in the cPanel, than they are in .httaccess file . If second the problem is bit complex.
To fix robots.txt :
1. Login your cPanel and go to File Manager or use FTP to access domain's root directory
2. Find robots.txt and edit
3. Search g00gle "robots txt generator" and generate code for your new robots.txt file
Strange issue. Very strange!
When you try to open it, it redirects correct?
Check the redirects in the host panel.
If have not redirection in the cPanel, than they are in .httaccess file . If second the problem is bit complex.
To fix robots.txt :
1. Login your cPanel and go to File Manager or use FTP to access domain's root directory
2. Find robots.txt and edit
3. Search g00gle "robots txt generator" and generate code for your new robots.txt file
I deleted the robot file but when I launched the exploration of Google it showed me that the urls are in redirection! !! Please explain me what is this redirection and why they are in the redirection state ??
I deleted the robot file but when I launched the exploration of Google it showed me that the urls are in redirection! !! Please explain to me what is this redirection and why they are in the redirection state ??
- Mar 21, 2013
- 24,786
- 31,565
Try ending your urls in / instead of a word and see if they still redirect.
For example /blabla/page-1
try /blabla/page-1/
Please tell me , can I send the backlinks of the website even if google didn't crawl my urls ??
naskootbg
Senior Member
- Nov 8, 2010
- 847
- 280
Can't get this. You want to send backlinks and hide them from g00gle?
robots.txt is only for the domain - non related to its backlinks. You can restrict bots like ahrefs and majesticseo from robots.txt so they will never crawl the backlinks to your pages (most likely).
But me I want to send the backlinks and I want to know if it is possible to do this action even if google didn't crawl my urls yet ? ??? ! !! ( and I am waiting for the crawling because I have already send the indexation request in order to allow To Google to crawl my website )
RayBay
Regular Member
- Sep 4, 2013
- 296
- 148
Test your site. Add in browser domain.com/robots.txt (replace domain.com with your domain name) and see if you actually have a robots.txt file. If its there and you don't like it - simply delete it via your cpanel.
For those who do not plan block search engines this file is of no use. Crawlers should be blocked via .htaccess.
For those who do not plan block search engines this file is of no use. Crawlers should be blocked via .htaccess.
I did all what you have said but what I want to know is , can I send the backlinks even if crawling my website still pending ? ?
RayBay
Regular Member
- Sep 4, 2013
- 296
- 148
Not sure if I understand. Do you mean that your site is not indexed and you want to start backlinking? If thats the case, than start linking as it will help to index your site faster.
Google - imtalk website submitter. Add your url and it will post it to 100 - 1000 sites. Not much of a link value, but will help you index it faster.
Google - imtalk website submitter. Add your url and it will post it to 100 - 1000 sites. Not much of a link value, but will help you index it faster.