Changing robots.txt

I

Iceman1978

Registered Member

Nov 15, 2013

80

7

• Aug 15, 2014

• #1

My adsense account says one of my sites is having a crawler error with a robot denied and a page not found error. I have(I think) fixed the page not found error but I am bamboozled by the robots.txt one. Being Google,there explanation and "remedy" was to talk in circles and not actually explain very much!
Can anyone tell me how to fix the robots.txt?

 

N

nethious

Newbie

Oct 25, 2011

31

7

• Aug 15, 2014

• #2

does it have the correct permissions?

 

I

Iceman1978

Registered Member

Nov 15, 2013

80

7

• Aug 15, 2014
• Thread Starter Thread Starter

• #3

What do you mean by correct permission?

 

I

Iceman1978

Registered Member

Nov 15, 2013

80

7

• Aug 15, 2014
• Thread Starter Thread Starter

• #4

bumpety bump bump

 

C

campo

Newbie

Aug 1, 2014

49

4

• Aug 15, 2014

• #5

depends on what page you have.
When you look at the robots.txt, have a look for any disallow statements and remove them unless you know they should be there.
cant post a link or even what to google :s
Disallow should only be for directories you dont want google to index such as admin, shopping carts, my account and so on....