Should I block this on Robots.txt ?

D

DO067

Registered Member

May 4, 2016

90

5

• Oct 26, 2016

• #1

Hi there,

I've started an audit of the new version of a site for one of my customers and the tool reports lot of errors for such pages http://www.mywebsite.fr/advancedmedia/index/ajax?id=124/

The errors are mainly duplicated title or empty meta, which is no so important for such URL in my opinion.

Do do you think I should block those URL via the robots.txt file ?

Thanks

 

altec

Regular Member

May 31, 2013

333

85

• Oct 26, 2016

• #2

Why don't you just remove them if they are useless?

 

D

DO067

Registered Member

May 4, 2016

90

5

• Oct 26, 2016
• Thread Starter Thread Starter

• #3

I don't really know if there are useless. Any idea to check that ?
FYI I am only responsible of the SEO for the site, and I don't have contact with the agency that made the new site.

 

growsocialworld

Junior Member

Jun 2, 2012

184

32

• Oct 26, 2016

• #4

You just block the bots from crawling...

 

D

DO067

Registered Member

May 4, 2016

90

5

• Oct 26, 2016
• Thread Starter Thread Starter

• #5

That's why you suggest or it is irony ?

 

growsocialworld

Junior Member

Jun 2, 2012

184

32

• Oct 26, 2016

• #6

You can block it....

 

W

webspero

Newbie

Sep 6, 2016

36

1

• Oct 26, 2016

• #7

If those pages are of no use then you can remove those pages if those are of use then you can restrict the crawler using robots.txt file.

 

WowLancer

Elite Member

Jun 12, 2016

2,707

486

• Oct 26, 2016

• #8

Remove them if they are of no use.

 

D

DO067

Registered Member

May 4, 2016

90

5

• Oct 26, 2016
• Thread Starter Thread Starter

• #9

Thanks all

 

BlackShield

Elite Member

Jr. VIP

Jun 25, 2015

4,009

861

• Oct 26, 2016

• #10

I think your link is not an SEO friendly one. Coz, it has the reserved characters like ?and =. It may cause the problem with the links. So better to change the link structure with SEO friendly.