XML sitemap: uncrawlable pages

Hey there

I'm doing SEO on a bilingual website, passeportvacances.com. The issue we have is that a very important section, the coupons one, is not being indexed by google.

We have a XML sitemap online on Webmaster Tools but it is incomplete. The sitemap generators are not able to get links from the coupon section as they consider them "broken links". Here is the jpg report of the links that are considered broken by the tool.



1) Does someone have an intuition as of why this section is the only one that screws up?
2) What if I add the URLs manually to the sitemap before submitting it, will it work anyways or Google will still consider the links are broken? If so, will it be bad for the SEO?

Thanks a lot for your strategic input!
Charles

have you done this in google?

site:yourwebsite.com

do that and see what is indexed.

Hey,

Thanks for the answer. I did use the command, and saw that the section I'm talking about (the dynamic coupon pages) is not there at all. What could make a section uncrawlable even by the sitemap generator if it works perfectly fine onsite?

when did you post those pages, cuz it might just be taking a bit to index them, are you getting an error in webmaster tools regarding those pages?

No error in Webmaster tools, and sadly, the pages have been up and running for months... and they're directly linked to the homepage :s

Any other feeling as why an XML sitemap generator could not detect the pages? :s I'm also scared Google is not gonna like it if I manually add pages to the sitemap and he doesn't understand them.

Cheebee, what tool are you using to create your sitemap? I use Inspyder Sitemap Creator. If it won't crawl your pages, contact them for help (they are good and they helped me solve a few indexing issues my site was having).

Just took a look at the site. Something very strange is going on. You need to check your htaccess or something. The coupons page is getting a 404 even though you can see the page. Put this link: http://www.passeportvacances [DOT] com/coupons.html in either of these tools:

http://web-sniffer.net/
http://validator.w3.org/

EDIT: This is probably a PHP error. Are you using WordPress?

I used a few different xml sitemaps generators, Mr Pasta, but all of them couldn't retrieve the section. I guess this makes sense, considering the W3 validator is not able to retrieve it.

Yellowhats, that's a great insight, thank you! I did not program the website myself (I'm only the SEO) but I know it's done with Coldfusion, using a homemade CMS. I can definitely update the programmer with what you just said, though... so you think that there's is a line in the PHP code that makes the pages make a 404? That's technically possible even though the CMS is able to show some content?

Thanks a lot for the analysis,
Cheebee

Hey Cheebee,
No problem. Yes - the problem is technically possible. It happens every once a while in WordPress if you improperly arrange some code. But more than that the developers will definitely have to figure out - no way to know without seeing the site's code.

Good luck!

Cheebee said:

I used a few different xml sitemaps generators, Mr Pasta, but all of them couldn't retrieve the section. I guess this makes sense, considering the W3 validator is not able to retrieve it.

Yellowhats, that's a great insight, thank you! I did not program the website myself (I'm only the SEO) but I know it's done with Coldfusion, using a homemade CMS. I can definitely update the programmer with what you just said, though... so you think that there's is a line in the PHP code that makes the pages make a 404? That's technically possible even though the CMS is able to show some content?

Thanks a lot for the analysis,
Cheebee

Click to expand...