Bad parameter issue, new site, low crawl budget now.

Hi everyone,

Having some indexing issues and could use some advice.

The site has around 500 pages.

Here's what happened:

• Around August 1, Google discovered a large number of URLs with ?amount= parameters.
• We also had internal links pointing to noindex pages returning 200, which Google spent a lot of time crawling.
• Both issues were fixed around end of August.
• Some parameterized URLs had already been indexed, creating duplication issues.

Current situation:

• Only the homepage is indexed.
• Most clean URLs are stuck in "Crawled – currently not indexed," including pages unrelated to the original issue.
• URL parameters are 301, but only some are showing 'Redirect', most are sitting in "Crawled - currently not indexed"
• Some parameters are still indexed in the retired sitemap, and individual URL inspection shows "Crawled - not indexed". (No longer indexed, but still shows indexed in pages)
• Google was showing outdated content in search results.
• Crawl stats now show very low HTML crawling activity.
• We have a clean sitemap, a separate sitemap for retired URLs, and proper canonicals.
• Manual indexing requests haven't helped.

It's been nearly two months since the fixes, with little improvement.

Has anyone experienced something similar? Could the initial crawl/duplication issues still be affecting Google's indexing decisions, or should I be looking elsewhere?

Any advice would be appreciated!

SEO Executive said:

Check crawl dates after the fixes and verify redirects directly. For clean URLs already recrawled, inspect fetched HTML, headers and canonical targets; crawl budget alone doesn’t explain their indexing status.

Click to expand...

Ive checked using the live test tool, all html, headers, and canonical targets are correct. Redirects are one hop, only http -> https is two.

On the retired sitemap, we have "16 indexed" last crawl dates were all 3 months ago, and they all have parameters. On individual inspection they have more recent crawl dates because of manual requests, but they're not updating.

Last crawl shows July on the pages, individual shows mid September.

I feel like Google is stuck in a SERP loop.

SEO Executive said:

The live test can’t confirm Google’s chosen canonical. I’d use individual URL Inspection for the current indexing status. Are those September crawl dates for clean URLs too, or only parameter versions? That distinction matters before calling it a loop.

Click to expand...

The individual URL "View craweled page" is grayed out. All but homepage.

When I view "Indexed pages" it says I have 15 indexed with the crawl dates in July, but upon inspecting each one, they're all "Crawled - currently not indexed" from September. View crawled pages is not available in any of these. This is happening on both clean and retired sitemaps.

When google discovers a parameter url, it crawls both parameter and canonical in the Crawl Data, but upon inspecting the parameters, the crawl date would not update sometmes, nor does the parameter version 'redirect'. In the crawl data we can see the parameter gets labeled at 301 redirect, but individual inspection just says "Crawled - currently not indexed". Althogh we did get a few that does redirect instead of "Crawled - currently not indexed".

In the beginning, I had manually requested most parameters and canonical at the same time, but now I'm waiting for the Googlebot to do it naturally.

Hopefully this make senses. I thought about starting over, but not sure.

UnknownW said:

Hi everyone,

Having some indexing issues and could use some advice.

The site has around 500 pages.

Here's what happened:

• Around August 1, Google discovered a large number of URLs with ?amount= parameters.
• We also had internal links pointing to noindex pages returning 200, which Google spent a lot of time crawling.
• Both issues were fixed around end of August.
• Some parameterized URLs had already been indexed, creating duplication issues.

Current situation:

• Only the homepage is indexed.
• Most clean URLs are stuck in "Crawled – currently not indexed," including pages unrelated to the original issue.
• URL parameters are 301, but only some are showing 'Redirect', most are sitting in "Crawled - currently not indexed"
• Some parameters are still indexed in the retired sitemap, and individual URL inspection shows "Crawled - not indexed". (No longer indexed, but still shows indexed in pages)
• Google was showing outdated content in search results.
• Crawl stats now show very low HTML crawling activity.
• We have a clean sitemap, a separate sitemap for retired URLs, and proper canonicals.
• Manual indexing requests haven't helped.

It's been nearly two months since the fixes, with little improvement.

Has anyone experienced something similar? Could the initial crawl/duplication issues still be affecting Google's indexing decisions, or should I be looking elsewhere?

Any advice would be appreciated!

Click to expand...

I would not do a fresh start at this point becuz it’s possible that google is still working on the old parameter URLs. Know which URLs Googlebot is crawling from the server logs. Also, know the internal linking nd uniqueness of the pages. even fixing the redirects does not guarantee that the URLs will be indexed.

UnknownW said:

Hi everyone,

Having some indexing issues and could use some advice.

The site has around 500 pages.

Here's what happened:

• Around August 1, Google discovered a large number of URLs with ?amount= parameters.
• We also had internal links pointing to noindex pages returning 200, which Google spent a lot of time crawling.
• Both issues were fixed around end of August.
• Some parameterized URLs had already been indexed, creating duplication issues.

Current situation:

• Only the homepage is indexed.
• Most clean URLs are stuck in "Crawled – currently not indexed," including pages unrelated to the original issue.
• URL parameters are 301, but only some are showing 'Redirect', most are sitting in "Crawled - currently not indexed"
• Some parameters are still indexed in the retired sitemap, and individual URL inspection shows "Crawled - not indexed". (No longer indexed, but still shows indexed in pages)
• Google was showing outdated content in search results.
• Crawl stats now show very low HTML crawling activity.
• We have a clean sitemap, a separate sitemap for retired URLs, and proper canonicals.
• Manual indexing requests haven't helped.

It's been nearly two months since the fixes, with little improvement.

Has anyone experienced something similar? Could the initial crawl/duplication issues still be affecting Google's indexing decisions, or should I be looking elsewhere?

Any advice would be appreciated!

Click to expand...

honestly, I would not recommend starting all over again, as google may take some time to delete your parameter URLs. check your log , internal link nd uniqueness. clean redirects do not guarantee indexing.

SEO Executive said:

I wouldn’t start over yet. A 301 in crawl data doesn’t guarantee the destination was indexed. Hover over the disabled “View crawled page” button and check “Page fetch”—what explanation does it show?

Click to expand...

It shows 'Page was not fetched'

I guess the right move is just to wait a bit longer.

with only 500 pages, i wouldn't focus too much on crawl budget yet. the fact that google already crawled the clean urls but didn't index them makes me think there's more to check. i'd inspect a few important pages and compare google's selected canonical with yours, then check the rendered content for anything missing or too similar across pages. fixing the parameter mess was a good start, but it won't automatically make google index everything again.

SEO Executive said:

If that’s the button’s tooltip, check the separate “Page fetch” value under Crawl. “Crawled—currently not indexed” records a crawl. Could you share one affected clean URL? I’d investigate further alongside monitoring.

Click to expand...

Here. This is the retired sitemap. Quite annoying , but maybe I just need to give it some time.