Directory Site Indexing Challenge

seoboyz01 said:

Seen this happen with directories before. Getting 10k pages indexed fast doesnt mean Google will keep them if a lot of them look too similar or thin.

What youre doing now sounds like the right direction. I’d focus on making the profile/city pages genuinely different, improve internal linking and only keep the better pages in the sitemap.

I wouldnt manually request thousands of URLs though. Submit the sitemap and let Google recrawl naturally, then watch which page types start coming back first. Google also says recrawling/reindexing can take several days, so recovery is usually gradual rather than instant.

Click to expand...

mony_men said:

Hey there,
First of all, kudos to you for taking the right programmatic steps. Shifting from a "trigger-happy" launch to focusing on data enrichment, content quality, and trimming the fat (removing thin sitemap pages) is exactly how you recover from a "Crawled - currently not indexed" status.

Click to expand...

ecc10394 said:

Hey everyone, several months ago I created a directory for a specific niche in home services (US only). I think I got trigger happy in the beginning and like any SEO noobie would, just pushed a lot of pages up (profile pages, city pages, etc...) to the sitemap to get indexed.. And Google, being the awesome company they are, indexed all of them!

I went from 0 to almost 10,000 indexed pages in a months or so...

But to my surprise... all of them started dying and now, only 200 of the 10,000 are actually indexed.. the rest of crawled but not indexed..

So this last month or so, I've spent all my time working to do the following:

- redesign/improve brand
- completely improve the quality of the pages.. thinking from first principles..what would a homeowner want to see in this page before they see it as 'valuable'...
- i've removed many pages from the sitemap intentionally
- I've relied heavily on SEMrush API to really help craft a perfect strategy for each page I worked on... whether thats design, copy/keywords, tech seo (meta description, titles, etc..), llms.txt, etc.... the list goes on..
- and most importantly, ive been working tirelessly on a pipeline to enrich profile pages with data that cant be easily found/scraped with a single google search...

I'm almost done with this massive undertaken and Im hoping once I start slowly and manually requesting indexing of pages.. that I can start to reverse this major problem.

Has anyone here gone through a similar issue of getting indexed on thousands of pages --> lost that indexing --> and came back?

Would love advice on what you saw work well and the strategic steps you took, if any, to reverse.

Thanks!

Click to expand...

Hot_Box said:

I wouldnt manually request thousands again. Start with a small batch of your strongest city and profile pages and see if they stay indexed for a few weeks before opening up the rest.

Click to expand...

celsusa said:

I’d focus on the 200 that survived and figure out what they have in common then use that as the standard for the rest.

Click to expand...

chhavi said:

Here’s a natural, positive, relevant BHW-style comment based on the thread’s indexing topic.

Interesting challenge. I’d focus on getting the important city/profile pages indexed first rather than pushing the whole sitemap at once. Would be useful to see which approach gives the best indexing rate over time.If you want, I can also make it shorter and more casual for BHW.

Click to expand...

lifework said:

If your directory contains thousands of items, I wouldn’t recommend indexing every page. Keep most of them set to noindex and focus on your /city/category pages instead. These are your gold pages ,pages most likely to attract traffic and generate revenue.
keep crawl budget safe and index what matter

Click to expand...