Google on Discovered vs Crawled - Currently Not Indexed - Quality Issues & AI-Generated Content

• AI-generated content is not outright bad, but if visitors can identify it as regular AI-generated content, then that might be a bad thing.
• If anyone could have wrote it, that there is nothing unique or valuable - then Google won't index the page.
• When you notice that Google is not indexing a lot of your pages and there's no technical reason, then you almost need to take a step back and think about the quality overall.

And if we haven't visited them, we can't put them in the index versus "crawled - currently not indexed," which means we visited them and we didn't put them in the index, and that can have all sorts of different reasons. Would you say that that is often or only sometimes a sign of a quality issue?

Click to expand...

Sometimes. So it's definitely the case if our systems are seriously worried about the quality of a website, that they will reduce the number of pages that they index. Because if we—if we have strong concerns about the overall quality, then it doesn't make much sense for our systems to spend a lot of time on the website. So we'll probably crawl a lot less, we'll index a lot less. And then you'll see things like "crawled not indexed" or "discovered not indexed." Which from our point of view is basically our system saying we know about this. And once we're happy—we looked at it—once we're happy, we will take another look and see if we can index it. It's not so much that I would say you should take these situations and try to fix them.

Like from a technical point of view. It's not that you need to fix this technical issue that Google is not indexing this page at the moment. But rather, you almost need to, when you recognize a bigger pattern like this—like Google is not indexing a lot of your pages and there's no technical reason—you almost need to take a step back and think about the quality overall. And thinking about quality is really challenging because a lot of times it's yourwebsite and it's your baby, and of course it's the best baby ever. But taking a step back and trying to look at it with the eyes of someone who is not directly involved with your website, sometimes that opens up some ideas for areas where you can improve. Where maybe if most of your website is AI generated and it worked for a while, it might be that people look at this AI-generated site and they're like, "Well, I can tell this is AI generated. There's nothing unique or valuable that is available here for me." That's not to say that all AI-generated content is bad. But sometimes you—you just run across websites where you're like, "Anyone could have written this. This tells me nothing."

Click to expand...