AI crawlers miss rendered content signals

SEOGami

Regular Member
Jr. VIP
Joined
Feb 28, 2024
Messages
201
Reaction score
145
Found an incredible study by Oncrawl. Rankings and Indexed pages seem to be missing entirely from AI rankings.

In tracking 500 million crawler fetches across major AI platforms, they found zero executions of a single line of JavaScript.

The key tracking results
- 500 million fetches covered major AI systems
- Zero pages ran any JavaScript code
- Client side rendered content stays completely invisible to AI

Why this creates SEO blind spots
- Google visible pages may never reach AI engines
- AI search traffic could bypass traditional ranking strategies
- The version of your site that ranks might not be the one AI reads
- AI Overviews and generative features rely on different signals

What I added to our workflow
- Run AI specific crawlers on your content
- Compare static vs rendered HTML versions
- Ensure pages work for JavaScript dependent fetches
- Monitor AI accessibility alongside Google crawlability


The study
https://www.oncrawl.com/ai/myth-vs-data/signals-overlook/
 
Interesting point i have not considered the difference between the rendering by Google and the possibilities of the AI crawlers before checking the raw HTML against the rendered one appears to be an additional step to consider for JS heavy sites.
 
That’s a good reason to avoid putting important content only behind JavaScript.

Server rendered or static HTML is generally safer if you want the content to be accessible to different crawlers, not just Google.
Checking the raw HTML is definitely worth adding to technical SEO checks.
 
yeah this is a pretty big blind spot, making sure the important content is in the static HTML seems worth checking imo.
 
Back
Top