If your site is a single-page application built in React, Vue, Angular, or something similar, there's a good chance ChatGPT, Perplexity, Claude, and Google's AI Overviews are seeing a blank page where your users see a fully rendered product. Googlebot solved most of its JavaScript rendering problems years ago, but LLM crawlers haven't caught up yet, and the gap is bigger than most engineering teams assume. This presents a significant challenge for JavaScript SEO AI crawlers.
This guide is written for the technical side of the house. We'll cover how Large Language Model (LLM) crawlers actually handle JavaScript, how to test what they see on your specific site in under five minutes, and which rendering strategy makes sense depending on what you're running. If you'd rather have this diagnosed and fixed than do it yourself, our AI SEO team handles exactly this problem.
These crawlers aren't a rounding error either. Vercel's network data from late 2024 recorded 569 million GPTBot fetches and 370 million Claude fetches in a single month, alongside Googlebot's 4.5 billion. That's a meaningful and fast-growing share of how content gets discovered, and almost none of those AI crawlers can see anything your JavaScript renders after the initial page load.
How LLM Crawlers Differ From Googlebot on JavaScript
Googlebot runs a full, evergreen version of Chromium. It queues pages for rendering, executes JavaScript, waits for network requests to settle, and indexes the resulting Document Object Model (DOM). It isn't instant. There's often a delay between the initial crawl and full JS execution. But it works, and it's worked reliably since Google moved to evergreen Chromium rendering. However, when it comes to Server-Side Rendering (SSR) vs Client-Side Rendering (CSR) SEO, Googlebot has a distinct advantage over other crawlers.
Most LLM crawlers don't do this, or they do it inconsistently. The crawlers behind AI-powered search and answer engines are generally optimized for speed and scale across a much larger, faster-moving set of "give me an answer right now" queries, not for the patience required to execute a full client-side rendering pipeline on every page.
Where LLM Crawlers Fall Short
This isn't a guess. Vercel and MERJ analyzed real crawler traffic in late 2024 and found that none of the major AI crawlers render JavaScript, including GPTBot, ClaudeBot, Meta-ExternalAgent, Bytespider, and PerplexityBot. ChatGPT's crawler fetches JavaScript files in 11.5% of requests and Claude's in 23.84%, but neither executes what it downloads. The one exception is Google's Gemini, which runs on Googlebot's existing rendering infrastructure. Everyone else gets raw HTML only.
That means a client side rendered React or Angular app returns an essentially empty <div id="root"> to every one of those crawlers, with none of your actual content behind it. Timeouts tend to be short too. The same Vercel data found ChatGPT's crawler hitting 404 pages on nearly 35% of its fetches, versus about 8% for Googlebot, suggesting these crawlers are still fairly unrefined about which URLs they even bother requesting.
The practical consequence: content that ranks and gets cited perfectly well in Google's traditional index can be functionally invisible to the systems now answering a growing share of user questions directly, including product comparisons, pricing pages, and documentation.
How to Test What a Crawler Actually Sees
Don't guess. Test it directly. The fastest way to see exactly what a non-JS executing crawler sees on your site is a plain curl request, no browser involved:
curl -A "Mozilla/5.0 (compatible; GPTBot/1.0)" https://yoursite.com/your-page
Compare that output to what a real browser renders. If your page is client-side rendered whether you're running React, Vue, or navigating Angular SEO and AI crawler visibility at the same time this command will typically return something close to:
<!DOCTYPE html>
<html>
<head><title>Your App</title></head>
<body>
<div id="root"></div>
<script src="/static/js/main.abc123.js"></script>
</body>
</html>
That's it. No headline, no product copy, no pricing, no FAQ content. Just an empty mount point and a script tag. Whatever your actual page contains, an LLM crawler reading only that raw HTML sees none of it. Angular SEO and AI visibility are tightly linked here: if your Angular app relies on client-side rendering to populate content, AI crawlers face exactly the same blank-page problem as any other non-executing crawler.
Other Ways to Verify
View source in your browser using Ctrl+U or Cmd+Option+U, not Inspect Element. View Source shows the raw HTML before JavaScript runs, which is much closer to what a non-executing crawler sees than the fully hydrated DOM shown in DevTools. This is also a quick way to get a feel for the SSR vs CSR SEO divide in practice: a server-rendered page will show full content in View Source, while a client-side rendered page will show little more than an empty shell.
Google's URL Inspection Tool in Search Console shows you the rendered HTML after Googlebot executes JS. That's useful for comparison, but it tells you nothing about LLM crawler behavior specifically, since Googlebot's rendering capability isn't representative of the field.
Server log analysis is worth the effort too. Check whether known AI crawler user agents such as GPTBot, ClaudeBot, and PerplexityBot are hitting your site at all, and what response codes and response sizes they're getting. A consistently small response size on content-heavy pages is a strong signal that JS-dependent content isn't reaching them.
If your curl test comes back empty and your key pages depend on client-side data fetching to populate their content, you have a rendering problem worth fixing.
The Rendering Options: CSR, SSR, SSG, Prerendering, ISR
Once you've confirmed there's a gap, the fix is choosing the right rendering strategy for your situation. These aren't interchangeable. Each comes with real tradeoffs in complexity, hosting cost, and developer experience.
Client-Side Rendering (CSR)
CSR refers to a process where the browser downloads a near-empty HTML shell plus a JavaScript bundle, then builds the page in the browser. It's fast to develop and cheap to host as static files, but it's the worst option for crawler visibility. This is the pattern causing the invisibility problem in the first place.
Server-Side Rendering (SSR)
SSR is a technique where the server renders full HTML for each request before sending it to the browser or crawler. The client then hydrates that HTML to make it interactive. This is the most direct fix for crawler visibility, since every request, including a crawler's, gets fully rendered content immediately with no JS execution required to see it. When weighing SSR vs CSR SEO impact, SSR wins clearly for public-facing pages that need to be crawlable and citable: where CSR hands crawlers an empty shell, SSR hands them a complete document. The tradeoff is server load because every request triggers a render, along with added infrastructure complexity compared to a static host. Our own web development team runs production Angular apps on this exact pattern, migrating public-facing routes to server rendering while leaving authenticated app views on standard CSR.
Static Site Generation (SSG)
Static Site Generation (SSG) refers to pages that are rendered to static HTML at build time, then served like any static file. For an LLM crawler, JavaScript execution is never required — the HTML is fully formed before any request happens, which means content is immediately readable without any rendering pipeline. There's also no per-request server rendering cost, making SSG one of the most efficient options for crawler visibility. The limitation is that content has to be known at build time, which doesn't suit highly dynamic, frequently changing, or user-specific pages without a rebuild.
Prerendering
Prerendering is a middle-ground approach. A headless browser renders your CSR app in advance, or on demand, and serves that static snapshot specifically to crawlers, while real users still get the CSR experience. For teams working through Angular SEO and AI crawler visibility without committing to a full Angular Universal migration, prerendering can be a practical first step that delivers meaningful gains without a wholesale architectural change. It avoids a full SSR migration, but it adds a separate rendering pipeline to maintain, and it risks serving crawlers a version of the page that's out of sync with what users actually see if the snapshots aren't kept fresh.
Incremental Static Regeneration (ISR)
Available in frameworks like Next.js, Incremental Static Regeneration (ISR) combines SSG's crawler-friendly static output with the ability to regenerate individual pages in the background after deployment, without a full rebuild. Unlike CSR, which hands crawlers an empty shell, ISR serves fully rendered HTML on every request making it a meaningful upgrade from an SSR vs CSR SEO standpoint without the per-request server rendering cost that full SSR carries. It's a strong fit for sites with a large number of pages that change periodically but not on every request, like product catalogs, documentation, or content hubs.
Choosing a Strategy by Site Type
There's no single correct answer here. The right rendering strategy depends heavily on what the site actually does, and specifically on how much of a problem LLM crawler JavaScript handling is for each page type.
Marketing sites and landing pages tend to do best with SSG, since content is largely static between deploys, crawler visibility is maximized, and hosting stays simple and cheap. Documentation and content hubs usually suit ISR or SSG depending on how often things update, since ISR's ability to refresh individual pages without a full rebuild is a real advantage for content that changes occasionally. E-commerce and product catalogs generally call for SSR or ISR depending on catalog size and how often prices and stock change, since product pages need to be crawlable and current, but full SSR on every request may be unnecessary if ISR can keep pages fresh enough on its own.
Dashboards and logged-in applications are often fine staying CSR since this content isn't meant to be publicly crawled or cited by AI systems in the first place. Don't over-invest in SSR for content that has no business being indexed. An LLM crawler hitting a JavaScript-heavy authenticated dashboard isn't a problem worth solving — that content was never meant to surface in AI-generated answers.
SaaS product, pricing, and feature pages deserve special attention. These are exactly the pages most likely to be cited in AI answers about "best tools for X," and exactly the pages most commonly left as CSR by teams who built their marketing site inside their app's existing framework. When an LLM crawler encounters JavaScript rendering on these pages and gets back an empty shell, your product simply doesn't exist in that answer. SSR or SSG here tends to be high leverage.
The pages worth prioritizing first are the ones you most want cited in AI-generated answers: pricing, comparisons, core feature explanations, and documentation. Internal tooling and logged-in states can stay CSR indefinitely without cost.
Migration Path for an Existing SPA
A full rewrite is rarely necessary. Most frameworks support an incremental path.
Step-by-Step
Start with an audit. Run the curl test above across your highest priority pages, the ones you most want surfaced in AI answers, before touching any code. Prioritize by traffic and business value, not by what's easiest to migrate.
Adopt a meta framework where you can. If you're on React, Next.js supports SSR, SSG, and ISR within the same app, which lets you migrate page by page rather than all at once. Vue has Nuxt. Angular has Angular Universal for SSR.
From there, start with your highest value static pages. Marketing, pricing, and documentation pages are usually the easiest to move to SSG first, since they change infrequently and carry the most weight in AI answer generation. Move dynamic but cacheable pages to ISR next, things like product pages and blog content that update periodically but not per request. Reserve full SSR for genuinely per-request dynamic content, such as personalized pages or search results where content meaningfully differs per visitor and can't be cached. Leave logged-in, non-indexable app views as CSR, since there's no crawler visibility argument for rendering content that will never be publicly crawled.
Crawlability issues that surface during this kind of audit are often tangled up with broader technical SEO problems too, things like redirect chains, orphaned pages, and indexation gaps, and those are worth fixing in the same pass rather than separately.
Verifying the Fix
Migration isn't done until you've confirmed the fix actually worked, using the same method you used to diagnose the problem in the first place.
Post-Migration Checklist
Re-run the curl test with an AI crawler user agent against the migrated pages. You should now see full rendered content in the raw HTML response, not an empty div. Check server logs for AI crawler user agents again after deployment, and confirm response sizes on migrated pages have increased to reflect actual content rather than an empty shell. Monitor AI citation and referral traffic where you're able to track it, since a growing share of AI-driven referral traffic to previously CSR-only pages is the clearest downstream signal that the fix is working.
Re-check periodically, not just once. Framework updates, new client-side data dependencies, or a shift back toward heavier client-side fetching can quietly reintroduce the same invisibility problem later on. The curl test that flagged the problem in the first place is also your ongoing regression check. It costs nothing to run and takes seconds, so there's no reason not to make it a routine part of pre-deploy verification for any page you care about AI systems being able to see. Teams that would rather have this monitored on an ongoing basis instead of checking manually can fold it into a Core Web Vitals & performance review, since rendering strategy and page performance are usually diagnosed together anyway.
Key Takeaways
Do LLM crawlers execute JavaScript at all? Will switching to SSR hurt my Core Web Vitals?
Do I need to migrate my entire site to SSR? Is prerendering a permanent fix or a workaround?
How is this different from traditional Googlebot SEO?
Final Thoughts on JavaScript SEO AI Crawlers
None of this is really about chasing a new algorithm. It's about a simple mismatch that's easy to miss if you're not looking for it: your users get a fully built page, and a growing share of the systems now answering questions on their behalf get an empty shell. That gap doesn't show up in traditional rank tracking, and it won't show up in Google Search Console either, which is exactly why most teams don't know it exists until someone runs a curl command and sees nothing come back.
The fix doesn't have to be dramatic. You don't need to rewrite your app or abandon the framework you've already invested in. Most of the value comes from identifying the handful of pages that matter most for AI visibility, pricing, comparisons, documentation, core product pages, and making sure those specifically render without depending on JavaScript execution. Everything else can wait.
Talk to an Engineer About Rendering
If you're not sure which rendering strategy fits your site, or you've run the curl test and found the gap, our development team can walk through a migration path specific to your framework and traffic patterns.