Skip to main content
Technical SEO

What is XML Sitemap

A machine-readable list of your important URLs, submitted to help discovery.

Overview

An XML sitemap is a structured file listing the URLs you want search engines to know about, optionally with last-modified dates. It helps crawlers discover pages, especially on large sites, deep pages, or sites with weak internal linking. It is a discovery aid, not a guarantee of crawling or indexing.

A sitemap should contain only canonical, indexable URLs that return 200. Filling it with redirects, noindex pages, or non-canonical URLs sends mixed signals and can slow discovery of the pages that matter.

Best practice

Keep it clean and current, and let it reflect the true canonical structure of the site.

  • Only 200-status, indexable, canonical URLs
  • Split into multiple files if over 50,000 URLs
  • Reference it from robots.txt
  • Keep lastmod dates accurate, not faked

Why it matters

An XML sitemap is how you hand Google a clean, prioritised list of the URLs you actually want indexed, along with signals like last-modified dates. It does not guarantee indexing and it is not a substitute for good internal linking, but on large or poorly-linked sites it is a genuine discovery aid — it surfaces pages that are buried deep in the architecture or newly published. Its most useful modern role is diagnostic: submitting a sitemap in Search Console lets you compare "submitted" vs "indexed" and see exactly how much of your intended set Google is actually keeping.

  • A discovery aid and a prioritised "please index these" signal
  • Does not force indexing — quality and links still decide
  • The submitted-vs-indexed gap in Search Console is a powerful diagnostic

Keeping it clean

A sitemap should contain only canonical, indexable, 200-status URLs — the pages you genuinely want in the index. A common mistake is dumping every URL in, including redirects, noindexed pages, non-canonical duplicates and 404s, which sends Google mixed signals and dilutes the sitemap's value. For large sites, split into multiple sitemaps under a sitemap index (the 50,000-URL / 50MB limits per file), and keep `lastmod` dates honest so Google can prioritise genuinely-changed pages.

  • Include only canonical, indexable, 200-status URLs — nothing you have noindexed or redirected
  • Keep lastmod dates accurate so Google prioritises real updates
  • Split large sites into multiple sitemaps under a sitemap index file
  • Reference the sitemap in robots.txt and submit it in Search Console

In practice

A 40,000-page marketplace wants to know why only 26,000 pages are indexed. Submitting a clean XML sitemap turns the question into a diagnostic: Search Console's sitemap report shows 40,000 submitted vs 26,000 indexed, and the Page Indexing report attributes most of the gap to 'Crawled - currently not indexed' on thin, near-duplicate listing pages. The sitemap did not fix the gap, but it made it measurable and located it. The response is to consolidate or improve the thin listings and remove non-canonical URLs from the sitemap so it reflects only pages worth indexing. Kept accurate over time — canonical, 200-status URLs only, honest lastmod dates — the sitemap becomes an ongoing monitor of how much of your intended index Google is actually keeping, which is arguably its most valuable modern role.

Connected in the graph

Related defined terms

Part of our defined terms knowledge graph — browse every entry in this branch.

Crawl Budget

SEO Concept

How many pages a search engine will crawl on your site, and how often.

Indexation

Technical SEO

Whether a page is stored in a search engine’s index and eligible to rank.

Robots.txt

Technical SEO

A root-level file that tells crawlers which URLs they may or may not fetch.

A/B Test

Analytics

A controlled experiment comparing two versions to see which performs better.

Anchor Text

SEO Concept

The visible, clickable words in a hyperlink, used as a relevance signal.

Common questions

XML Sitemap — questions

Straight answers on how this fits your marketing and build.

Does submitting a sitemap guarantee indexing?
No. A sitemap helps Google discover URLs, but it still decides whether to crawl and index each one based on quality and canonical signals. It is a discovery aid, not a promise.
What URLs should I keep out of my sitemap?
Anything non-canonical, noindexed, redirected, or returning an error. Including those wastes trust and dilutes the signal. List only the clean, indexable pages you actually want ranked.
Do I need an XML sitemap if my site is small and well-linked?
It helps but is less critical — Google can crawl a small, well-linked site from its internal links alone. Even so, a sitemap is cheap insurance and, more importantly, gives you the submitted-vs-indexed diagnostic in Search Console, which is worth having at any size.
Should redirected or noindexed pages go in my sitemap?
No. A sitemap should list only the final, canonical, indexable URLs you want indexed. Including redirects, non-canonical duplicates or noindexed pages sends contradictory signals and reduces how much Google trusts the sitemap as a priority list.

Still have questions? Talk to a specialist