What Is a Sitemap (and When Do You Need One)

A sitemap is a file that lists the URLs on a website so that search engine crawlers can discover them. It is not a page visitors see — it is an instruction set for Googlebot. Google Search Central defines a sitemap as a way to tell Google which pages you consider important and to provide structured metadata about each one. Submission does not guarantee crawling, and crawling does not guarantee indexing.

Google accepts seven sitemap formats: the standard XML sitemap, sitemap index files, RSS 2.0 and Atom 1.0 feeds, plain text, the Google News sitemap extension, image sitemaps, and video sitemaps. Each serves a different content type and discovery need. The standard XML format handles most sites. The Google News format serves news publishers with time-sensitive editorial content that needs Googlebot attention within minutes of publication. Image and video sitemaps extend discovery to media assets Googlebot cannot find as easily by following links alone.

Not every site needs a sitemap. Google's documentation states that sites with roughly 500 pages or fewer — where all pages are reachable via internal links from the homepage — can rely on Googlebot's link-following to discover pages without one. A well-linked 30-page business site gains little from a sitemap. A 10,000-page e-commerce catalog — where product pages three levels deep are not linked from the homepage — depends on it as the primary crawl-discovery mechanism. For a broader grounding in crawling, indexing, and technical SEO concepts, the SEO glossary provides the conceptual framework.

The URL Inspection tool covers the second half of this pillar and is equally misunderstood. It provides two distinct views of any URL: what Google recorded at its most recent crawl (the Google Index view), and what Googlebot would find if it fetched the page right now (the Live Test). These views regularly differ. A page blocked by robots.txt two weeks ago and unblocked today still shows as blocked in the Google Index view. The Live Test shows the current state. Using both views in the right order — index view to understand what Google holds, live test to confirm a fix has taken effect — is the core skill for diagnosing indexing problems. Start with What Is Google Search Console if you are new to the tool before going deeper here.

Together, sitemaps and the URL Inspection tool are the two direct levers a site owner has over what Google crawls and indexes. Sitemaps tell Google which URLs exist and which the site owner considers indexable. The URL Inspection tool shows what Google actually did with those URLs — and what it would do right now. Neither one causes indexing. Both improve the speed and reliability of discovery and diagnostic confidence.

• Sitemaps are discovery hints, not indexing instructions. Submitting a URL tells Google the URL exists. It does not cause Google to crawl it, and crawling does not cause indexing.
• Use both submission methods. Declare the sitemap in robots.txt for passive all-crawler discovery. Submit via the GSC Sitemaps report for the monitoring and error-detection feedback loop. Neither replaces the other.
• The URL Inspection tool has two views. The Google Index view reflects past crawl state. The Live Test reflects current server state. Use both when diagnosing a missed or excluded page — they answer different questions. Reconcile results with the Index Coverage & Indexing report for the full picture.
• Request Indexing is a scarce resource. The button has a small daily limit per property — around 10–12 URLs per day per secondary sources. Run the Live Test first to confirm no blocking issues exist before using it.
• Seven sitemap formats exist, each with different limits. The Google News sitemap allows 1,000 URLs in a rolling 48-hour window — a fundamentally different tool from a standard XML sitemap (50,000 URLs / 50 MB). Treat them as separate instruments.
• The Sitemaps report shows discovery, not indexing. The URL count in the report reflects how many URLs Googlebot found in the sitemap — not how many it indexed. Use the Performance report to see which discovered URLs are actually ranking.

robots.txt Sitemap Directive vs. GSC Submission: Why You Need Both

There are two ways to notify Google about a sitemap: declaring it in robots.txt with the Sitemap: directive, and submitting it directly through the GSC Sitemaps report. They are not interchangeable. Each serves a different function, and using only one leaves a gap.

The robots.txt Sitemap: directive is passive. It announces the sitemap's location to every crawler that fetches robots.txt — not just Googlebot. Per the Google robots.txt specification, the Sitemap: line is independent of any User-agent: rules and applies to all compliant crawlers. There is no status report, no error feedback, and no URL count visible to the site owner. It is an announcement, not a registration.

GSC Sitemaps submission is active and Google-only. It registers the sitemap with the verified property and provides the full feedback loop: fetch status, URL count discovered, parsing errors, and the date of Googlebot's last fetch. It requires a verified GSC property with owner-level permissions. It does nothing for Bingbot or other crawlers.

Best practice is both. Declare the sitemap in robots.txt so all crawlers find it passively — and so Googlebot finds it even if the GSC property is lost or re-verified. Submit via GSC for the monitoring and error-detection feedback loop that robots.txt cannot provide.

Sitemap Formats Google Accepts: A Reference Guide

Google supports seven sitemap formats. Choosing the right one depends on content type, update frequency, and the discovery problem the sitemap needs to solve. The standard XML format covers the vast majority of use cases. The Google News format is a functionally distinct tool — a rolling freshness feed, not a URL inventory. Google Search Central's sitemap guide covers the full format specifications.

The Google News sitemap is the format most frequently confused with a standard XML sitemap. Its 1,000-URL ceiling and 48-hour rolling window make it a freshness feed — not a URL registry. It is designed to be updated continuously (minutes to hours after publication), not batched weekly. CMS plugins for major publishing platforms handle this automatically. A news publisher submitting 5,000 article URLs into a News sitemap and expecting normal XML behavior will find Googlebot ignoring everything outside the 48-hour window.

Image sitemaps underwent a deprecation in August 2022. Google removed support for the extension tags <image:caption>, <image:geo_location>, <image:title>, and <image:license>. They have had no effect on indexing since that date. Only <image:loc> is required. Leaving deprecated tags in an image sitemap is harmless but adds file size without benefit — remove them on the next sitemap refresh.

Sitemap Limits: URLs, File Size, and GSC Submission Caps

Sitemap limits operate at three levels: per-file limits (URL count and file size), sitemap index limits (child sitemaps per index), and GSC property limits (sitemaps submitted per property). These are separate constraints that can be hit independently. The 500-sitemaps-per-property cap is the one most frequently conflated with the 50,000-URLs-per-file limit — they measure different things. Source: Google Search Central, Manage Your Sitemaps With Sitemap Index Files, 2026.

The hreflang nuance matters for international sites. A URL with 10 language alternates declared via <xhtml:link> in the sitemap still counts as one URL against the 50,000 limit — not 11. Only the primary <loc> tag increments the counter. This means a 40,000-URL multilingual site with 5 alternate languages per URL can keep everything in a single sitemap file without approaching the ceiling.

For the URL Inspection API — relevant to programmatic crawl workflows — the per-site limit is 2,000 queries per day and 600 queries per minute, per the Search Console API usage limits page. Full programmatic URL inspection is covered in the GSC API & Automation sibling pillar.

Google Index View vs. Live Test: What Each Shows

The URL Inspection tool presents two distinct datasets within the same interface. The Google Index view shows Google's stored state from the most recent crawl. The Live Test fetches the page from the live server at the moment you click the button. Conflating them is the most common diagnostic mistake — and it leads to acting on the wrong data. Source: GSC Help, URL Inspection tool; Inspect and troubleshoot a single page, 2026.

The gap between the two views is most damaging when debugging a recent change. If a page was blocked by robots.txt two weeks ago and the block was removed today, the Google Index view still shows the page as blocked — because that is what Google last recorded. The Live Test shows the block is gone. Acting on the index view alone leads to a false conclusion that the problem persists. Acting on the live test alone ignores the indexing history Google holds.

The Live Test result "URL is available to Google" does not mean the page is indexed or will be indexed. It means the page is currently reachable by Googlebot and not blocked by robots.txt or HTTP errors. The full indexing diagnosis comes from the index view's coverage status code — a page showing "URL is available to Google" on a live test can simultaneously show "Duplicate without user-selected canonical" or "Crawled — currently not indexed" in the index view. Once pages are indexed, use the Performance report to track clicks, impressions, and position for each URL.

How to Submit a Sitemap in Google Search Console

Submitting a sitemap in GSC takes three steps: verify the site property, navigate to Sitemaps in the left menu, enter the sitemap URL (relative to the property root), and click Submit. The submitted URL must return HTTP 200 and be accessible without authentication. If the sitemap is at https://example.com/sitemap.xml, submit sitemap.xml — GSC prepends the property root automatically.

After submission, the Sitemaps report shows the fetch status (Success, Pending, or an error), the number of URLs discovered in the sitemap, and the last time Googlebot fetched it. The "URLs discovered" count does not equal "URLs indexed." It is the count of URLs Googlebot found listed in the sitemap file — not a count of pages that are live in Google's index. Reconcile the discovered count with the Index Coverage report to see how many of those URLs are actually indexed. Also declare the sitemap in robots.txt with a Sitemap: directive pointing to the full URL (including https://) so all crawlers find it passively, independent of GSC status.

Common Sitemap Errors in the GSC Sitemaps Report

The Sitemaps report surfaces six error types. Each has distinct causes — and each requires a different fix. The "Couldn't fetch" error is the most commonly misdiagnosed: it does not always mean the sitemap is broken. It includes transient Googlebot-side fetch failures that resolve without action. Wait 24–48 hours before acting on a first "Couldn't fetch" occurrence to distinguish transient from persistent. Source: Sitechecker, Google Search Console Couldn't Fetch Sitemap, 2026.

The "Submitted URL not indexed" error is a diagnostic dead end if treated as a sitemap problem. Removing the URL from the sitemap does not fix the underlying issue, and resubmitting it more frequently does not override Google's quality assessment. The correct path is URL Inspection on the individual URL, reading the coverage status reason code, and addressing the root cause — whether that is a canonical conflict, thin content, or a noindex directive the CMS is injecting automatically.

Request Indexing: When to Use It and When Not To

The Request Indexing button asks Google to prioritize recrawling a specific URL. It does not guarantee indexing will happen, and it does not bypass Google's quality signals. Google does not publish an official daily quota for the button. Secondary sources consistently place the practical ceiling at around 10–12 URLs per day per GSC property, after which the button becomes temporarily unavailable. The limit is property-level, not account-level, and resets the following day. Source: Alevdigital, Google Search Console Request Indexing: Limits, Time, Steps & Fixes, 2026.

MB Adv Agency has found that the most common waste of the Request Indexing quota is submitting URLs that are excluded for quality reasons — pages with thin content, canonical mismatches, or near-duplicate issues. No volume of resubmission resolves a quality signal problem. The budget is better reserved for genuinely new or meaningfully updated pages where the Live Test confirms the page is clean and indexable.

For bulk discovery after a site relaunch or major content migration, submit an updated sitemap instead. Sitemap submission has no daily URL ceiling — it is the correct mechanism for sets of URLs that exceed the Request Indexing daily limit. Googlebot's crawl of a newly submitted sitemap is not instantaneous; processing time varies by site authority and crawl budget, but the sitemap submission approach is structurally more efficient than serial Request Indexing for large URL sets.

Sitemap Architecture: Single File, Segmented, and News

Sitemap architecture is not just a technical convenience — it is a content freshness signal. A flat sitemap containing 50,000 old and new URLs gives Googlebot no signal about which URLs are recent. A segmented architecture with a sitemap index referencing a rolling news sitemap and separate static sitemaps by content type allows Googlebot to prioritize high-freshness sections without re-fetching stable ones. Source: Google Search Central, Manage Your Sitemaps With Sitemap Index Files, 2026.

MB Adv Agency consistently recommends segmenting sitemaps by content type for any site managing more than a few hundred pages. The per-segment visibility in the GSC Sitemaps report — seeing exactly how many blog URLs vs. product URLs Googlebot discovered from each child sitemap — is a practical diagnostic advantage. When product pages are not indexing but blog pages are, a segmented sitemap architecture makes that discrepancy immediately visible without manual URL-by-URL investigation.

Programmatically generated sitemaps require one quality discipline that is frequently skipped: excluding noindex, 404, and soft 404 URLs. Including them pollutes the sitemap with non-indexable content, inflates the "URLs discovered" count in the GSC report, and signals to Googlebot that the site owner has not curated the sitemap. The sitemap should contain only URLs the site actively wants indexed. For a deeper treatment of canonicalization and coverage status in relation to sitemap architecture, the Index Coverage & Indexing guide covers the diagnostic framework end-to-end.

URL Inspection Coverage Status Codes Explained

When the Google Index view shows a URL is not indexed, it displays one of 16 distinct reason codes across four status buckets: Indexed, Excluded, Error, and Warning. Understanding which bucket a URL falls into is the fastest way to determine the right action. Most non-indexed URLs land in the Excluded bucket — and most Excluded reasons are content or canonicalization issues, not technical errors. Source: GSC Help, Page indexing report; Search Console API usage limits, 2026.

The Excluded bucket (7 reason codes) vastly outnumbers the Error bucket (5 codes). This distribution matters for triage: most non-indexed URLs are excluded for content or canonicalization reasons — not technical failures. A site with 2,000 pages showing "Crawled — currently not indexed" has a content quality problem, not a server error. Treating it as a technical problem (resubmitting via sitemap, checking robots.txt) wastes diagnostic time. URL Inspection on a representative sample of excluded URLs is the correct starting point.

The Warning bucket includes "Indexed, though blocked by robots.txt" — a situation where Google indexed a page despite a Disallow: rule, typically because the URL was linked from other indexed pages. This is a common source of confusion: robots.txt blocks crawling, not indexing. If a URL is linked prominently from the homepage, Google indexes it even if robots.txt tells Googlebot not to crawl it. To prevent indexing of a blocked page, add a noindex meta tag (which requires making the page temporarily crawlable so Google can read the tag). For the full diagnostic framework connecting URL Inspection data to indexing actions, see Index Coverage & Indexing in GSC.

Methodology

All sitemap limits and URL Inspection mechanics in this article are sourced directly from Google Search Central documentation and the Google Search Console Help center, cross-referenced June 2026: Build and Submit a Sitemap; What Is a Sitemap; URL Inspection tool; and the Page indexing report. The Request Indexing daily limit (~10–12 URLs per day per property) is sourced from secondary practitioners — Google does not publish an official quota. Keyword data is from Ahrefs, June 2026 (operator-supplied). FAQ rich result removal timeline sourced from Search Engine Land, May 2026. Reviewed by MB Adv Agency, June 2026.