Sitemap vs Crawl

Sitemap vs Crawl produces sitemap only, crawl only, and both. The same 30-page sample crawl is split three ways. Sitemap only means the address was stored from the sitemap and was not seen in the crawl. Crawl only means the crawl saw it and the sitemap did not. Both means the address was in each set. You can paste XML into the box, or you can enter a public sitemap URL and let the server fetch it. A pasted urlset is parsed on the server without a second copy of the parser in the browser. The form stays near the top of the page, under the single title. (Sitemap vs Crawl, guide 1.)

Sitemap vs Crawl shares one fetcher with the other sitemap tools. The fetcher allows http and https only. It rejects file, gopher, and data URLs, then blocks localhost, loopback, link-local, private ranges, CGNAT space, and the cloud metadata address before it opens a socket. DNS answers are checked before the connection, and a redirect is checked again. A redirect loop is reported instead of being followed forever. Three hops is the maximum. (Sitemap vs Crawl, guide 3.)

When a URL must be fetched, the page sends the job to the server and shows a short progress line while it polls. Small pastes return immediately with the same result shape: source URL, index or urlset, child sitemaps, page rows, errors, duplicates, redirect hops, status, truncated flag, and counts. Repeat fetches of the same normalized URL can reuse a finished parse for 10 minutes. Jobs sit in a file folder so more than one server process can see them, and they expire after 30 minutes. (Sitemap vs Crawl, guide 5.)

A concrete pass through Sitemap vs Crawl starts with https://example.com/sitemap.xml. If that address is an index whose first child is https://example.com/about, the file list and the page list answer different questions. The page address https://example.com/pricing belongs in the page list only when a urlset contains it. Keep the two lists apart when you copy them into a spreadsheet or a ticket. (Sitemap vs Crawl, guide 7.)

Take Sitemap vs Crawl as one step. The next page in this set is Orphan Page Finder, which answers a different question about the same sitemap. Sitemap Compare is the other close check. Finish the reading on this page before you switch, so you do not mix sitemap only, crawl only, and both with a different report. (Sitemap vs Crawl, guide 9.)

Use https://example.com/sitemap.xml as the file you are checking and https://example.com/pricing as a page address you expect to see or not see, depending on the job. Use sitemap only to spot pages the sample crawl did not reach. Use crawl only as a hint that a linked URL is missing from the sitemap. Use both as the overlap, not as a ranking. If the file is a sitemap index, child documents such as https://example.com/about are fetched only when you supplied the index URL, only while the depth is within 2, and only until 20 child sitemap requests have been made. (Sitemap vs Crawl, guide 2.)

Large files are cut at 5 MB, and the stored page list stops at 5,000 URLs. Gzip responses and addresses that end in .gz are decompressed before parsing. A DOCTYPE or an entity declaration is rejected so the parser cannot expand an external entity. An HTML document or a Cloudflare challenge body is reported as not a sitemap. The raw response body is not written to the log. (Sitemap vs Crawl, guide 4.)

Treating sitemap only as an error. The crawl stops at 30 pages, so most sitemap URLs will be sitemap only on a large site. After you read that distinction, look at the severity of any finding. An error means the document or the fetch failed the check this page is responsible for. A warning, such as a duplicate loc or a truncated file, means the rest of the result is still usable. A note explains a cap, a paste that did not download child files, or a sample crawl. (Sitemap vs Crawl, guide 6.)

The result panel uses the same dark block as the other tools on this site. Copy is there when a text result is useful. A download button appears only for a file this page actually builds, such as CSV, XLSX, XML, or llms.txt. Sitemap vs Crawl does not add a second title to the page, and it does not restyle the tools around it. (Sitemap vs Crawl, guide 8.)