Published on

TIL: a gitignored sitemap goes stale (93 vs 123 URLs)

TIL: a gitignored sitemap goes stale (93 vs 123 URLs)
Authors
On this page

I wrote a small script to tell IndexNow about every page on this site. It read public/sitemap.xml, found 93 URLs, and the API returned 200. Done, I thought.

The live sitemap had 123.

Why the counts didn't match

The site generates sitemap.xml at build time, and *.xml is in .gitignore. So the file on disk is whatever my last local build produced. It was missing 30 pages, including every post I'd written since that build. A 200 from the API only meant it accepted the 93 I sent.

I only noticed because I compared the two counts before calling it done:

curl -s https://www.nadimtuhin.com/sitemap.xml | grep -c '<loc>'

The fix

Stop reading the local file and fetch the live one:

const xml = await (await fetch(`https://${HOST}/sitemap.xml`)).text()
const urls = [...xml.matchAll(/<loc>([^<]+)<\/loc>/g)].map((m) => m[1])

The second run sent all 123.

Anything that reads a generated, ignored file is reading a snapshot. If the live site is what matters, ask the live site.