Scraping: fail loudly when the page layout changes
A scraper that parses nothing and exits 0 looks healthy while it fills your database with gaps. A count check and a saved fixture catch it.
A scraper that parses nothing and exits 0 looks healthy while it fills your database with gaps. A count check and a saved fixture catch it.
Bank SMS and email alerts flow through one parser, a dedupe hash and a Google Sheet. A morning cron job drafts the summary. Only the flow, no figures.
A retry helper that backs off with jitter, reads Retry-After, and gives up fast on a 404. Run against a local server that fails on purpose.
Many pages ship their data as JSON inside the HTML. Reading that survives a redesign that breaks every class-name selector.