Scraping: retry with backoff, honour Retry-After, and know what not to retry
A retry helper that backs off with jitter, reads Retry-After, and gives up fast on a 404. Run against a local server that fails on purpose.
A retry helper that backs off with jitter, reads Retry-After, and gives up fast on a 404. Run against a local server that fails on purpose.