capped-fetch

fetch with hard limits on how much a server can make you decompress, buffer, and wait.

one response: 199 KB on the wire, 200 MB decoded (1029x)

fetch          176ms   rss + 642 MB   read 200 MB
cappedFetch     19ms   rss +  19 MB   refused — Response exceeded maxBytes (10485760 bytes decoded)

That's node examples/bomb-demo.ts. The response is 199 KB, its Content-Length is honest, and reading it costs 642 MB of resident memory.

The problem

Node's built-in fetch has no maximum response size and no total time budget. It also decompresses Content-Encoding automatically, which turns a small response into an unbounded one. The two defences people reach for don't work:

• Checking Content-Length first. It describes the compressed body. A truthful 199 KB declaration decodes to 200 MB. It's also attacker-controlled, and often absent.

• Asking for Accept-Encoding: identity. Node decodes Content-Encoding: gzip whether or not you asked for it:

  request:  accept-encoding: identity
  response: content-encoding: gzip   (50 KB)
  fetch gives you:  52428800 bytes
  

Because decoding happens inside fetch, before your code or a custom dispatcher sees a single byte, a decompressed-size cap cannot be bolted on from outside. This library owns the transport (node:http / node:https) and runs the decoder itself, which is the only place the check can live.

Anything that fetches URLs it doesn't control is exposed: webhook delivery, link previews, scrapers, SSRF-adjacent proxies, and agent/LLM tooling that retrieves arbitrary pages.

Install

npm install capped-fetch

Node >= 20.6. No runtime dependencies.

Use

import { cappedFetch } from 'capped-fetch';

const res = await cappedFetch('https://example.com/data.json', {
  maxBytes: 10 * 1024 * 1024,   // decoded body cap
  timeout: 30_000,              // whole operation, redirects and body included
});

const data = await res.json();

It returns a standard Response, so .json(), .text(), .body and the rest work as usual. Limits are enforced while the body streams — the read rejects as soon as a limit trips, and the socket is torn down. Nothing is buffered past the cap.

import { cappedFetch, CompressionBombError, ResponseTooLargeError } from 'capped-fetch';

try {
  await (await cappedFetch(url)).text();
} catch (err) {
  if (err instanceof CompressionBombError) {
    console.warn(`${err.compressedBytes} bytes inflated to ${err.decompressedBytes}`);
  } else if (err instanceof ResponseTooLargeError) {
    console.warn(`over the ${err.limit} byte limit`);
  }
}

Options

Option	Default	What it bounds
maxBytes	10 MiB	Body size after decompression.
maxCompressedBytes	maxBytes	Bytes accepted off the wire.
maxCompressionRatio	100	Decoded ÷ wire size. Infinity disables.
ratioGraceBytes	1 MiB	Decoded bytes before the ratio check starts.
timeout	30 s	Connect, redirects and body transfer, together.
maxRedirects	5	Redirect hops.
acceptEncoding	gzip, deflate, br, zstd	The header sent. false omits it.

method, headers, body, signal and redirect behave as they do in fetch.

Why the ratio has a grace floor

A 2 KB response that decodes to 400 KB is a 200x ratio and completely ordinary — small, repetitive payloads compress absurdly well. Applying a ratio limit to them produces nothing but false positives. ratioGraceBytes holds the check back until enough has decoded for the ratio to mean something; below that floor maxBytes is the only limit that applies.

Errors

All extend CappedFetchError.

• ResponseTooLargeError — limit, encoded (whether the wire cap or the decoded cap tripped)
• CompressionBombError — ratio, limit, compressedBytes, decompressedBytes
• RequestTimeoutError — timeout
• TooManyRedirectsError — maxRedirects
• UnsupportedEncodingError — encoding, supported

Content encodings

gzip, deflate, br, and zstd where the runtime provides it (Node 22.15+). Only encodings this runtime can actually decode are advertised in accept-encoding, and chained encodings (content-encoding: gzip, br) are unwound in order. An encoding with no decoder is an error rather than a body quietly handed back still compressed.

What it does not do

• It is not SSRF protection. It bounds the size and duration of a response, not where the request goes. Pair it with an agent that rejects private and link-local addresses if the URL comes from a user.
• HTTP/1.1 only. No HTTP/2, no proxy or dispatcher support yet.
• Request bodies are string or Uint8Array. Streaming uploads aren't supported.
• No cookie jar, no automatic retry.

Develop

Tests are TypeScript run directly by Node's test runner — no build, no install:

node --test "test/*.test.ts"    # full suite, needs node 24+ for type stripping
node examples/bomb-demo.ts

npm run build && npm run test:dist   # what CI runs against node 20 and 22

License

MIT