Test in-flight requests during shutdown

To test in-flight requests during shutdown, shutdown-check must know that the request is still active. A start barrier supplies that proof. The signal is sent only after the barrier passes.

Why is a start barrier required?

Without a barrier, a fast request could finish before SIGTERM. The service would then pass even if it had no shutdown handler because there was nothing left to drain.

shutdown-check prevents that false result. If it cannot prove active work, it returns SC111 and sends no signal.

Choose a barrier

Barrier	It passes when	Use it when	Requests
`response-headers`	Every response sent headers while its body remains open	The route can stream or delay its body	1–20
`probe`	A separate route changes from inactive to active	The handler sends nothing until its operation completes	1

response-headers proves that an HTTP response is open. A probe can prove that a specific internal operation has begun.

Use the response-headers barrier

Configure the workload:

shutdown-check.json

{
  "workload": {
    "path": "/slow",
    "concurrent": 2,
    "started": {
      "type": "response-headers",
      "timeoutMs": 5000
    }
  }
}

Each request must receive headers within timeoutMs, and its response body must still be open. With two requests, both must satisfy the barrier.

Build a suitable route

Node.js may hold headers until the first body write. Call flushHeaders() to send them immediately:

server.js

if (request.url === "/slow") {
  response.writeHead(200, { "content-type": "text/plain" });
  response.flushHeaders();
 
  setTimeout(() => {
    response.end("work complete\n");
  }, 2000);
  return;
}

The delay must be long enough for shutdown-check to send the signal, but shorter than shutdown.deadlineMs. The route should be safe to run repeatedly and should not modify production data.

Understand failure behavior

The barrier fails when:

a response ends before all headers are confirmed;
one request in a concurrent group ends early;
headers do not arrive before timeoutMs;
the route refuses or resets the connection.

A successful timeline contains:

  +  114 ms  work confirmed active — 2 response(s) sent headers; bodies still in progress

Use the probe barrier

Choose a probe when the handler computes its full result before sending any response. The probe reports whether that operation is currently running.

shutdown-check.json

{
  "command": ["node", "server.js"],
  "env": { "ENABLE_TEST_ROUTES": "1" },
  "baseUrl": "http://127.0.0.1:3000",
  "readiness": { "path": "/health" },
  "workload": {
    "path": "/reports",
    "method": "POST",
    "status": 200,
    "bodyIncludes": "report ready",
    "started": {
      "type": "probe",
      "path": "/test/work-active",
      "inactiveStatus": 204,
      "activeStatus": 200,
      "timeoutMs": 5000,
      "intervalMs": 50
    }
  },
  "shutdown": { "deadlineMs": 10000 }
}

The probe runs in two phases:

Before the workload, it must return inactiveStatus.
After the workload starts, it must return activeStatus while the workload response is still open.

If the first phase fails, the result is SC110. If the second phase times out or the workload ends first, the result is SC111.

Build a probe route

Track the real operation with a counter. Expose the route only in the test environment.

server.js

const http = require("node:http");
const { setTimeout: delay } = require("node:timers/promises");
 
let activeReports = 0;
 
async function buildReport() {
  await delay(2000);
  return { rows: 42 };
}
 
const server = http.createServer(async (request, response) => {
  if (request.url === "/health") {
    response.writeHead(200).end("ok");
    return;
  }
 
  if (
    request.url === "/test/work-active" &&
    process.env.ENABLE_TEST_ROUTES === "1"
  ) {
    response.writeHead(activeReports > 0 ? 200 : 204).end();
    return;
  }
 
  if (request.url === "/reports" && request.method === "POST") {
    activeReports++;
    try {
      const report = await buildReport();
      response.writeHead(200, { "content-type": "application/json" });
      response.end(JSON.stringify({ status: "report ready", ...report }));
    } finally {
      activeReports--;
    }
    return;
  }
 
  response.writeHead(404).end();
});
 
server.listen(3000, "127.0.0.1");
process.on("SIGTERM", () => server.close());

The counter changes before the slow operation and resets in finally, so the probe reflects failures as well as success.

Read a passing probe timeline

PASS SC000: Graceful shutdown verified
 
Timeline:
  +    6 ms  process launched — pid=73999
  +  115 ms  service ready — HTTP 200
  +  116 ms  work request sent — #1 POST /reports
  +  117 ms  work confirmed active — probe /test/work-active returned HTTP 200
  +  117 ms  signal sent — SIGTERM
  + 2120 ms  work request finished — #1 HTTP 200
  + 2122 ms  process exited — code=0, signal=none
  + 2123 ms  shutdown verified — work completed and service exited before deadline

Prove that the operation completed

A status 200 can still contain an error response. Set bodyIncludes to text that appears only after successful work:

{
  "bodyIncludes": "report ready"
}

The comparison is case-sensitive. A missing marker returns SC203. Choose stable text, not a generated ID or timestamp.

Design test-only routes safely

Enable them only through a test environment variable.
Do not expose internal state in production.
Avoid real billing, email, or destructive side effects.
Reset counters in finally.
Keep the workload deterministic on slow CI machines.
Give parallel tests isolated ports and state.

Fix common barrier failures

Symptom	Likely cause	Fix
Response finishes immediately	Route is too fast	Delay or stream the controlled test response
Headers never arrive	Node.js is buffering them	Call `flushHeaders()` or write a body chunk
Probe starts active	Old work or incorrect probe state	Reset state and verify `inactiveStatus` first
Probe never becomes active	Counter changes too late or wrong path	Set state before the operation and verify the route
CI fails but local passes	Timing window is too small	Make workload duration clearly longer
Multiple requests reject the config	Probe supports one request	Use `response-headers` for concurrency

In-flight work and start barriers