Question: `wailUntil()` lifetime on SSE response.

[cometkim]

I have used waitUntil for parsing streaming responses in a non-blocking manner.

Code looks like:

  const proxyHeaders = new Headers(req.headers);
  proxyHeaders.set("host", UPSTREAM.host);

  const contextPromise = extractRequestContext(req);
  const proxyRequest = new Request(targetUrl, {
    method: req.method,
    body: req.body,
    headers: proxyHeaders,
    redirect: "follow",
  });

  const response = await fetch(proxyRequest);

  if (!response.body) {
    return response;
  }

  const [passthrough, metricsStream] = response.body.tee();

  const metricsPipeline = (async () => {
    try {
      const context = await contextPromise;
      
      await metricsStream
        .pipeThrough(new TextDecoderStream())
        .pipeThrough(new LineSplitTransform())
        .pipeThrough(new SSEParserTransform())
        .pipeThrough(new SSEMetricsTransform(context, writeMetrics))
        .getReader()
        .read();
    } catch (error) {
      console.error("Metrics pipeline error:", error);
    }
  })();

  ctx.waitUntil(metricsPipeline);

  return new Response(passthrough, {
    status: response.status,
    headers: response.headers,
  });

But I got this warning from the production worker:

waitUntil() tasks did not complete within the allowed time after invocation end and have been cancelled. See: https://developers.cloudflare.com/workers/runtime-apis/context/#waituntil

I read the documentation, but I'm not sure whether "invocation end" refers to immediately after the response is returned, or when the response stream ends.

If I implemented it correctly, the metricsPipeline terminates as soon as the response stream ends and doesn't run for more than 30 seconds. There are no additional IO except for writing Analytics Engine data points.

Does this mean that waitUntil does not wait for the SSE stream response?

[nathanschram]

I dug into the workerd source to understand this and I think "invocation end" means when your fetch handler returns the Response - not when the response body stream finishes.

Based on the source code (in worker-entrypoint.c++), when your handler returns, the runtime resolves the DeferredProxy and calls drain() on the IncomingRequest via a kj::defer block. That's when the waitUntil soft timeout should start counting down. The response body streaming (proxyTask) actually runs after drain begins - it continues in a .then() continuation on the same promise chain.

So I think your metricsPipeline is likely racing against the 30s clock from the moment you return the Response, not from when the SSE stream ends. If the upstream stream takes longer than ~30 seconds, the waitUntil task gets cancelled.

There's an extra wrinkle with tee() specifically - when you tee a system stream (like a fetch response body), workerd wraps both branches in NoDeferredProxyReadableStream (in readable.c++). This should mean neither branch can be deferred past the IoContext lifetime, which would constrain things further.

I think the cleanest fix for your use case is to process the metrics inline instead of splitting the stream. A single TransformStream that both extracts SSE data and passes it through:

const metrics = { tokens: 0, events: 0, startTime: Date.now() };

const { readable, writable } = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk);
    // extract metrics synchronously as data flows
    parseSSEChunk(chunk, metrics);
  },
  flush() {
    // stream is done - single analytics write fits easily in 30s
    ctx.waitUntil(env.ANALYTICS.writeDataPoint(metrics));
  }
});

response.body.pipeTo(writable);
return new Response(readable, response);

This way the metrics processing happens as data flows through rather than in a separate waitUntil pipeline. The flush() fires when the stream closes, and a single writeDataPoint call should fit comfortably within the timeout.

Based on reading the workerd source - I haven't run this against the production runtime, but the code path is fairly clear. Let me know if that lines up with what you're seeing.

[cometkim]

@nathanschram Thanks, I'll look to see if this works.

However, I didn't want to add anything that could block existing request-response flow, even I did wrong something.

Not waiting for an stream end seems like a major limitation, as it's hard to anticipate that waitUntil might terminate early during an response. Of course, I could always move the runtime to a durable object, but I don't want to add unnecessary complexity here...

[nathanschram]

Yeah that's a fair concern. The TransformStream itself won't block the response - transform() enqueues chunks synchronously so data flows through unchanged. The risk is more about timing.

Since the 30s drain clock starts from handler return, for longer SSE streams the flush() at the end could be racing the timeout (or potentially past it). One way around that would be to write metrics in batches inside transform() - say every 50 events - rather than accumulating for flush(). Each writeDataPoint call is lightweight so the individual waitUntil calls should complete quickly.

The underlying limitation is real though - a "stream-aware" waitUntil that delays the timer until stream completion would make patterns like yours much more natural. Might be worth raising as a feature request on the workerd repo.

[cometkim]

Found a similar issue from SO: https://stackoverflow.com/questions/75243797/in-a-cloudflare-worker-why-the-faster-stream-waits-the-slower-one-when-using-the

[nathanschram]

Nice find - that SO answer is actually from Kenton Varda (workerd core team) and lines up with what we traced in the source. The tee() backpressure to the slower consumer is by design (otherwise the worker would need to buffer potentially huge amounts of data in 128MB of memory), and the waitUntil timeout starting from handler return rather than stream completion is the same behaviour we saw in the drain() path via kj::defer.

The TransformStream workaround Kenton describes there is essentially the same approach as the inline processing pattern - it forces the runtime to keep the worker "in-use" rather than detecting a pass-through and delegating the stream pumping to a different system.

Sounds like this timeout behaviour has been a known issue for a while now. Might be worth adding a +1 or link to this thread on any existing feature request for stream-aware waitUntil, if one exists.