asyncio timeout: your thread may still be running

An asyncio timeout can end an awaiting request while its blocking worker thread keeps running. If you offload a synchronous SDK call, treat the caller deadline and the worker lifetime as separate controls. Give the SDK its own bounds and release concurrency capacity only when the submitted call actually finishes. The executor cannot cancel a function that is already running. ^[1][2]

Scenario: a report endpoint calls a blocking export library. The browser disconnects, the request task is cancelled, and the endpoint stops waiting. The export library still reads a remote response and writes its result. A second request enters the same path. Whether that second request may start should depend on the export work still occupying capacity, rather than the disappearance of its first caller.

Why did the timeout leave a thread running?#

Because the awaitable and the ordinary function have different lifetimes. asyncio.to_thread() returns a coroutine that exposes the eventual function result; it makes the blocking function awaitable without moving that function into the event loop. Cancelling the waiting task does not add cancellation checks inside the blocking function. Python documents that executor futures already running cannot be cancelled through executor shutdown cancellation. ^[1][2]

async with asyncio.timeout(caller_budget):
    result = await asyncio.to_thread(blocking_export)
# TimeoutError here does not prove blocking_export has stopped.

There may be several outcomes after the caller gives up: a successful result nobody consumes, an exception that arrives later, a remote operation that commits, or a worker stuck inside an unbounded library call. Those outcomes require different handling. Converting all of them into a generic request timeout loses information the service needs for capacity accounting and recovery.

Keep the caller timeout: it lets the endpoint stop waiting for a response it can no longer use. Treat it as a response policy, then check worker completion separately before returning the capacity permit. For side-effecting work, it also says nothing conclusive about whether the destination applied the action. The retry-budget guide explains how to keep replacement attempts inside a shared deadline rather than starting a fresh budget at every layer. If a worker writes business data before dispatching an event, the transactional outbox pattern can record the database change and dispatch intent together. It does not cancel the thread or make an uncertain remote request safe to repeat.

Where does the semaphore release too early?#

It releases too early when its context follows the awaiting task instead of the submitted function. A semaphore around to_thread() looks like a concurrency bound, but cancellation unwinds that context even though the worker can remain busy. The semaphore is then counting interested callers rather than physical work.

# This bounds waiting callers, not necessarily running blocking work.
async def export_with_limit():
    async with export_slots:
        async with asyncio.timeout(caller_budget):
            return await asyncio.to_thread(blocking_export)

Scenario: the endpoint intends to allow one export at a time. Its first caller times out while the export function is still blocked. The context releases its permit, so another caller submits an export. A fixed-size executor may keep the second function queued; a larger shared executor may run it alongside the first. Either way, the application permit no longer describes the unfinished work it was supposed to protect.

A dedicated executor limits running workers and separates this workload from unrelated thread offloads. That is a useful boundary, but it does not by itself provide an application admission deadline or a bound on how many requests your service creates. The repair below bounds submitted calls until completion. The inbound server still needs its own admission policy, because tasks waiting for a permit consume memory too.

A bridge whose permit belongs to the submitted call#

The bridge below transfers permit ownership to the executor future as soon as submission succeeds. Only that future’s completion callback releases the permit. The caller awaits a shielded wrapper, so cancellation of the caller does not cancel the wrapper used for completion accounting. This is an illustrative implementation for ordinary callables on a single event loop; construct it inside that running loop. ^[1][2]

import asyncio
import contextvars
from concurrent.futures import ThreadPoolExecutor


class BoundedThreadBridge:
    """One event loop; graceful close after stopping inbound work."""

    def __init__(self, capacity):
        if capacity < 1:
            raise ValueError('capacity must be positive')
        self.loop = asyncio.get_running_loop()
        self.slots = asyncio.Semaphore(capacity)
        self.pool = ThreadPoolExecutor(max_workers=capacity)
        self.pending = set()
        self.closed = False

    def _finished(self, future):
        self.pending.remove(future)
        self.slots.release()  # Actual executor completion owns release.

    async def call(self, function, *, timeout):
        if self.closed:
            raise RuntimeError('bridge is closing')
        if timeout <= 0:
            raise TimeoutError('no admission budget')
        async with asyncio.timeout(timeout):
            await self.slots.acquire()
            if self.closed:
                self.slots.release()
                raise RuntimeError('bridge is closing')
            try:
                context = contextvars.copy_context()
                future = self.pool.submit(context.run, function)
            except BaseException:
                self.slots.release()
                raise
            self.pending.add(future)
            future.add_done_callback(
                lambda f: self.loop.call_soon_threadsafe(self._finished, f)
            )
            result = asyncio.wrap_future(future)
            # Consume late exceptions; replace with redacted telemetry in service code.
            result.add_done_callback(
                lambda f: None if f.cancelled() else f.exception()
            )
            return await asyncio.shield(result)

    async def close(self):
        self.closed = True
        if self.pending:
            await asyncio.gather(
                *(asyncio.wrap_future(f) for f in tuple(self.pending)),
                return_exceptions=True,
            )
        self.pool.shutdown(wait=True)

Acquire the permit and submit the function without an intervening await. That keeps the ownership transfer in one uninterrupted stretch of event-loop code. If submission raises, the caller returns the permit immediately because there is no executor future to own it. If submission succeeds, the future enters pending before its callback is installed. A function that completes very quickly is still accounted for.

The callback schedules _finished() with call_soon_threadsafe(). An executor callback may run on a worker thread, while the semaphore and pending set belong to the event loop. Python explicitly requires the thread-safe scheduling method when another thread schedules a callback on the loop. Updating those async objects directly from an executor callback would cross the ownership boundary. ^[5]

The implementation copies the current contextvars context for each submission. That preserves request-local context for the function, matching an important convenience of to_thread(). Do not use that copied context as authorization by itself: validate the request before dispatch, and avoid placing secrets in diagnostic context that might later be logged. ^[1]

The late-exception callback deliberately retrieves the wrapper’s exception. It prevents an abandoned wrapper from leaving an unobserved exception, but it is not an observability policy. In service code, record the eventual outcome with an operation identifier and redacted error classification. A timeout response and a later successful worker completion should be distinguishable in your telemetry.

How can a test prove the worker still owns capacity?#

Put a gate inside the blocking function. The test can then hold the worker open after its caller has timed out. The caller must time out after the first function has started, and the gate must remain closed while a replacement caller tries to enter. That ordering exposes the bookkeeping defect more clearly than comparing two approximate sleep durations. threading.Event supports waiting on a flag and wakes its waiting threads when the flag becomes true. ^[3]

Scenario: the bridge has capacity one. The first function sets a started event and waits on a release event with a bounded fixture safety timeout. After its caller times out, a second call has a short admission budget. That second call must time out without its function starting. Finally, the test opens the release gate and drains the bridge. The timing values below are test inputs, not production recommendations or benchmark results.

import asyncio
import threading
from bridge import BoundedThreadBridge

async def demonstrate():
    bridge = BoundedThreadBridge(1)
    started, release, second_started = (
        threading.Event(), threading.Event(), threading.Event()
    )
    def blocking():
        started.set()
        if not release.wait(2):
            raise RuntimeError("fixture release missing")

    first = asyncio.create_task(bridge.call(blocking, timeout=0.1))
    try:
        async with asyncio.timeout(2):
            while not started.is_set():
                await asyncio.sleep(0)
        try:
            await first
        except TimeoutError:
            pass
        assert len(bridge.pending) == 1
        try:
            await bridge.call(second_started.set, timeout=0.02)
        except TimeoutError:
            pass
        assert not second_started.is_set()
    finally:
        release.set()
        await bridge.close()

asyncio.run(demonstrate())

The useful evidence is that the second function never starts while the first function owns capacity. Do not turn this fixture into a claim about an SDK’s cancellation latency. The separate local checks exercised caller cancellation, successful results with copied context, function exceptions, late exceptions after a timeout, and submission failure. Those checks validate this sample’s bookkeeping; they do not contact a network service or establish production throughput.

When adapting the test, make the failure visible. Replace the bridge with a caller-owned semaphore and enough executor capacity, then confirm that the second function can start before the release gate opens. That is the regression the repair should prevent. Keep all fixture waits bounded and release the gate in finally, so a failed assertion does not leave the test process waiting on its own worker.

Which deadlines belong inside the blocking call?#

Every potentially unbounded blocking operation needs a bound that its own library understands. An outer asyncio.timeout() controls the request task. It cannot supply a missing socket read deadline, interrupt an opaque C extension, or impose a total operation budget inside an SDK retry loop. The bridge protects capacity while that work continues; it does not make the work finish.

Inspect the exact client version rather than assuming a parameter named timeout is a total deadline. Work through connection establishment, response reads, paginated operations and SDK retries. A caller deadline shorter than the library’s worst-case work time is a deliberate abandonment policy; if you choose it, reserve capacity until the library actually returns. Do not immediately retry an uncertain write just because the local caller stopped waiting.

If you control the blocking function, add cooperative cancellation between safe units of work with a threading.Event. The function must observe the event itself. Setting a flag cannot interrupt a library call that never returns to your code. Cancellation checks also need a defined result: a partially written file, an incomplete export and a committed remote action cannot all be reported as clean cancellation. ^[3]

A bounded queue can make overload explicit, but choose a positive maxsize. The Python queue documentation states that a nonpositive maximum creates an infinite-size queue. Bound admission before creating an unlimited collection of waiting tasks, decide what rejection means for the caller, and avoid placing the queue behind another unbounded buffer that hides the real backlog. ^[6]

What must finish before the event loop closes?#

The bridge’s submitted functions must finish and their callbacks must run before its event loop closes. Stop inbound admission, let existing callers leave, then await close() while the loop is alive. The sample rejects calls after closing starts and drains its recorded futures before shutting down its dedicated executor. This is graceful drainage, not a hard shutdown deadline.

Python’s runner documentation describes a default-executor shutdown timeout of five minutes for asyncio.run() in the documentation read for this article. That applies to the runner’s default executor; it is not a deadline supplied to this dedicated executor and does not make an arbitrary function cancellable. Never interpret the runner’s shutdown warning as proof that a thread stopped cleanly. ^[4]

If a dependency can block indefinitely and the service requires a hard termination boundary, evaluate a separately supervised process. That changes serialization, startup and cleanup costs. Terminating a process also cannot roll back a remote action it already submitted. The decision depends on the operation’s side effects and the destination’s recovery contract, rather than a preference for one concurrency primitive.

Before rollout, prove both properties: the caller leaves within the response policy, and submitted work keeps its capacity until completion, including during cancellation and shutdown. Record admission waits separately from worker completion and late outcomes. If abandoned work dominates the pool, shorten or repair the blocking client’s own operation policy; increasing thread count can merely allow more abandoned work to accumulate.

Does asyncio.wait_for stop a running thread?

No. It controls the awaitable and requests cancellation on timeout, but a function already executing in an executor is not forcibly stopped. Give the blocking client its own deadlines, or use cooperative cancellation if the function can check a stop signal. ^[1][2]

Does asyncio.shield make blocking work safe?

No. Shielding changes cancellation propagation for the awaited object; it does not add a deadline or stop a thread. In this bridge it preserves the result wrapper while actual executor completion owns the permit. The library’s lifetime and side effects still require separate controls. ^[1][2]

Can I release capacity as soon as the HTTP request disconnects?

Release request-specific resources when appropriate, but keep the worker’s admission permit until the submitted function finishes. Otherwise new requests can enter while abandoned work still uses the dependency. A disconnected client is evidence about the caller, not the completion of a remote operation.

What happens to exceptions after the caller times out?

They can arrive after the request has ended. The sample retrieves late wrapper exceptions so they are observed, but production code should also emit redacted outcome telemetry. Include an operation identifier so a late success or error can be associated with the original request.

Should I replace every to_thread call with this bridge?

No. Use the extra machinery where a dependency needs a dedicated admission policy or can outlive its callers materially. Short bounded utility calls may not justify it. A suitable native async client can reduce thread lifecycle code, but its cancellation and remote-effect semantics still need verification.