warming up your workspace

Processes vs threads vs async, explained by running the same job three ways

"Just make it concurrent" hides a real choice with three answers: processes, threads, and async. Pick wrong and your program gets slower, not faster. The difference is not style, it is about what your work is actually waiting on. Run the same job three ways and the rule falls out on its own.

The one idea

Concurrency is about doing something useful while waiting. The question is what you wait on, and it splits work into two kinds. CPU-bound work is limited by computation: hashing, crunching numbers, the CPU never idles. I/O-bound work is limited by waiting: for a network reply, a disk, a database, the CPU is idle most of the time. The three tools map cleanly onto this split, and mismatching them is where performance goes to die.

Threads, and the wall Python hits on CPU work

A thread is a strand of execution inside one process, sharing its memory. Threads look perfect for parallelism, but CPython has a Global Interpreter Lock (GIL): only one thread runs Python bytecode at a time. For CPU-bound work that means threads take turns, not run together.

import time
from concurrent.futures import ThreadPoolExecutor

def burn(n):                      # pure CPU work
    return sum(i * i for i in range(n))

N, TASKS = 5_000_000, 4
start = time.time()
with ThreadPoolExecutor(max_workers=4) as ex:
    list(ex.map(burn, [N] * TASKS))
print(f"threads: {time.time() - start:.2f}s")   # ~ same as sequential

Four threads on four "cores" and it is barely faster than doing them one after another, because the GIL serialized them. Threads did not help CPU work.

Processes, which actually parallelize CPU work

A process has its own memory and its own interpreter, so its own GIL. Run four processes and four cores genuinely work at once.

from concurrent.futures import ProcessPoolExecutor

start = time.time()
with ProcessPoolExecutor(max_workers=4) as ex:
    list(ex.map(burn, [N] * TASKS))
print(f"processes: {time.time() - start:.2f}s")   # ~ 4x faster on 4 cores

On a four-core machine this finishes in roughly a quarter of the time. Processes are the answer for CPU-bound work. The cost: they do not share memory, so passing data means copying it, and starting them is heavier than a thread.

Async, which shines when you are only waiting

Now change the workload to I/O-bound: mostly waiting, like a network call (we simulate it with asyncio.sleep, which yields instead of blocking).

import asyncio

async def fetch(i):
    await asyncio.sleep(1)        # stands in for a 1s network reply
    return i

async def main():
    start = asyncio.get_event_loop().time()
    await asyncio.gather(*(fetch(i) for i in range(100)))
    print(f"async: {asyncio.get_event_loop().time() - start:.2f}s")

asyncio.run(main())               # ~ 1.0s, not 100s

A hundred one-second waits finish in about one second, on a single thread. Async does not use extra cores; it uses the gaps. While one call waits, the event loop runs another. For thousands of concurrent waits, this is dramatically lighter than a thousand threads or processes.

The rule, and why it is the rule

  • CPU-bound (compute): use processes. Only true parallelism beats the GIL.
  • I/O-bound (waiting): use async for lots of concurrent waits (cheap, one thread), or threads for a modest number, or when the library is blocking and has no async version.
  • Sharing lots of data cheaply favors threads (shared memory); isolation and crash-safety favor processes.

The mistake people make is reaching for threads because they are familiar, then wondering why the number-crunching did not speed up. It could not: the GIL was never going to let it. Match the tool to what the work waits on, and concurrency starts paying off instead of adding complexity. That match, work out what you are waiting on first, is a habit the general coding track builds deliberately, because it is the difference between concurrency that helps and concurrency that just adds bugs.