all Python 3multiprocessingterm

multiprocessing.JoinableQueue

communication

A process queue that tracks finished work: the consumer calls task_done() for each get(), and join() waits until every item put has been marked done.

Syntax

q = ctx.JoinableQueue(); q.put(x); q.get(); q.task_done(); q.join()

Forget one task_done() and q.join() hangs forever without any error; one call too many raises ValueError. task_done() must also be called for the poison pill (None) used to stop the consumer, or the count never matches. As with Queue, the consumer is usually an endless loop, so stop it after q.join() with a sentinel or daemon=True.

Examples

import multiprocessing as mp
def consumer(q, out):
    while True:
        item = q.get()
        if item is None:
            q.task_done()
            break
        out.put(item * 10)
        q.task_done()
if __name__ == '__main__':
    ctx = mp.get_context('spawn')
    q, out = ctx.JoinableQueue(), ctx.Queue()
    p = ctx.Process(target=consumer, args=(q, out))
    p.start()
    for item in (1, 2, 3, None):
        q.put(item)
    q.join()  # ждёт task_done() на каждый put()
    print(sorted(out.get(timeout=10) for _ in range(3)))  # → [10, 20, 30]
    p.join()

See also

Open in the full glossarydocs.python.orgReport an inaccuracy