all Python 3multiprocessingterm

multiprocessing.Queue

communication

A queue for passing objects between processes: put() in one, get() in another. Objects are pickled and sent by a background thread.

Syntax

q = ctx.Queue(maxsize=0); q.put(obj); q.get(block=True, timeout=None)

put() returns at once while a background thread pickles and sends the object, so a "cannot pickle" error is not raised by put() but printed to stderr, and the item silently disappears. A child that put a lot of data will not exit until it is consumed: join() before get() is the classic deadlock, so read first. qsize() raises NotImplementedError on macOS and empty() is unreliable; get(timeout=...) raises queue.Empty from the regular queue module.

Examples

import multiprocessing as mp
def worker(q, n):
    q.put(n * n)
if __name__ == '__main__':
    ctx = mp.get_context('spawn')
    q = ctx.Queue()
    ps = [ctx.Process(target=worker, args=(q, n)) for n in (2, 3)]
    for p in ps:
        p.start()
    results = sorted(q.get(timeout=10) for _ in ps)  # сначала читаем, потом join
    for p in ps:
        p.join()
    print(results)  # → [4, 9]

import multiprocessing as mp
import queue
if __name__ == '__main__':
    q = mp.get_context('spawn').Queue()
    q.put(lambda: 1)  # pickle упадёт в фоновом потоке, не здесь
    print('put returned')  # → put returned
    try:
        q.get(timeout=0.5)
    except queue.Empty:
        print('nothing arrived')  # → nothing arrived

See also

Open in the full glossarydocs.python.orgReport an inaccuracy