How to run two functions in parallel in Python?

October 22, 2022

I am trying to run two chunks of code in parallel, but fail to do so. Both functions are for loops that might take around 30 minutes to run in total, these functions in the end return a list of dictionaries as results. Running them separately works fine, but I can’t get them to run in parallel…

#When I run these functions separately, it sort of looks like this:
import time

def functionA(A, B):
    dictA=[]
    for i in list(range(A, B)):
        print(i, "from A")
        time.sleep(1)
        for p in list(range(0, 10)):
            dictA.append({i:p})
    return(dictA)

def functionB(C, D):
    dictB=[]
    for i in list(range(C, D)):
        print(i, "from B")
        time.sleep(1)
        for p in list(range(0, 10)):
            dictB.append({i:p})
    return(dictB)

DictA          = functionA(0, 10)
DictB          = functionB(10, 20)

#I have tried to run them in parallel, but they still run separately, and in the end the variable I would like to write to is just a thread:
import threading
e = threading.Event()
DictA  = threading.Thread(target=functionA(0, 10))
DictA.start()               
DictB  = threading.Thread(target= functionB(10, 20))
DictB.start()
#Further pieces of code to be executed after threading completed

This does not make the process run in parallel but rather in logical sequence as can be seen from the print statements:

0 from A
1 from A
2 from A
3 from A
4 from A
5 from A
6 from A
7 from A
8 from A
9 from A
Exception in thread Thread-12:
Traceback (most recent call last):
  File "/home/maestro/.pyenv/versions/3.9.13/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/maestro/.pyenv/versions/3.9.13/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
TypeError: 'list' object is not callable
10 from B
11 from B
12 from B
13 from B
14 from B
15 from B
16 from B
17 from B
18 from B
19 from B
Exception in thread Thread-13:
Traceback (most recent call last):
  File "/home/maestro/.pyenv/versions/3.9.13/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/maestro/.pyenv/versions/3.9.13/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
TypeError: 'list' object is not callable

How can these functions be run in parallel?

>Solution :

Your problem is here:

DictA  = threading.Thread(target=functionA(0, 10))

Here you’re not passing functionA as the argument to target; you’re calling functionA and passing the value to target. That’s why you’re functions appear to be executing synchronously rather than in parallel — because that’s how you’re calling them.

You want:

DictA = threading.Thread(target=functionA, args=(0, 10))

A runnable version of your code looks like this:

import threading
import time


def functionA(A, B):
    dictA = []
    for i in list(range(A, B)):
        print(i, "from A")
        time.sleep(1)
        for p in list(range(0, 10)):
            dictA.append({i: p})
    return dictA


def functionB(C, D):
    dictB = []
    for i in list(range(C, D)):
        print(i, "from B")
        time.sleep(1)
        for p in list(range(0, 10)):
            dictB.append({i: p})
    return dictB


DictA = threading.Thread(target=functionA, args=(0, 10))
DictA.start()
DictB = threading.Thread(target=functionB, args=(10, 20))
DictB.start()

# Calling `.join()` on each thread here to wait for them to complete.
DictA.join()
DictB.join()

Running that code produces as output:

0 from A
10 from B
11 from B
1 from A
2 from A
12 from B
3 from A
13 from B
4 from A
14 from B
5 from A
15 from B
6 from A
16 from B
7 from A
17 from B
18 from B
8 from A
9 from A
19 from B