Home How use multiprocessing.Pool.map with iterator and separat input for function

Questions

How use multiprocessing.Pool.map with iterator and separat input for function

December 15, 2021

How can I pass complete parameters, without splitting them among the cores, to function myfun and at the same time pass the respective element from the collection [1,2,3,4,5]

p=Pool(5)
p.map(myfun(df_A,df_B),[1,2,3,4,5])

If I implement it this way, the function gets the parameters df_A and df_B but not an element from the collection

Here is an example how myfun can look like:

def myfunc(df_A, df_B, e):
    do_something
    print(df_A.iloc[e],df_A.iloc[e])

e is one element of the collection [1,2,3,4,5]

>Solution :

You could consider something like this (there are probably better ways):

from multiprocessing import Pool

def myfunc(a, b, c):
    print(a, b, c)

df_A = 1
df_B = 2

def main():
    with Pool() as pool:
        pool.starmap(myfunc,[[df_A, df_B, x] for x in range(1, 6)])

if __name__ == '__main__':
    main()

Output: