Number of cores a Python script is using

June 11, 2023

I am using multiprocessing module to parallelize the script but not sure if it is using all the available cores. Is there a straightforward way to find it?

import numpy as np
import multiprocessing
import time

def fibonacci(n):
    sequence = [0, 1]
    while len(sequence) < n:  
        next_number = sequence[-1] + sequence[-2]  
        sequence.append(next_number)  
    return np.array(sequence)  # Convert the sequence to a NumPy array

if __name__ == '__main__':
    start_time = time.time()
    pool = multiprocessing.Pool()

    last_result = None
    n = 100
    for i in range(0, n): 
        result = fibonacci(i)
        #print([result])
    
    U = time.time() - start_time
    print("Elapsed time =", U)

>Solution :

The main code to use all cores may look something like:

if __name__ == '__main__':
    start_time = time.time()
    with multiprocessing.Pool() as pool:
        results = pool.map(fibonacci, range(0, n))

    result = results[-1]
    
    U = time.time() - start_time
    print("Elapsed time =", U)

Be aware that this is not necessarily faster than the single process. Due to the overhead of multiprocessing it may need some fine tuning (especially by setting a chunk size for map).