Numba: indexing a vector is giving an error

March 26, 2022

I started using python and numba recently. My problem is: I have a matrix (n rows and m columns).In a for loop I have to change the values of specific columns.

Without numba, the code is running fine. But when I use njit(), it just crashes.

Note: In my real project, each row don’t have the same values.

This is an example of what I need.

import numpy as np


def func_test(matrix_size, columns_idx, replace_values):
    matrix = np.zeros((matrix_size, matrix_size), dtype='int32')

    for i in range(0, matrix_size):
        matrix[i, columns_idx] = replace_values
    return matrix


columns_idx = [0,2,4]
replace_values = [1, 3, 5]

new_matrix = func_test(5, columns_idx, replace_values)
print(new_matrix)

[[1 0 3 0 5]
 [1 0 3 0 5]
 [1 0 3 0 5]
 [1 0 3 0 5]
 [1 0 3 0 5]]

Now, I will apply Numba.

import numpy as np
from numba import njit


@njit()
def func_test(matrix_size, columns_idx, replace_values):
    matrix = np.zeros((matrix_size, matrix_size), dtype='int32')

    for i in range(0, matrix_size):
        matrix[i, columns_idx] = replace_values
    return matrix


columns_idx = [0,2,4]
replace_values = [1, 3, 5]

new_matrix = func_test(5, columns_idx, replace_values)
print(new_matrix)

And this error appears:

No implementation of function Function(<built-in function setitem>) found for signature:

\>>> setitem(array(int32, 2d, C), Tuple(int64, reflected list(int64)<iv=None>), reflected list(int64)<iv=None>)

There are 16 candidate implementations:

Of which 14 did not match due to:
Overload of function ‘setitem’: File: <numerous>: Line N/A.
With argument(s): ‘(array(int32, 2d, C), Tuple(int64, reflected list(int64)<iv=None>), reflected list(int64)<iv=None>)’:
No match.

Of which 2 did not match due to:
Overload in function ‘SetItemBuffer.generic’: File: numba/core/typing/arraydecl.py: Line 176.
With argument(s): ‘(array(int32, 2d, C), Tuple(int64, reflected list(int64)<iv=None>), reflected list(int64)<iv=None>)’:
Rejected as the implementation raised a specific error:
NumbaTypeError: unsupported array index type reflected list(int64)<iv=None> in Tuple(int64, reflected list(int64)<iv=None>)
raised from /Users/goncaloguedes/.conda/envs/Python_Fold_UnFold/lib/python3.9/site-packages/numba/core/typing/arraydecl.py:72

During: typing of setitem at /Users/goncaloguedes/Desktop/Fold_Unfold/Python_Fold_UnFold/Testes.py (10)

File "Testes.py", line 10:
def func_test(matrix_size, columns_idx, replace_values):
<source elided>
for i in range(0, matrix_size):
matrix[i, columns_idx] = replace_values

>Solution :

Numba does not supports well reflected list (CPython lists with objects of possibly different types). You need either to use typed lists or Numpy arrays. The reason is that Numba need to work on well-defined typed variables and not on dynamically-typed objects so to be efficient (and actually be able to compile the code in a native binary). In your case, this is better to use Numpy arrays.

Additionally, it appear Numba is not able to compile a line like matrix[i, columns_idx] = replace_values. Not all Numpy features are supported by Numba yet. Thus, you need to use a basic loop instead.

Here is an example of corrected code:

import numpy as np
from numba import njit


@njit()
def func_test(matrix_size, columns_idx, replace_values):
    matrix = np.zeros((matrix_size, matrix_size), dtype=np.int32)

    for i in range(0, matrix_size):
        for j in range(0, columns_idx.size):
            matrix[i, columns_idx[j]] = replace_values[j]
    return matrix


columns_idx = np.array([0, 2, 4], np.int64)
replace_values = np.array([1, 3, 5], np.int32)

new_matrix = func_test(5, columns_idx, replace_values)
print(new_matrix)