Reshaping numpy array raises ValueError

byMR

March 20, 2023

I have a NumPy array as follows:

arr = np.array([np.zeros(s) for s in range(2, 10)])

I want to reshape each subarray form shape (s) to shape (1, s), so i wrote this:

arr = np.array([np.zeros(s).reshape(1, s) for s in range(2, 10)])

However, ValueError is raised:

ValueError: could not broadcast input array from shape (2,) into shape (1,)

How can I fix this?

>Solution :

In numpy 1.23

Your list comprehension produces arrays that vary in size:

In [87]: [np.zeros(s).reshape(1, s) for s in range(2, 10)]
Out[87]: 
[array([[0., 0.]]),
 array([[0., 0., 0.]]),
 array([[0., 0., 0., 0.]]),
 array([[0., 0., 0., 0., 0.]]),
 array([[0., 0., 0., 0., 0., 0.]]),
 array([[0., 0., 0., 0., 0., 0., 0.]]),
 array([[0., 0., 0., 0., 0., 0., 0., 0.]]),
 array([[0., 0., 0., 0., 0., 0., 0., 0., 0.]])]

Trying to make an array from that produces a warning (did you see this?), and an error:

In [88]: np.array(_)
C:\Users\paul\AppData\Local\Temp\ipykernel_6648\2978863899.py:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  np.array(_)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[88], line 1
----> 1 np.array(_)

ValueError: could not broadcast input array from shape (2,) into shape (1,)

Even with object dtype we get the error:

In [91]: np.array(_87, dtype=object)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[91], line 1
----> 1 np.array(_87, dtype=object)

ValueError: could not broadcast input array from shape (2,) into shape (1,)

But if we take off the leading size 1 shape, we can make a ‘ragged’ object dtype array:

In [92]: np.array([np.zeros(s).reshape(s) for s in range(2, 10)], object)
Out[92]: 
array([array([0., 0.]), array([0., 0., 0.]), array([0., 0., 0., 0.]),
       array([0., 0., 0., 0., 0.]), array([0., 0., 0., 0., 0., 0.]),
       array([0., 0., 0., 0., 0., 0., 0.]),
       array([0., 0., 0., 0., 0., 0., 0., 0.]),
       array([0., 0., 0., 0., 0., 0., 0., 0., 0.])], dtype=object)

Make an object dtype array from these (1,s) shapes requires a more indirect construction – making a np.empty(n, object) array, and filling that with the list.

In [94]: res = np.empty(8,object); res[:]=_87
In [95]: res
Out[95]: 
array([array([[0., 0.]]), array([[0., 0., 0.]]),
       array([[0., 0., 0., 0.]]), array([[0., 0., 0., 0., 0.]]),
       array([[0., 0., 0., 0., 0., 0.]]),
       array([[0., 0., 0., 0., 0., 0., 0.]]),
       array([[0., 0., 0., 0., 0., 0., 0., 0.]]),
       array([[0., 0., 0., 0., 0., 0., 0., 0., 0.]])], dtype=object)