I have written a function that reshapes the numpy array per given window length.
The function does the following.
My function as follows:
def w_s(df, l):
"""
Convert numpy array into desired shape with lag 1.
Args:
df (numpy.ndarray): Numpy array.
l (integer): Length of the sample window.
Returns:
Returns numpy array in a desired shape to be used in decision trees.
"""
data = np.zeros((l, 1))
data = np.append(data, df)
data = data[l:]
for i in range(1, l):
s1 = np.roll(df,0-i)
data = np.append(data,s1)
data = data.reshape(l, len(df)).T
return data[:-(l-1)]
I have two arrays with the length of 1780000. The function takes around 3 hours.
CPU times: user 5min 20s, sys: 43min 45s, total: 49min 6s Wall time: 3h 5min 46s
My machine is Mac M1. I am running this on Jupyter cell where server runs on Firefox. How can I do this faster?
>Solution :
That’s sliding_window_view:
>>> import numpy as np
>>> arr = np.arange(10)
>>> np.lib.stride_tricks.sliding_window_view(arr, 5)
array([[0, 1, 2, 3, 4],
[1, 2, 3, 4, 5],
[2, 3, 4, 5, 6],
[3, 4, 5, 6, 7],
[4, 5, 6, 7, 8],
[5, 6, 7, 8, 9]])