Home python dataframe timeseries check if value changed more than x in last n rows and forward n rows

Questions

python dataframe timeseries check if value changed more than x in last n rows and forward n rows

June 3, 2022

I have a sunlight data coming from the field. I am checking if the sunlight changed more than a value in last 1 min and future 1 min. Below I am giving an example case. Where I am checking if the data value changed more than 4 in the last 10s.
code:

xdf = pd.DataFrame({'data':np.random.randint(10,size=10)},index=pd.date_range('2022-06-03 00:00:00', '2022-06-03 00:00:45', freq='5s'))
# here data frequency 5s, so, to check last 10s
# I have to consider present row and last 2 rows
# Perform rolling max and min value for 3 rows
nrows = 3
# Allowable change
ac = 4
xdf['back_max'] = xdf['data'].rolling(nrows).max()
xdf['back_min'] = xdf['data'].rolling(nrows).min()
xdf['back_min_max_dif'] = (xdf['back_max'] - xdf['back_min'])
xdf['back_<4'] = (xdf['back_max'] - xdf['back_min']).abs().le(ac)
print(xdf)

## Again repeat the above for the future nrows
## Don't know how?

expected output:

                     data  back_max  back_min  back_min_max_dif  back_<4
2022-06-03 00:00:00     7       NaN       NaN               NaN    False
2022-06-03 00:00:05     7       NaN       NaN               NaN    False
2022-06-03 00:00:10     5       7.0       5.0               2.0     True
2022-06-03 00:00:15     8       8.0       5.0               3.0     True
2022-06-03 00:00:20     6       8.0       5.0               3.0     True
2022-06-03 00:00:25     2       8.0       2.0               6.0    False
2022-06-03 00:00:30     3       6.0       2.0               4.0     True
2022-06-03 00:00:35     1       3.0       1.0               2.0     True
2022-06-03 00:00:40     5       5.0       1.0               4.0     True
2022-06-03 00:00:45     5       5.0       1.0               4.0     True

Is there way I can simplify the above procedure? Also, I have to perform rolling max for future nrows, and how?

>Solution :

For future/forward roll, you can roll on the reversed data. This might not work with time-window roll:

rolling = xdf['data'].rolling(nrows)
xdf['pass_<'] = (rolling.max()-rolling.min()).le(ac)

future_roll = xdf['data'][::-1].rolling(nrows)
xdf['future_<'] = future_roll.max().sub(future_roll.min()).le(ac)

Output:

                     data  pass_<  future_<
2022-06-03 00:00:00     7   False      True
2022-06-03 00:00:05     7   False      True
2022-06-03 00:00:10     5    True      True
2022-06-03 00:00:15     8    True     False
2022-06-03 00:00:20     6    True      True
2022-06-03 00:00:25     2   False      True
2022-06-03 00:00:30     3    True      True
2022-06-03 00:00:35     1    True      True
2022-06-03 00:00:40     5    True     False
2022-06-03 00:00:45     5    True     False