Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

How to shift all columns through a loop in Pandas?

I’m new to Python and Pandas and I hope that someone can help me. I have the following Dataframe:

import pandas as pd
from pandas import Timestamp

pd.DataFrame({'timestamp': {0: Timestamp('2021-06-01 00:00:00'),
  1: Timestamp('2021-06-01 01:00:00'),
  2: Timestamp('2021-06-01 02:00:00'),
  3: Timestamp('2021-06-01 03:00:00'),
  4: Timestamp('2021-06-01 04:00:00')},
 'column_0': {0: 384.0,
  1: 389.0,
  2: 352.0,
  3: 352.0,
  4: 356.0},
 'column_1': {0: 386.0,
  1: 352.0,
  2: 352.0,
  3: 356.0,
  4: 375.0},
 'column_2': {0: 352.0,
  1: 352.0,
  2: 356.0,
  3: 375.0,
  4: 365.0},
 'column_3': {0: 350.0,
  1: 356.0,
  2: 375.0,
  3: 365.0,
  4: 358.0},
 'column_4': {0: 359.0,
  1: 375.0,
  2: 365.0,
  3: 358.0,
  4: 347.0})
            timestamp  column_0  column_1  column_2  column_3  column_4
0 2021-06-01 00:00:00     384.0     386.0     352.0     350.0     359.0
1 2021-06-01 01:00:00     389.0     352.0     352.0     356.0     375.0
2 2021-06-01 02:00:00     352.0     352.0     356.0     375.0     365.0
3 2021-06-01 03:00:00     352.0     356.0     375.0     365.0     358.0
4 2021-06-01 04:00:00     356.0     375.0     365.0     358.0     347.0

What I want to do is to shift the numbers in column_1 down one row, in column_2 down two rows, in column_3 down 3 rows and so on.

I know that this can be done with the code

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

df['column_1'].shift(+1)
df['column_2'].shift(+2)
...

The manual method is too troublesome since there are actually over 40 such columns in total. That’s why I would like to add a loop, so that the code doesn’t get too long.
In the end the dataframe should look like this:

            timestamp  column_0  column_1  column_2  column_3  column_4
0 2021-06-01 00:00:00     384.0     NaN       NaN       NaN       NaN
1 2021-06-01 01:00:00     389.0     386.0     NaN       NaN       NaN
2 2021-06-01 02:00:00     352.0     352.0     352.0     NaN       NaN
3 2021-06-01 03:00:00     352.0     352.0     352.0     350.0     NaN  
4 2021-06-01 04:00:00     356.0     356.0     356.0     356.0     359.0
...

Does someone have an idea how to do that? I’m not really skilled in programming and my attempts so far have failed. I would be very grateful for any help.

>Solution :

You can access the ordered sequence of column names using df.columns. From there it is quick to iterate over them and shift the column with the help of enumerate. Since your first column is your timestamp, set the enumerate start value to -1 to align the column shift.

for i, col in enumerate(df.columns, -1):
    if i > 0:
        df[col] = df[col].shift(i)
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading