Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

question when using interpolate for missing value

I had a dataset looks like below:

import pandas as pd
test = pd.DataFrame({
    'date': ['2018-01-03 00:00:00', '2018-01-04 00:00:00', '2018-01-05 00:00:00'],
    'coal': [2669.0, np.nan ,2731.0],
     'hydro': [222.0, np.nan ,230.0],
    'unit': ['Gwh', 'Gwh', 'Gwh'],
})

test['date'] = pd.to_datetime(test['date'])

and when i was trying to fill the null by using interpolate method:

for x in test.columns.to_list():
    test[x] = test[x].interpolate())

it got error :

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Invalid fill method. Expecting pad (ffill) or backfill (bfill). Got linear

and when i remove the

test['date'] = pd.to_datetime(test['date'])

it will work just fine, so i dont know why this happened:(

and also, it works fine when i using the sample df, but when i try it on my own dataset, it has no error, but the null value still exist, like the fillna() never used:(

and when i was using below code, then use the fillna code:

test['date'] = test['date'].astype(object)

it works on my sample, but not my own dataset:( my own dataset still like i never used the fillna method

im so confused by now:( was wondering if someone could explain why this happened?
I try to google it, but no result:(

or maybe i dont know how to google it:(

P.S.
it works fine if i fill it one column at a time when i try it on my own dataset, like:

test['coal'] = test['coal'].interpolate())
test['hydro'] = test['hydro'].interpolate())

but not the for loops:(

Its working when i first change the dtype to float, so maybe something wrong when it was object(not all object)?

for x in total_list:
    df_all[x] = df_all[x].astype(float)
    df_all[x] = df_all[x].interpolate()

anyway, thank you all:) this took me 2hours already:(

>Solution :

You just need to interpolate the given columns

cols_to_interpolate = ["coal", "hydro"]
test[cols_to_interpolate] = test[cols_to_interpolate].interpolate()
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading