Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

drop dataframe rows that contain entries being either -inf or NaN

In a pandas data frame, if there have entries being equal to -inf or NaN, how to remove all the related columns.

>Solution :

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

If it doesn’t matter that you retain other null like values (e.g. None, NaT) then replace -inf with nan and drop columns with any null values (default value of how arg of dropna is "any")

import numpy as np
import pandas as pd

df = pd.DataFrame({
    'a': [1,2,np.nan],
    'b': [1,-np.inf,3],
    'c': [1,2,3]
})
df = df.replace({-np.inf: np.nan}).dropna(axis=1)
df

Output:

    c
0   1
1   2
2   3

Otherwise I’d do something like this:

import numpy as np
import pandas as pd

TARGET_VALUES = [-np.inf, np.nan]

df = pd.DataFrame({
    'a': [1,2,np.nan],
    'b': [1,-np.inf,3],
    'c': ['a','b',None]
})
column_value_counts = df.isin(TARGET_VALUES).sum(axis=0)
df = df[column_value_counts[column_value_counts == 0].index.values].copy()
df

Output:

    c
0   a
1   b
2   None
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading