In a pandas data frame, if there have entries being equal to -inf or NaN, how to remove all the related columns.
>Solution :
If it doesn’t matter that you retain other null like values (e.g. None, NaT) then replace -inf with nan and drop columns with any null values (default value of how arg of dropna is "any")
import numpy as np
import pandas as pd
df = pd.DataFrame({
'a': [1,2,np.nan],
'b': [1,-np.inf,3],
'c': [1,2,3]
})
df = df.replace({-np.inf: np.nan}).dropna(axis=1)
df
Output:
c
0 1
1 2
2 3
Otherwise I’d do something like this:
import numpy as np
import pandas as pd
TARGET_VALUES = [-np.inf, np.nan]
df = pd.DataFrame({
'a': [1,2,np.nan],
'b': [1,-np.inf,3],
'c': ['a','b',None]
})
column_value_counts = df.isin(TARGET_VALUES).sum(axis=0)
df = df[column_value_counts[column_value_counts == 0].index.values].copy()
df
Output:
c
0 a
1 b
2 None