I have a pivoted df:
data = np.column_stack([["alpha", "beta", "gamma", "delta"], ["a", "b", "c", "d"], [0, 1, 2, 3]])
df = pn.DataFrame(data, columns=["greek", "latin", "values"])
piv = df.pivot(index = "greek", columns="latin", values="values")
and would like to access piv‘s values by name, so I figured .loc is what I need.
Passing piv.loc["gamma", "c"] works as intended, but what if I wanted to access piv in a loop where I am iterating on random combinations of the greek and latin column names? In that case, one of the two combinations would return NaN.
In other words, is there a way to have .loc retrieve the non-nan value of a given combination of row/column names?
>Solution :
If you want to avoid NaNs in general, just stack, this will drop all the NaNs:
tmp = piv.stack()
Output:
greek latin
alpha a 0
beta b 1
delta d 3
gamma c 2
dtype: object
Then you can slice directly:
tmp.loc[('alpha', 'a')]
Or, to handle possibly missing combinations:
tmp.get(('alpha', 'a'), 'missing')
Output: 0
Note that if you want a random item, no need to know the indices, just sample:
tmp = piv.stack()
chosen = tmp.sample(1)
chosen.index[0]
# ('beta', 'b')
chosen.squeeze()
# 1
Or for multiple values at once:
tmp.sample(5, replace=True)
Output:
greek latin
delta d 3
alpha a 0
a 0
gamma c 2
beta b 1
dtype: object
If you have arbitrary pairs in any order, the best would be to design your loop to provide the combination in the correct order.
Now, assuming that you can’t, you could use a try except:
idx, col = 'a', 'alpha'
try:
piv.loc[idx, col]
except KeyError:
piv.loc[col, idx]
Alternatively, stack again and make your index a frozenset:
idx, col = 'a', 'alpha'
tmp = piv.stack()
tmp.index = tmp.index.map(frozenset)
tmp.get(frozenset((idx, col)), None)
Output: 0