Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Query the non-nan value from a pivoted df

I have a pivoted df:

data = np.column_stack([["alpha", "beta", "gamma", "delta"], ["a", "b", "c", "d"], [0, 1, 2, 3]])
df = pn.DataFrame(data, columns=["greek", "latin", "values"])
piv = df.pivot(index = "greek", columns="latin", values="values")

and would like to access piv‘s values by name, so I figured .loc is what I need.
Passing piv.loc["gamma", "c"] works as intended, but what if I wanted to access piv in a loop where I am iterating on random combinations of the greek and latin column names? In that case, one of the two combinations would return NaN.

In other words, is there a way to have .loc retrieve the non-nan value of a given combination of row/column names?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

If you want to avoid NaNs in general, just stack, this will drop all the NaNs:

tmp = piv.stack()

Output:

greek  latin
alpha  a        0
beta   b        1
delta  d        3
gamma  c        2
dtype: object

Then you can slice directly:

tmp.loc[('alpha', 'a')]

Or, to handle possibly missing combinations:

tmp.get(('alpha', 'a'), 'missing')

Output: 0

Note that if you want a random item, no need to know the indices, just sample:

tmp = piv.stack()
chosen = tmp.sample(1)
chosen.index[0]
# ('beta', 'b')

chosen.squeeze()
# 1

Or for multiple values at once:

tmp.sample(5, replace=True)

Output:

greek  latin
delta  d        3
alpha  a        0
       a        0
gamma  c        2
beta   b        1
dtype: object

If you have arbitrary pairs in any order, the best would be to design your loop to provide the combination in the correct order.

Now, assuming that you can’t, you could use a try except:

idx, col = 'a', 'alpha'
try:
    piv.loc[idx, col]
except KeyError:
    piv.loc[col, idx]

Alternatively, stack again and make your index a frozenset:

idx, col = 'a', 'alpha'

tmp = piv.stack()
tmp.index = tmp.index.map(frozenset)
tmp.get(frozenset((idx, col)), None)

Output: 0

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading