Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

get intersection in single column of pandas dataframe

I have a dataframe like this:

>>> df = pd.DataFrame({
    'Date': [
        pd.to_datetime("2022-07-01"),
        pd.to_datetime("2020-07-02"),
        pd.to_datetime("2020-07-03"),
        ],
    "Price": [
        {24.9, 23.0, 22.5, 23.5},
        {24.9, 25.0, 26.5, 23.7},
        {25.2, 24.5, 23.6, 23.8},
  ]})

>>> df
        Date                     Price
0 2022-07-01  {24.9, 23.5, 22.5, 23.0}
1 2020-07-02  {24.9, 25.0, 26.5, 23.7}
2 2020-07-03  {24.5, 25.2, 23.8, 23.6}

I want to add a new column ‘intersec’ and get the intersection of the Price column and its shift value. But when I use

df['intersec'] = df.price[1:]&df.price.shift()[1:] 

It doesn’t work, I get the following error:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

TypeError: unsupported operand type(s) for &: 'set' and 'bool'

what should I do? My expected result is:

>>> df
       Date                     Price  intersec
0  2022/7/1  {24.9, 23.5, 22.5, 23.0}       NaN
1  2020/7/2  {24.9, 25.0, 26.5, 23.7}      24.9
2  2020/7/3  {24.5, 25.2, 23.8, 23.6}       NaN

>Solution :

shift the price

df["shifted_price"] = df.Price.shift()

find intersect

df["intersec"] = df[1:].apply(lambda x: list(set(x["Price"]) & set(x["shifted_price"])), axis=1)

sample output

        Date                     Price             shifted_price intersec
0 2022-07-01  {24.9, 23.5, 22.5, 23.0}                       NaN      NaN
1 2020-07-02  {24.9, 25.0, 26.5, 23.7}  {24.9, 23.5, 22.5, 23.0}   [24.9]
2 2020-07-03  {24.5, 25.2, 23.8, 23.6}  {24.9, 25.0, 26.5, 23.7}       []
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading