Convert a list of lists containing a dictionary to dataframe

August 25, 2022

I have the following output from a model I built

test = [
    [
        {"label": "positive", "score": 0.005163147579878569},
        {"label": "negative", "score": 0.0949820727109909},
        {"label": "neutral", "score": 0.8998547792434692}
    ],
    [
        {"label": "positive", "score": 0.8533585667610168},
        {"label": "negative", "score": 0.13094310462474823},
        {"label": "neutral", "score": 0.01569831557571888}
    ],
    [
        {"label": "positive", "score": 0.007672784384340048},
        {"label": "negative", "score": 0.9619094133377075},
        {"label": "neutral", "score": 0.030417803674936295}
    ],
    [
        {"label": "positive", "score": 0.007140590343624353},
        {"label": "negative", "score": 0.9494256973266602},
        {"label": "neutral", "score": 0.04343372955918312}
    ]
]

I want to convert this output to a dataframe with columns positive, negative, neutral and rows their respective score. For example, for the first dictionary in my list of lists the desired output is:

 Positive                           Negative                     Neutral
 0.005163147579878569               0.0949820727109909           0.8998547792434692

I used the following function to convert it to a dataframe, but I can’t set the label as columns and the score as rows

df = pd.DataFrame(test).stack().apply(pd.Series)

>Solution :

You could use a dictionary comprehension to get a more pandas friendly structure and then construct the dataframe:

pd.DataFrame(({d['label']:d['score'] for d in subl} for subl in test))

   positive  negative   neutral
0  0.005163  0.094982  0.899855
1  0.853359  0.130943  0.015698
2  0.007673  0.961909  0.030418
3  0.007141  0.949426  0.043434