I’m trying to explode dataframe that have multiple columns as list and list of length is different. Below is my dataframe:
import pandas as pd
import numpy as np
df = pd.DataFrame({'Name': ['Name1','Name2'],
'Time': [[0.0, 5.0, 10.0, 15.0],[0.0, 4.0, 28.0, 48.0]],
'Values': [[0.0, 5.0, 10.0],[0.0, 4.0, 48.0]]})
df
Name Time Values
0 Name1 [0.0, 5.0, 10.0, 15.0] [0.0, 5.0, 10.0]
1 Name2 [0.0, 4.0, 28.0, 48.0] [0.0, 4.0, 48.0]
I tried with df2 = df.explode(['Time','Values']) but it raised error that said ValueError: columns must have matching element counts.
Below is my expected Output:
df2 = pd.DataFrame({'Name': ['Name1','Name1','Name1','Name1','Name2','Name2' ,'Name2' ,'Name2'],
'Time': [0.0, 5.0, 10.0, 15.0,0.0, 5.0, 10.0, np.nan],
'Values': [0.0, 4.0, 28.0, 48.0, 0.0, 4.0, 48.0, np.nan]})
df2
Name Time Values
0 Name1 0.0 0.0
1 Name1 5.0 4.0
2 Name1 10.0 28.0
3 Name1 15.0 48.0
4 Name2 0.0 0.0
5 Name2 5.0 4.0
6 Name2 10.0 48.0
7 Name2 NaN NaN
Thank and best regards
>Solution :
Here is trick with GroupBy.cumcount for create unique MultiIndex for exploded values:
cols = ['Time','Values']
df1 = pd.concat([df[x].explode().to_frame()
.assign(g=lambda x: x.groupby(level=0).cumcount())
.set_index('g', append=True)
for x in cols], axis=1)
df = df.drop(cols, axis=1).join(df1.droplevel(1))
print (df)
Name Time Values
0 Name1 0.0 0.0
0 Name1 5.0 5.0
0 Name1 10.0 10.0
0 Name1 15.0 NaN
1 Name2 0.0 0.0
1 Name2 4.0 4.0
1 Name2 28.0 48.0
1 Name2 48.0 NaN