how to concat horizontally two dataframes with duplicated indices?

Imagine I want to merge 2 dataframes to give this result:

How can this be achieved in pandas?

>Solution :

Assuming "index" the index, you need to deduplicate the index with groupby.cumcount and concat:

out = pd.concat([
  df1.set_index(df1.groupby(level=0).cumcount(), append=True),
  df2.set_index(df2.groupby(level=0).cumcount(), append=True)
], axis=1).droplevel(-1)

Output:

  var1 var2 var1 var2
1    a    b    k    l
2    c    d    m    n
2    e    f  NaN  NaN
3    g    h    o    p
3    i    j    q    r
4  NaN  NaN    s    t

Reproducible inputs:

import numpy as np
import pandas as pd

df1 = pd.DataFrame(np.array(list('abcdefghij')).reshape(-1,2),
                   columns=['var1', 'var2'], index=[1,2,2,3,3])
df2 = pd.DataFrame(np.array(list('klmnopqrst')).reshape(-1,2),
                   columns=['var1', 'var2'], index=[1,2,3,3,4])