Merge two list of dicts based on an index in the dicts

December 13, 2022

These two are related dataset, but coming from seperate json files, so I would like to merge them. They can match on index, but I did not really find a good way of doing this 🙂

List of dicts 1:

[
  {'index': 217, 'name': 'Battery'}
  {'index': 218, 'name': 'Fluffy'}
  {'index': 219, 'name': 'Dazzling'}
  {'index': 220, 'name': 'Soul-Heart'}
]

List of dicts 2:

[
  {'index': 217, 'desc': 'Text info 2'}
  {'index': 218, 'desc': 'will be very informative'}
  {'index': 219, 'desc': 'dont know what else i could write here'}
  {'index': 220, 'desc': 'Boosts my wallet'}
]

Result should be something like:

[
  {'index': 217, 'name': 'Battery', 'desc': 'Text info 2'}
  {'index': 218, 'name': 'Fluffy', 'desc': 'will be very informative'}
  {'index': 219, 'name': 'Dazzling', 'desc': 'dont know what else i could write here'}
  {'index': 220, 'name': 'Soul-Heart', 'desc': 'Boosts my wallet'}
]

There is a lot more data, but as soon as I know how to merge, i think i can do the rest

>Solution :

Pandas handles merges like a breeze.

First convert the data into dataframes:

import pandas as pd

data1 = [
    {'index': 217, 'name': 'Battery'},
    {'index': 218, 'name': 'Fluffy'},
    {'index': 219, 'name': 'Dazzling'},
    {'index': 220, 'name': 'Soul-Heart'},
]
data2 = [
    {'index': 217, 'desc': 'Text info 2'},
    {'index': 218, 'desc': 'will be very informative'},
    {'index': 219, 'desc': 'dont know what else i could write here'},
    {'index': 220, 'desc': 'Boosts my wallet'},
]
df1 = pd.DataFrame(data1)
df2 = pd.DataFrame(data2)

Then merge on the index column:

df_out = df1.merge(df2, on='index')

Which looks like this:

   index        name                                    desc
0    217     Battery                             Text info 2
1    218      Fluffy                will be very informative
2    219    Dazzling  dont know what else i could write here
3    220  Soul-Heart                        Boosts my wallet

Docs: pandas.DataFrame.merge()

Then convert back to lists of dicts:

df_out.to_dict(orient='records')

[{'index': 217, 'name': 'Battery', 'desc': 'Text info 2'},
 {'index': 218, 'name': 'Fluffy', 'desc': 'will be very informative'},
 {'index': 219, 'name': 'Dazzling', 'desc': 'dont know what else i could write here'},
 {'index': 220, 'name': 'Soul-Heart', 'desc': 'Boosts my wallet'}]