Home Creating a dataframe after extracting value from an oddly nested dictionary of a dataset

Questions

Creating a dataframe after extracting value from an oddly nested dictionary of a dataset

June 18, 2023

I am loading a tricky (to me) dataset from an api that is in the format {[{}]} and I need to create a dictionary by pairing the items in a list with a specific set of values found in their respective dataset.

lis = ['A', 'B']
for i in lis:
    chata = requests.get(urlhalf1 + i + urlhalf2)
    chart = json.loads(chata.text)
    print(chart)

Sample Output for item ‘A’ in lis (B not shown)

{'symbol': 'A', 'historical': [{'date': '2018-06-28', 'volume': 78952, 'changePercent': 5.55}, {'date': '2018-06-27', 'volume': 95867, 'changePercent': 2.22}]}

I would like to extract the values from key, ‘changePercent’ from this Sample Output of ‘A’ and somehow get to this next output:

data = {'A': [5.55, 2.22], 'B': [3.33, 4.44]}

With this format I would like to make a dataframe with a row for each key, value pair and with the index being [1, 2] (In this example). My code attempt:

df = pd.DataFrame(data), index = [1, 2]

>Solution :

Use a list comprehension to get all the changePercent values into a list.

data = {}
lis = ['A', 'B']
for i in lis:
    chata = requests.get(urlhalf1 + i + urlhalf2)
    chart = chata.json()
    print(chart)
    data[i] = [h['changePercent'] for h in chart['historical']]