How to unnest json data to dataframe in using python

October 10, 2022

import json
import pandas as pd

resp = requests.get('https://data.cms.gov/provider-data/api/1/datastore/query/77hc-ibv8/0?offset=0&count=true&results=true&schema=true&keys=true&format=json')
data = resp.json()
data = json.dumps(resp_dict,indent=4)
print(data)

I am trying to unnest the json data into a dataframe using Python. There are a few ways to do it but wasn’t able to fully unnest the json data. What would be the best way to accomplish this?

>Solution :

Nothing special just pass it to pd.DataFrame and refer to the main key.

import json

import pandas as pd

import requests


def get_data() -> pd.DataFrame:
    url = (
        "https://data.cms.gov/provider-data/api/1/datastore/query/77hc-ibv8/0?"
        "offset=0&"
        "count=true&"
        "results=true&"
        "schema=true&"
        "keys=true&"
        "format=json"
    )

    with requests.Session() as request:
        response = request.get(url, timeout=10)
    if response.status_code != 200:
        print(response.raise_for_status())

    data = json.loads(response.text)

    return pd.DataFrame(data=data["results"])


print(get_data())