Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

JSON API to Dataframe

I created the function below to read from an API, it happens that I need the result to return in a datatrame, each key being a value, does anyone know how I can do this? When I transform into dataframe it doesn’t come columnar, it just brings the values ​​in the dataframe as json.

Function:

    def get_ip_list(self):
        response_all = list()

        while True:
            url = "www.apitest.com/testing/"
            response = requests.request("GET", url)

            try:
                if response.status_code != 200:
                    self.logger.log_info(f"An error has occured. Status code of API Page: {response.status_code}")
                else:
                    response = response.json()
                if response.count('no blacklist monitors found in account') > 0:
                    break
            except Exception as e:
                break
            response_all += response

            self.counter += 1
        df = pd.DataFrame.from_records(response_all)
        
        print(response_all)

Sample:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

[[{'ID': '5a6sd465sdf465asd498a798a', 'Type': 'IPv4', 'Target': '300.400.500.600', 'Add_Date': 1519929239, 'Last_Check': 1658187303, 'Status': 'Active', 'Label': '', 'Contact_List_ID': 'f56sdf465sdf465sd4f', 'Blacklisted_Count': '0', 'Blacklisted_On': None, 'Links': {'Report_Link': 'https://hetrixtools.com/report/blacklist/456465465/', 'Whitelabel_Report_Link': 'https://rbl.allin.com.br/report/blacklist/545665/'}}, {'ID': 'mdkljkljsdflkjsd123232', 'Type': 'IPv4', 'Target': '200.109.500.900', 'Add_Date': 1519929239, 'Last_Check': 1658188249, 'Status': 'Active', 'Label': '', 'Contact_List_ID': 'jfkldjflksjflkjsdf5456465', 'Blacklisted_Count': '0', 'Blacklisted_On': None, 'Links': {'Report_Link': 'https://hetrixtools.com/report/blacklist/23565456465/', 'Whitelabel_Report_Link': 'https://rbl.allin.com.br/report/blacklist/2156465465/'}}, {'ID': '165sdf465s4df654d56', 'Type': 'IPv4', 'Target': '155.111.255.300', 'Add_Date': 1519929239, 'Last_Check': 1658187644, 'Status': 'Active', 'Label': '', 'Contact_List_ID': 'ds54f65sd4f65d4sf654df65', 'Blacklisted_Count': '0', 'Blacklisted_On': None, 'Links': {'Report_Link': 'https://hetrixtools.com/report/blacklist/45465465/', 'Whitelabel_Report_Link': 'https://rbl.allin.com.br/report/blacklist/5465465/'}}], {'Meta': {'Total_Records': '24576'}, 'Links': {'Pages': {'Prev': 'https://api.hetrixtools.com/v2/a8b6c925e6d01613deaf5f5c48581f8f/blacklist/monitors/22/1024/'}}}]

>Solution :

I’m assuming the sample you’ve provided is the contents of response_all. To create DataFrame from it you can do:

df = pd.DataFrame(response_all[0])
print(df.to_markdown(index=False))

Prints:

ID Type Target Add_Date Last_Check Status Label Contact_List_ID Blacklisted_Count Blacklisted_On Links
5a6sd465sdf465asd498a798a IPv4 300.400.500.600 1519929239 1658187303 Active f56sdf465sdf465sd4f 0 {‘Report_Link’: ‘https://hetrixtools.com/report/blacklist/456465465/’, ‘Whitelabel_Report_Link’: ‘https://rbl.allin.com.br/report/blacklist/545665/’}
mdkljkljsdflkjsd123232 IPv4 200.109.500.900 1519929239 1658188249 Active jfkldjflksjflkjsdf5456465 0 {‘Report_Link’: ‘https://hetrixtools.com/report/blacklist/23565456465/’, ‘Whitelabel_Report_Link’: ‘https://rbl.allin.com.br/report/blacklist/2156465465/’}
165sdf465s4df654d56 IPv4 155.111.255.300 1519929239 1658187644 Active ds54f65sd4f65d4sf654df65 0 {‘Report_Link’: ‘https://hetrixtools.com/report/blacklist/45465465/’, ‘Whitelabel_Report_Link’: ‘https://rbl.allin.com.br/report/blacklist/5465465/’}
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading