Converting Byte Literal to Pandas

November 30, 2023

I have a data that looks like this:

In bytes literal form

data

b'[{"Name":"USA Stocks","Code":"US","OperatingMIC":"XNAS, XNYS","Country":"USA","Currency":"USD","CountryISO2":"US","CountryISO3":"USA"},{"Name":"London Exchange","Code":"LSE","OperatingMIC":"XLON","Country":"UK","Currency":"GBP","CountryISO2":"GB","CountryISO3":"GBR"}]'

I coverted

data_decode = data.decode("utf-8")

I am trying to get a pandas data frame from this.

df = pd.DataFrame((data_decode))

ValueError: DataFrame constructor not properly called!

df = pd.DataFrame(eval(data_decode))

NameError: name ‘null’ is not defined

>Solution :

Feed the byte string to pandas.read_json(), which supports a file-like object. io.BytesIO(data) wraps the data in a file-like object. No need to decode the bytes unless the JSON data is in a non-UTF8 encoding. If it is decode with that encoding and use io.StringIO instead.

import pandas as pd
from io import BytesIO

data = b'[{"Name":"USA Stocks","Code":"US","OperatingMIC":"XNAS, XNYS","Country":"USA","Currency":"USD","CountryISO2":"US","CountryISO3":"USA"},{"Name":"London Exchange","Code":"LSE","OperatingMIC":"XLON","Country":"UK","Currency":"GBP","CountryISO2":"GB","CountryISO3":"GBR"}]'
df = pd.read_json(BytesIO(data))
print(df)

Output:

              Name Code OperatingMIC Country Currency CountryISO2 CountryISO3
0       USA Stocks   US   XNAS, XNYS     USA      USD          US         USA
1  London Exchange  LSE         XLON      UK      GBP          GB         GBR

Note: If you received this data via a web API via the requests module, just use the .json() method of the reponse.