I’m new to scraping and would like to scrape this the "Historical Data" table from this url: https://coinmarketcap.com/currencies/bitcoin/historical-data/
I have tried to use bs4 but nothing seems to work for me as it just returns an empty list…
As far as I understand, what I need to do is to find all "tr" in the container – or what?
I don’t have that much code, but I think it make sense to show it to you so there is something to work with:
My code:
page = requests.get("https://coinmarketcap.com/currencies/bitcoin/historical-data/")
soup = BeautifulSoup(page.content, 'html.parser')
soup.find_all('tr')
>Solution :
The data you are looking for is added to the page via XHR/Fetch call. You can get it like the below
import requests
r = requests.get('https://api.coinmarketcap.com/data-api/v3/cryptocurrency/historical?id=1&convertId=2781&timeStart=1633910400&timeEnd=1639180800')
if r.status_code == 200:
print(r.json())