I want to scrape the table "Implied Volatility" from the following url ideally as a dataframe

https://optioncharts.io/options/AAPL/overview/option-statistics

I have identified the table with the classname "table border table-sm optioncharts-table-styling table-light"

<table class="table border table-sm optioncharts-table-styling table-light" style="">
            <tbody>
            <tr>
              <th>
                Implied Volatility
                <i class="bi bi-info-circle" rel="tooltip" data-bs-toggle="tooltip" data-bs-placement="right" title="" data-bs-original-title="The average implied volatility of options expring nearest to 30-days." aria-label="The average implied volatility of options expring nearest to 30-days."></i>
              </th>
              <td>17.49%</td>
            </tr>
...

So my idea was to use soup.find to search for the div and class name.
Both pandas and soup though fail to find the table.
I tried the code below which outputs a mix of random HTML and errors but not the table.

import requests
import pandas as pd
import json
from pandas.io.json import json_normalize
from bs4 import BeautifulSoup

url = 'https://optioncharts.io/options/AAPL/overview/option-statistics'
res = requests.get(url)
soup = BeautifulSoup(res.content, "lxml")

>Solution :

The data you see is loaded from external URL via Javascript. To load the data into pandas dataframe you can use:

import pandas as pd

api_url = "https://optioncharts.io/async/options_ticker_info?ticker=AAPL"

df = pd.read_html(api_url)[0].set_index(0).rename_axis(index=None)
print(df)

Prints:

                                        1
Implied Volatility                 17.49%
Historical Volatility              13.18%
IV Percentile                         12%
IV Rank                             9.07%
IV High                42.79% on 01/03/23
IV Low                 15.42% on 12/15/23

---

To get "Implied Volatility":

print(df.loc["Implied Volatility"])

Prints:

1    17.49%
Name: Implied Volatility, dtype: object