How to scrape specific element with a certain id in BeautifulSoup?

I am trying to scrape the table from baseball reference: https://www.baseball-reference.com/players/b/bondsba01.shtml, and the table I want is the one with id="batting_value", but when I trying to print out what I have scraped, the program returned an empty list instead. Any information or assistance is appreciated, thanks!

from bs4 import BeautifulSoup
from urllib.request import urlopen

root_page = "https://www.baseball-reference.com/players/b/bondsba01.shtml"
soup = BeautifulSoup(urlopen(root_page), features = 'lxml')

table = soup.find('table', id = 'batting_value')
print(table)

I’ve tried to print the <div> with id="div_batting_value" which contains the table in it, but still doesn’t work. However, I can successfully print out other <div> elements with different id.

>Solution :

Main issue here is that the table is hidden in the comments, so you have to bring it up first, before BeautifulSoup could find it – simplest solution in my opinion is to replace the specific characters in this case:

.replace('<!--','').replace('-->','')

Alternative is to be more specific and use bs4.Comment

Example

import requests
from bs4 import BeautifulSoup

soup = BeautifulSoup(
        requests.get('https://www.baseball-reference.com/players/b/bondsba01.shtml').text.replace('<!--','').replace('-->','')
)
soup.select_one('#batting_value')

Or in use with pandas.read_html():

import requests
import pandas as pd

df = pd.read_html(requests.get('https://www.baseball-reference.com/players/b/bondsba01.shtml').text.replace('<!--','').replace('-->',''), attrs={'id':'batting_value'})[0]
df[(~df.Lg.isna()) & (df.Lg != 'Lg')]

Results in: