The code below was working as of a few days ago, but now it is only finding the first script for the url
url = 'https://understat.com/team/{}/2022'.format('Brentford')
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')
scripts = soup.find_all('script')
scripts
>Solution :
Try to set a cookie when accessing the page:
import requests
from bs4 import BeautifulSoup
url = "https://understat.com/team/{}/2022".format("Brentford")
response = requests.get(url, cookies={"beget": "begetok"}) # <-- note the cookies= parameter
soup = BeautifulSoup(response.content, "html.parser")
scripts = soup.find_all("script")
print(scripts)
Prints:
...
window.onload = function() { (adsbygoogle = window.adsbygoogle || []).push({}); }
</script>, <script defer="" src="js/date.format.min.js?v=2" type="text/javascript"></script>, <script defer="" src="js/calendar.js?v=2.1" type="text/javascript"></script>, <script defer="" src="js/team.js?v=2.5" type="text/javascript"></script>]