Scrapping some data from a football website using Selenium and Python

April 26, 2022

I’m trying to make a Python program that extracts some data using Selenium where first I have to close two alerts then click on "Show all matches" button and finally I need to click on each "stats" button (there are multiple and they all have the same class name) to extract a specific row from this table.

I need to extract the 4 values highlighted in blue for each game

I already did the first two steps but now I’m stuck in the final one where I have to click on each "stats" button, extract the 4 values from each table then close the window and move to the next game.

Here is my code

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import time

s=Service("C:/Users/dhias/OneDrive/Bureau/stgg/chromedriver.exe")
driver=webdriver.Chrome(service=s)
driver.get("https://www.soccerstats.com/matches.asp?matchday=1#")
driver.maximize_window()
time.sleep(1)
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[mode='primary']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID,"steady-floating-button"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Show all matches']"))).click()

and I tried this to click on every "stats" button that have the same class name but it didn’t work

for element in driver.find_elements(By.XPATH,"//a[@class='myButton' and text()='stats']"):
    WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//a[@class='myButton' and text()='stats']"))).click()

link to the website : soccerstats website

>Solution :

save links to array and click after because after click you are not on the page that have the links anymore

stat_links = []
#get all urls
for element in driver.find_elements(By.XPATH, "//a[@class='myButton' and text()='stats']"):
    stat_links.append(element.get_attribute('href'))
    
for link in stat_links:
    driver.get(link)
    # do your stuff