Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Selenium can not find elements in dynamic web page, page source does not be loaded completely

I try to recover some elements in a web page with selenium but the page_source I’m getting it does not have that elements loaded.

Find element returns elem.text empty and driver.page_source does not have the id titulotramitedocu.

What am I missing?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Code:

URL = "https://seu.conselldemallorca.net/fitxa?key=91913"
driver = webdriver.Chrome()
driver.get(URL)
try:
    driver.implicitly_wait(20)
    elem = driver.find_element(By.ID,"titulotramitedocu")
    print(elem.text)
finally:
    driver.quit()

I also tried with a wait..

element = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "titulotramitedocu"))
)

>Solution :

To locate and print the text from the visible element instead of presence_of_element_located() you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.titulotramitedocu#titulotramitedocu > h1"))).text)
    
  • Using XPATH:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='titulotramitedocu' and @id='titulotramitedocu']//h1"))).text)
    
  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
  • Console Output:

    Concurs de mèrits de personal funcionari del Consell de Mallorca per a la categoria d'enginyer-a tècnic-a industrial d'administració especial (codi CFCEA2/024)
    
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading