Python/Selenium: How to grab value from the HTML?

January 14, 2022

I am trying to grab some values from the following code example but am unable to get a successful answer.

Using the US Patent and Trademark Office’s website with a random trademark entered in this link. I want to grab the Registration Date which has a value of May 12, 2015

    <div class="double table">
        <div class="row">
               <div class="key">US Serial Number:</div>
               <div class="value">85931937</div>
               <div class="key">Application Filing Date:</div>
               <div class="value">May 14, 2013</div>
        </div>
       
        <div class="row">
               <div class="key">US Registration Number:</div>
               <div class="value">4735834</div>
               <div class="key">Registration Date:</div>
               <div class="value">May 12, 2015</div>
        </div>

Notice this would be the fourth time the class name value is being called in the code example and is nested within other div classes.

This is what I have tried so far:

values = browser.find_elements(By.CLASS_NAME, 'value')
print(values[3])

but values returns an empty list []

Please advise on what I am doing wrong, thank you in advance.

>Solution :

To print the text May 12, 2015 you can use either of the following Locator Strategies:

Using xpath and text attribute:

print(driver.find_element(By.XPATH, "//div[@class='key' and text()='Registration Date:']//following-sibling::div[1]").text)

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

Using XPATH and get_attribute("innerHTML"):

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='key' and text()='Registration Date:']//following-sibling::div[1]"))).get_attribute("innerHTML"))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC