Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Unable to print href in web scraper, no errors

I am able to extract the names from the html however trying to get the href links isn’t working out. I am needing to get it so it prints the link next to the name its attached too. before I can do that I need to be able to print the link in the first place.

from selenium import webdriver
from selenium.webdriver import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

user_input = input("character: ")
options = Options()
options.headless = True
options.add_argument("--window-size=1920,1080")

driver = webdriver.Chrome("C:\Program Files (x86)\chromedriver.exe")
driver.get("https://www.marvel.com/search")

search_input_xpath = "//input[@placeholder='Search']"
search_input = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, search_input_xpath)))
search_input.send_keys(user_input)

first_item_in_auto_suggest_area_xpath = "//div[contains(@id,'react-autowhatever')]/ul"
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, first_item_in_auto_suggest_area_xpath)))
search_input.send_keys(Keys.ENTER)

section = WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.CSS_SELECTOR, ".search-list p.card-body__headline a")))


character = driver.find_elements(By.CSS_SELECTOR, ".search-list p.card-body__headline a")
for character_tag in character:
    print(character_tag.text)

href_elements = driver.find_elements(By.CSS_SELECTOR, ".search-list p.card-body__headline href")
for href in href_elements:
    print(href.get_attribute("href"))

>Solution :

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

You can’t access an attribute using a css selector.

I’d try first to change href to a:

href_elements = driver.find_elements(By.CSS_SELECTOR, ".search-list p.card-body__headline a")
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading