Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

web scrapping issue for item availability

I’m new to web scrapping and was trying to learn by starting small project of tracking availablity of an item from online store. Website itself
So far I have just was able to get class part of what I need to get, but not quite sure how to get the text from it for item Name, Availability and Price.
I’d like your advice on what is the least effort solution to get this information from it. Or should I just treat it like a string and slice until I get my desired three values?

my code

import requests from bs4 
import BeautifulSoup



productURL = "https://allbest.kz/p98634475-oneplus-9rt-12256gb.html"
result = requests.get(productURL)
doc = BeautifulSoup(result.text, "html.parser")
classname = doc.select("[class~=b-sticky-panel__holder]")
print(classname)

Output:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

[<div class="b-sticky-panel__holder"><span class="b-sticky-panel__product-name" data-qaid="sticky_product_name">OnePlus 9RT 12/256Gb 5G Black</span><span class="b-sticky-panel__product-status" data-qaid="product_status_sticky_panel" title="Нет в наличии">Нет в наличии</span><div class="b-sticky-panel__cost" data-qaid="product_price_sticky_panel"><span class="b-sticky-panel__price">213 400 <span class="notranslate">Тг.</span></span></div></div>]

FYI: Here in the output "Нет в наличии" basically means "Out of stock".

Desired result:

Name = "OnePlus 9RT 12/256Gb 5G Black"
Availability = "Out of stock"
Price = 213 400

>Solution :

The following code snippet will get you the info you’re looking for:

import requests
from bs4 import BeautifulSoup

headers= {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}

url = 'https://allbest.kz/p98634475-oneplus-9rt-12256gb.html'
r = requests.get(url, headers=headers)
soup = BeautifulSoup(r.text, 'html.parser')

availability = soup.select_one('li[data-qaid="presence_data"]').get_text(strip=True)
name = soup.select_one('span[data-qaid="product_name"]').get_text(strip=True)
price = soup.select_one('span[data-qaid="product_price"]').get_text(strip=True)
print('Name =', name)
print('Availability =', availability)
print('Price =', price)

This will print out in terminal:

Name = OnePlus 9RT 12/256Gb 5G Black
Availability = Нет в наличии
Price = 213 400

BeautifulSoup docs: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#

Requests docs: https://requests.readthedocs.io/en/latest/

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading