I’m trying to scrape the name of the sofas from this webpage using the requests module, as shown below. When I observe network activity for that request, I see the same logic applied there as I’ve tried below, but I always end up getting status 400. How can I scrape the name of the sofas… Read More Failed to scrape the names of a few products from a webpage using the requests module
I have to code multiple Web Services calls. For this, i use requests (import requests). Each call has the same structure: try: retrieved_response = requests.get(url_call, auth=authentication, params=search_parameters, verify=False) except requests.exceptions.HTTPError as http_error: logger.error("Bad Status Code", http_error.response) raise http_error except [requests.exceptions.ConnectionError, requests.exceptions.ConnectTimeout] as connection_error: logger.error("Connection Problem ", connection_error.response) raise connection_error except requests.exceptions.Timeout as timeout_error: logger.error("Time Out… Read More How launch a web service request from a string sroring this request in Python?
I’m trying to download the rss feed from "https://www.straitstimes.com/news/singapore/rss.xml". I have the following Python script: import requests r = requests.get(‘https://www.straitstimes.com/news/singapore/rss.xml’) for k, v in r.headers.items(): print("{}: {}".format(k, v)) print(r.content) When I run this, I get the following response: Cache-Control: max-age=0, no-cache, no-store Content-Type: text/html Date: Wed, 13 Dec 2023 03:06:00 GMT Expires: Wed, 13 Dec… Read More HTTP request works with curl but fails with Python with 403
import requests from bs4 import BeautifulSoup URL = "https://habr.com/ru/hubs/gamedev/articles/" # Url to website page = requests.get(URL).content soup = BeautifulSoup(page, "html.parser") post = soup.find("article", class_="tm-articles-list__item") # Last post thah i need to parse discription = post.find_all(‘p’) for post_text in discription: # Trying to separate the text text = post_text.get_text() print(text) Getting this error: File "d:\CODING\Projects\net N… Read More Can't get text with Beautifull Soup from between <p> </p>
I’m trying to get the current location of International Space Station(ISS) and display it in a json format. The code seems okay but I keep getting error and I don’t know why. import requests sample = requests.get(url="http://open-notify.org/Open-Notify-API/ISS-Location-Now/") conv= sample.json() print(conv) I was expecting the code to return a json of the current location of the… Read More Python requests.json keeps giving errors
I’ve developed a code to access a supermarket’s API and pull product information, the problem is the info I pull seems to differ from that seen on the webpage’s Network -> Response tab. This is my code: import requests import numpy as np def fetch_data(base_url): params = { ‘take’: 30, ‘skip’: 0, ‘page’: 1 }… Read More Scraping website API with requests – returning unexpected results
I’m currently working on a Python script that continuously fetches the current time from a web server and prints it. However, I’m encountering an issue with the authentication, resulting in a 401 status code. I’ve provided all the necessary headers and access token, but the response indicates an authentication problem. The web server’s link: http://cyber.cs.du.edu/timekeeper/… Read More Unable to Fetch Time from Web Server using Python Requests Library (Status Code 401)
I am attempting to fetch an API using Python requests. The API is documented here: https://consumerdatastandardsaustralia.github.io/standards/#get-data-holder-brands The following request works on reqbin: I have the following Python code import requests r = requests.get(‘https://api.cdr.gov.au/cdr-register/v1/energy/data-holders/brands/summary’, headers={"x-v":"1"}) print(r.content) This returns a 403 error, I’m not sure what other difference there could be between the two requests which could… Read More Python Requests API 403 Error with same HTTP Request
Hello, I can’t find a way to retrieve the words "TSV SCHOTT Mainz" from the HTML code because I don’t understand which section to target here. I’ve tried the following: import requests from bs4 import BeautifulSoup # URL of the Borussia Dortmund "Alle Spiele" page url = "https://www.bvb.de/Spiele/Alle-Spiele" # Send an HTTP GET request to… Read More How can I retrieve "TSV SCHOTT Mainz" from HTML using Python
I use beautiful soup to parse data from site. I have problem there is a text in tag on site but I gets an empty tag (without data). I want to solve this without selenium and web-drivers. My code: import requests import lxml from bs4 import BeautifulSoup as BS get_data = requests.get(‘https://dentalia.com/clinica/’) html = BS(get_data.content,… Read More Get an empty tags from site