So, I’m trying to post a JSON to a Node.JS server running Express with Python using the "requests" module. I’ve made a lot of tries, all of them failed. Closest I got was this: Server code: const fs = require(‘fs’); const express = require(‘express’); const app = express(); app.use(express.static("public")); app.use(express.json()); app.get(‘/’, function(_, res) { var… Read More How can I POST a JSON to an Express server with python module "requests"?

For example, from https://docs.python-requests.org/en/latest/api/#requests.cookies.RequestsCookieJar.set: set(name, value, **kwargs) Dict-like set() that also supports optional domain and path args in order to resolve naming collisions from using one cookie jar over multiple domains. Where can I find information about what other arguments the function takes as **kwargs? I mean these arguments, domain, path, expires, max_age, secure, httponly.… Read More Where can I find Python requests library functions **kwargs parameters documented?

When writing data to a csv file with Pandas, I used to use the method below. It still works, but throws this warning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. import requests import pandas as pd from bs4 import BeautifulSoup url = "https://www.breuninger.com/de/damen/luxus/bekleidung-jacken-maentel/" headers… Read More How to use "concat" in place of "append" while sticking with the same scraping logic in Python (Pandas)

I am facing an issue while trying to scrape information from a website using the requests.get method. The information I receive from the website is inconsistent and doesn’t match the actual data displayed on the website. As an example, I have tried to scrape the size of an apartment located at the following link: https://www.sreality.cz/en/detail/sale/flat/2+kt/havlickuv-brod-havlickuv-brod-stromovka/3574729052.… Read More request.get changes the content of the website? (Webscraping)

I am using a flask tutorial that I found here… https://andrewgriffithsonline.com/blog/180412-deploy-flask-api-any-serverless-cloud-platform/ In the "Test the App" section, there is a "http-prompt" command used. I do not want to use that. Instead I will like to use python requests module. >Solution : That’s great! Using the requests library is a common and powerful way to interact… Read More pass post request using url generated by flask

My project involves web scraping using python. In my project I need to get data about a given its registration. I have managed to get the html from the site into python but I am struggling to extract the values. I am using this website: https://www.carcheck.co.uk/audi/N18CTN from bs4 import BeautifulSoup import requests url = "https://www.carcheck.co.uk/audi/N18CTN"… Read More Extracting values from HTML in python

I am using python requests library to stream the data from a streaming API. response = requests.get(‘http://server/stream-forever’, stream=True) for chunk in response.iter_content(chunk_size=1024): print len(chunk) # prints 1905, 1850, 1909 I have specified the chunk size as 1024. Printing the length of the chunk read gives the chunk size greater than 1024 like 1905, 1850, 1909… Read More Python requests stream reads more data than chunk size

I have my response dictionary. Example: {10874: <Response [400]>, 11233: <Response [400]>, 13360: <Response [400]>, 15008: <Response [400]>, 11638: <Response [200]>, 14150: <Response [400]>, 15323: <Response [400]>, 14814: <Response [400]>, 12007: <Response [400]>, 11337: <Response [400]>, 13342: <Response [200]>} I’m trying to create new dictionary with only response 200. My code: new_dict = {} for… Read More Filtering dictionary of Response objects by value

Hi ! I’m new to Beautifulsoup, I was trying to webscrapp the info from this website: The problem is that when I try to inspect the elements on the website everything is called "td" and class"sch1". Therefore when I try to import I get a big mess. How can I import this information in a… Read More Is there a way to webscrapp a site where everything has the same name?

I have JSON data from a website, where I am trying to create a pandas dataframe from the data. It seems like I have a list of dictionaries that is nested in a dictionary and I am not sure what to do. My goal was to create key,value pairs and then make them into a… Read More Trying to make a pandas dataframe from a dictionary in a list of dictionaries