Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Simpler way to write headers

I am looking for a simplier way to write these headers in the example below. The reason I have one for each loop is so that it’ll generate a new User Agent at random for each request it makes; however, I am looking for a way to not have to write it so many times for each loop.

Code Example:

for example in examples:
    headers = {
    'accept': '*/*',
    'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
    'referer': 'https://www.google.com/',
    'user-agent': random.choice(all_user_agents),
    }
    response = request.get(url, headers=headers)

    while 10 > i:
        headers = {
        'accept': '*/*',
        'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
        'referer': 'https://www.google.com/',
        'user-agent': random.choice(all_user_agents),
        }
        response = request.get(url, headers=headers)

    for test in tests:
        headers = {
        'accept': '*/*',
        'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
        'referer': 'https://www.google.com/',
        'user-agent': random.choice(all_user_agents),
        }
        response = request.get(url, headers=headers)

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

You can extract common header values like accept-language and referer into a common dict object, and then just clone and update those default headers as needed:

import random
import string

import requests as req

default_headers = {
    'accept': '*/*',
    'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
    'referer': 'https://www.google.com/',
    # 'user-agent': None,
}

# - SETUP -
examples = ['a']
tests = []
url = 'TODO'
all_user_agents = string.ascii_lowercase
i = 5

# - LOOP -
for _ in examples:
    # >  .copy() is optional
    headers = default_headers.copy()

    response = req.get(url, headers=headers)

    while 10 > i:
        headers['user-agent'] = random.choice(all_user_agents)
        response = req.get(url, headers=headers)
        # >  I added the line below, so we can avoid an infinite loop
        i += 1

    for _ in tests:
        headers['user-agent'] = random.choice(all_user_agents)
        response = req.get(url, headers=headers)
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading