Date search and date output from the same class name when parsing

January 23, 2023

I’m trying to parse a site. It has the same named classes, while the number of such classes varies from page to page. I’m interested in the class that contains the date, which is written in the following pattern: 24 January 2020.

Code:

import pandas as pd
import numpy as np
import requests
from bs4 import BeautifulSoup
import re
from user_agent import generate_user_agent
import time

my_user_agent = generate_user_agent(os=('mac', 'linux'))

headers = {
    "Accept": "*/*",
    "user-agent": my_user_agent}

source = 'https://coronavirus-graph.ru/kanada'
req = requests.get(source, headers=headers)
html = req.content
soup = BeautifulSoup(html, 'lxml')
items_list = soup.find_all('div', class_='rr')

print(items_list)
[<div class="rr"><span>In Canada, ≈</span><b>11</b><span> people die out of </span><b>1000</b><span> infected ( 1.1%)</span></div>, <div class="rr"><b>99</b><span> a person in serious condition</span></div>, <div class="rr"><span>First person infected</span><b>24 January 2020</b></div>]

In this case, I’m interested in the last class and its value in <b>.
That is the date. How can I get it?

>Solution :

Here is one way of getting that data:

[...]
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36'
}

source = 'https://coronavirus-graph.ru/kanada'
r = requests.get(source, headers=headers)
soup = BeautifulSoup(r.text, 'lxml')
items_list = soup.select('div[class="rr"]')[-1].select_one('b')

print(items_list.text)

Result in terminal: