Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Scrapy Authentication to some site

I need my parser to log in to the site https://kmt5.com.ua/access-denied

class KMTSpider(scrapy.Spider):
name = 'kmt'
start_urls = ['https://kmt5.com.ua/kompyuternaya-periferiya/']
login_url = 'https://kmt5.com.ua/login'

def parse(self, response):
    yield response.follow(response.css('div[data-href="https://kmt5.com.ua/login/"]::attr(data-href)').get(),
                          self.login)

def login(self, response):
    form = response.css('div.remodal.modal-login form[action="/login/"]')

    return FormRequest.from_response(
        response,
        formdata={
            'email': 'email',
            'password': 'password'
        },
        formcss='div.remodal.modal-login form[action="/login/"]',
        callback=self.after_login
    )

def after_login(self):
    pass

I tested many options, here is one of them, but I do not have much experience in developing such parsers, I suspect that there is some small error, please help

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

This site to which you are trying to log in is a JavaScript-based site. Therefore, you cannot log in with Scrapy because Scrapy only allows static websites. If you want to log in to this type of website, which uses JavaScript, you can use Selenium as an alternative. Selenium is known for browser automation.

Here is the official Selenium documentation Link

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading