I’ve a Python script which I combine with the selenium-webdriver to crawl a certain website and extract the content of a class. After that I save the result inside a MySQL database.
The file (automation.py) is located in the server directory /www/htdocs/xxx/wow/test:
import mysql.connector
from time import sleep
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
mydb = mysql.connector.connect(
host="xy.com",
user="dddddd",
password="xxxxxxx",
database="xxxxx"
)
service_object = Service('./chromedriver')
driver = webdriver.Chrome(service=service_object)
driver.get('https://www.xxxxxx.com/')
sleep(5)
article_element = driver.find_element(By.CLASS_NAME, 'line-name ')
article_text = article_element.text
sql = "REPLACE INTO dfbase (name) VALUES (%s)"
val = [(article_text)]
cursor = mydb.cursor()
cursor.execute(sql, val)
print("Inserted",cursor.rowcount,"row(s) of data.")
mydb.commit()
mydb.close()
driver.quit()
On my local environment the script is working as expected, but when I use it on my webserver I get the following error:
ssh-xxxx@xxxx:/www/htdocs/xxx/wow/test$ python automation.py
Traceback (most recent call last):
File "automation.py", line 19, in <module>
driver = webdriver.Chrome(service=service_object)
TypeError: __init__() got an unexpected keyword argument 'service'
Outout of "pip list":
ssh-xxxx@xxxx:/www/htdocs/xxx/wow/test$ pip list
Package Version
---------------------- --------
apparmor 2.12
beautifulsoup4 4.6.0
bs4 0.0.1
certifi 2023.5.7
charset-normalizer 2.0.12
chromedriver-installer 0.0.6
idna 3.4
LibAppArmor 2.12
lxml 4.9.2
matplotlib 2.1.1
numpy 1.13.3
pexpect 4.2.1
pip 21.3.1
python-dotenv 0.20.0
requests 2.27.1
selenium 3.141.0
setuptools 59.6.0
urllib3 1.26.16
webdriver-manager 3.7.1
wheel 0.37.1
>Solution :
It’s because your web server is using a very old version of selenium (3.141.0). The service arg doesn’t exist there: https://github.com/SeleniumHQ/selenium/blob/selenium-3.141.0/py/selenium/webdriver/chrome/webdriver.py#L33
You may want to compare pip list between your local environment and your server environment, as you are likely using selenium 4 on your local environment, which is why service exists there: https://github.com/SeleniumHQ/selenium/blob/selenium-4.10.0/py/selenium/webdriver/chrome/webdriver.py#L32