I am trying to scrap some phone numbers from a local broker.
Their website source code goes:
<div class="officeagent-comm-item ng-scope" ng-if="agent.phone != null">
<a class="icon-btn-phone" href="tel:910 600 334" alt="tel:910 600 334"></a> </div>
And I coded the following to scrap the phone numbers:
while weDone == 0:
sleep(5)
apt_Cookies()
try:
AllAgents = driver.find_elements("xpath", "//div[@class='agent-details']")
except:
print(' No agents ')
for x in range(0,len(AllAgents)):
print(AllAgents[x].text)
try:
phoneFound = AllAgents[x].find_element("xpath", "//a[@alt]")
phoneNumber = phoneFound.get_attribute("alt")
except:
print(' No Phone ')
print(phoneNumber)
The output for my code is:
Abraão Vieira
tel:926319959
Acácia Consulting
tel:926319959
...
Acacio Garcia
tel:926319959
Adelaide Carrilho
tel:926319959
You can see that different elements are being scrap as the names of the agents are different (Abraão, Acácio, Acacio…), but when it comes to the phone number, the first number in the list of elements repeats itself.
Any idea on what is going on?
>Solution :
I had this issue recently and it turned out that I had to add a ‘.’ to the start of the xpath within the for loop to specify that I was looking for the element within the list value class. If it’s the same issue here then it’d be a very subtle change of:
while weDone == 0:
sleep(5)
apt_Cookies()
try:
AllAgents = driver.find_elements("xpath", "//div[@class='agent-details']")
except:
print(' No agents ')
for x in range(0,len(AllAgents)):
print(AllAgents[x].text)
try:
phoneFound = AllAgents[x].find_element("xpath", ".//a[@alt]")
phoneNumber = phoneFound.get_attribute("alt")
except:
print(' No Phone ')
print(phoneNumber)