I’m attempting to click on an href in puppeteer js. Here is my code: await page.goto(‘https://example.net/search/images?search=real+estate+postcards’); await page.waitForSelector(".GridItem-item-tVq", {visible: true}); const els = await page.$$(".GridItem-item-tVq"); els.forEach(async el => { const href = await el.$eval(‘a’, a => a.getAttribute(‘href’)); await page.click(href) }) when I console.log href I get: /gallery/93911757/High-End-Real-Estate-Postcard-Design/modules/542619723 /gallery/69359021/Postcard-Design/modules/492523237 /gallery/158597277/Post-Card-Design/modules/894845353 /gallery/126032017/Real-Estate-Agent-Postcard/modules/715550465 This is a url in… Read More Unable to click on an href or go to new page in Pupetteer js

Presume the website layout looks like this: <div id="monday"> … <div class="dish"> Potato Soup <br> <span>With smoked tofu</span> </div> </div> How, using puppeteer, would I be able to grab just the text node’s content, not everything inside .dish? I’ve tried let selector = await page.waitForSelector("#monday .dish"); let text = await selector.evaluate(el => el.textContent) ?? "";… Read More How to get just the node text, not the children's text in puppeteer

Link: http://quotes.toscrape.com/ Click on the above link and scroll to the bottom. I don’t know how to click on the Next button. My code: await page.click(".pager > .next > a") The above code is not working. I want to know, how to click on this button. >Solution : I use below code and can click… Read More How to click on button using Puppeteer?

I’m hoping someone can suggest a better method for what I’m trying to achieve. While performing a rather long and complicated webflow using puppeteer, occasionally an error will occur that disrupts the actions I’m trying to take on the page. There’s nothing I can do about it in these situations other than return a useful… Read More How to catch an error thrown in an un-awaited async function inside try/ catch?

I am having a problem where I can only see the contents of my output array vars from inside of my main Puppeteer function scrapeEbay(). Is this because this function is asynchronous? My intent is to have the main function be able to see the contents of the output arrays; to return them. But they… Read More Why can Array Contents can only be seen inside of main async puppeteer function, not outside of it

So I’m using Puppeteer to scrap text from social media, I want to only scrap the text from a post, when I use the chrome developer tool to read what is the class name of the div which contains the text, it always displays a different class name when I reload the page but stay… Read More How do I select a div with a class name that ends with a specific string and keep the text content?

So, I’m trying to scrape web in order to get their elements link as well as id. However, it throws me an undefined as a result. I know that these classes exist because I use page.evaluate, which gets specified class without hesitation. Cannot figure out why it is not working. Moreover, it looks like, that… Read More Puppeteer gives undefined with $$eval

When using puppeteer I use page.type() and I’ve noticed that when working with more than one input field the code begins to repeat itself. Here is the shortened version of what I’m doing: await page.type(‘#idFirstName’, user.first_name); await page.type(‘#idEmailAddress’, user.email); await page.type(‘#idLastName’, user.last_name); await page.type(‘#idCity’, user.city); Is there any way to fill multiple input fields without… Read More Clean way to fill multiple inputs with puppeteer

I’m trying to collect and save all the api calls from a website, using puppeteer. At the moment I can print the request that include /api/ let page = await browser.newPage(); page.on("request", async (request) => { const url = await request.url(); if (url.includes("/api")) console.log(request.url()) }) // Go to the URL await page.goto(‘http://www.twitter.com&#8217;, {}); But whenever… Read More How save request to JSON with Puppeteer

I need to check for a certain service worker being registered. Unfortunately, page.evaluate returns undefined no matter what I do. let page = await chrome.newPage(); await page.goto(‘http://127.0.0.1:8089/&#8217;); await page.waitFor(10000); const isCorrectSW = await page.evaluate(async () => { await navigator .serviceWorker .getRegistrations() .then(registrations => registrations[0].active.scriptURL.endsWith(‘/target.js’) ); }); console.log(isCorrectSW); isCorrectSW ends up being undefined, but if I… Read More Puppeteer's `page.evaluate` result differs from devTools console