How save request to JSON with Puppeteer

I’m trying to collect and save all the api calls from a website, using puppeteer.

At the moment I can print the request that include /api/

    let page = await browser.newPage();
    
    page.on("request", async (request) => {
        const url = await request.url();
        if (url.includes("/api"))
        console.log(request.url())
    })
    

    // Go to the URL
    await page.goto('http://www.twitter.com', {});

But whenever I try to save the request.url() into JSON

page.on("request", async (request) => {
        const url = await request.url();
        if (url.includes("/api"))
        fs.writeFile('./data.json', JSON.stringify(url), err => err ? console.log(err): null);
        console.log(request.url())
    })

It only logs the last request.url.

I would like to know if there is a way to save all the request 🙂

>Solution :

The request event is when the request is made, not when the response is received.

You want to listen for the page.on(“response”, handler)

Leave a Reply