Im currently trying to get all posts of a paginated website, everything is working fine, my only problem is that i dont know how to end my for loop if axios catches an error
getMaxPageAmount(url: any) {
let maxPage = 600;
let allLinks = [] as any;
let collection = [] as any;
for (let i = 1; i < maxPage; i++) {
allLinks.push(
axios.get(url + i + "/").then(urlResponse => {
let $ = cheerio.load(urlResponse.data);
$("div.main-posts").each((i, element) => {
let link = $(element)
.find("div#entry-pic").find("a").get().map(x => $(x).attr('href'))
collection.push(link);
console.log(collection);
});
})
.catch((reason: AxiosError) => {
if (reason.response!.status == 404) {
//Need to break
}
})
)
}
Promise.all(allLinks).then(() => console.log(collection));
}
I already tried to exit the for loop with break, but then i get "Jump target cannot cross function boundary.". A while loop was also not an option because it seems like it breaks the axios.get function.
>Solution :
With your current code, you can’t, because your for loop finishes before any of the axios calls completes. Your code starts all the calls (without waiting), then waits for them all to complete.
If you want to do them one at a time, in sequence, then the easiest thing is to make your function an async function and await each result:
async getMaxPageAmount(url: any) {
let maxPage = 600;
let collection = [] as any; // *** Best to avoid using `any`
for (let i = 1; i < maxPage; i++) {
try {
const urlResponse = await axios.get(url + i + "/");
let $ = cheerio.load(urlResponse.data);
$("div.main-posts").each((i, element) => {
let link = $(element)
.find("div#entry-pic")
.find("a")
.get()
.map((x) => $(x).attr("href"));
collection.push(link);
// console.log(collection);
});
} catch (reason: any) {
if (reason.response!.status == 404) {
break; // *** This will break the `for` loop
}
}
}
console.log(collection);
}