I’ve set up my first script to scrape webpages, but currently by defining a start and stop point in my range like so:
for page in range(1, 4):
However, I have no idea how many pages there will be at any given moment (value will regularly change) – what should I use as my stop value in this scenario? I want to avoid the hack of putting a ridiculously high value.
>Solution :
There are many ways to skin this cat, but I would use the count method in itertools:
import itertools
for page in itertools.count(1):
...