Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Writing a large collection of lists to a txt file in Python

I am trying to save links to photos in a topic on an internet forum in a txt file. I tried many ways, but the links of one page are saved in a txt file, and when the loop goes to the next page of the topic, the previous links are deleted and new links are replaced! I want to have all the links together.

This is my code:

from bs4 import BeautifulSoup
import requests

def list_image_links(url):
    response = requests.get(url)

    soup = BeautifulSoup(response.content, "html.parser")
    
    # Separation of download links
    image_links = []
    for link in soup.find_all('a'):
        href = link.get('href')
        if href is not None and 'attach' in href and href.endswith('image')==False:
            image_links.append(href)
    
    # Writing links in a txt file
    with open('my_file.txt', 'w') as my_file:
        my_file.write('image links:' + '\n')    
        for branch in image_links:
            my_file.write(branch + '\n')
            print('File created')
    
# Browse through different pages of the topic
i = 0
while i <= 5175:
    list_image_links(f'https://forum.ubuntu.ir/index.php?topic=211.{i}')
    i = i+15

It is clear from the comments what each section does.

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Thank you in advance for your help.

>Solution :

You need to append to the file. This can be achieved by using 'a' instead of 'w' as an argument to open().

When using 'w' a file will be created if it does not exist and it will always truncate the file first, meaning it will overwrite its contents. With 'a' on the other hand the file will also be created if it does not yet exists, but it won’t truncate but instead append to the end of the file if it already exists, meaning the content will not be overridden.

See Python docs.

So for your example the line

with open('my_file.txt', 'w') as my_file:

would need to be changed to:

with open('my_file.txt', 'a') as my_file:
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading