Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Reading multiple log files from a folder using python

In the past I have read a log file using python with the following code, which has always worked fine:

with open(r"24T23.log") as f, open('logfile.csv', 'w') as f2:
    writer = csv.writer(f2)
    writer.writerow(['Index','Date', 'Time', 'Logic', '(Logic)','Type', 'Code','Connector', 'Message', 'Extra 1', 'Extra 2'])

    i = 0
    for line in f:
        writer.writerow([i] + line.rstrip().split('\t'))
        i += 1

For a particular use case that I am working on, I need to read multiple files contained in a folder. Can someone please suggest how to modify the above code (I tried that using blob but could not succeed)?
Thanks in advance.

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

How’s this?

import csv
import os

# From root of cwd
DIRECTORY = 'test'

# This gets the path of every file in DIRECTORY if it is a log file
# os.path.join(os.getcwd(),x) to include cwd in directory name
logs = [os.path.join(os.getcwd(), x) for x in os.listdir(
    DIRECTORY) if os.path.splitext(x)[1] == '.log']

for log in logs:
    # Note the 'a' append write, to append the rows to the files
    with open(log) as f, open('logfile.csv', 'a') as f2:
        writer = csv.writer(f2)
        writer.writerow(['Index', 'Date', 'Time', 'Logic', '(Logic)',
                        'Type', 'Code', 'Connector', 'Message', 'Extra 1', 'Extra 2'])

        i = 0
        for line in f:
            writer.writerow([i] + line.rstrip().split('\t'))
            i += 1

Docs on os.listdir()

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading