Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Merge all txt files in dataframe pandas

I have a series of txt files in a folder and I wanted to move these files to a data frame

but I only managed to do this by saving a csv file, how do I work with the data frame directly without having to save to a csv file?

follow my code below

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

import os
import csv
import pandas as pd
main_folder = ('F:\PROJETOS\LOTE45\ARQUIVOS\RISK\RISK_CUSTOM_FUND_N1'

def get_filename(path):
filenames = []
files = [i.path for i in os.scandir(path) if i.is_file()]

for filename in files:
    filename = os.path.basename(filename)
    filenames.append(filename)
return filenames

files = get_filename(main_folder)

with open('some.csv', 'w',  encoding = 'utf8', newline = '') as csv_file:
for _file in files:

    file_name = _file
    with open(main_folder +'\\'+ _file,'r') as f:
        text = f.read()

        writer = csv.writer(csv_file)
        writer.writerow([file_name, text])

df = pd.read_csv('some.csv')

>Solution :

You can try to run something like this:

df = pd.DataFrame()
for _file in files:
    df = df.append(pd.read_csv(_file),  ignore_index=True, sort=False)
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading