Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Jupyter Notebook specify path to directory for concatenation of multiple .csv files

The following code works great for concatenating multiple .csv files into one. All of these .csv files reside in the same directory. The problem is that it only works if my current file is in the same directory as those multiple .csv files. I have tried different syntaxes to specify the path to the directory where those mutliple .csv files reside, but no success. I wondered where I should specify the path to the .csv files directory in the below code. Btw, I am working in Jupyter Notebook:

import pandas as pd
import os

filepaths = [f for f in os.listdir(".") if f.endswith('.csv')]
df = pd.concat(map(pd.read_csv, filepaths)

>Solution :

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

os.listdit(dir) lists the files in the path dir. In your example, you have dir='.', which corresponds to the current working directory (the directory from where you run your script). You can change this variable to the directory where your .csv files reside.

You can avoid using endswith() by globbing,

import pandas as pd
import os
import glob

base_dir = os.path.join('path', 'to', 'files')
filepaths = [f for f in glob.glob(f'{base_dir}*.csv')]
df = pd.concat(map(pd.read_csv, filepaths)

This expands the wildcard * to find all files that ends with .csv in base_dir.

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading