search for a string in multiple csv files in python using pandas

July 10, 2023

I am trying to search a string/value in multiple csv files and save that in a new csv file. My code doesn’t give me any error but returns an empty csv file

Here’s my code:

import pandas as pd
import glob
path='path/to/multiple/csv/files'

files=glob.glob(path + '/*.csv')
for filename in files:
    df=pd.read_csv(filename,sep=',')
    x=df[df.apply(lambda col: col.astype(str).str.contains('myvalue').any(),axis=1)]

x.to_csv('result.csv',index=False)

>Solution :

Create empty list, fill it by DataFrames and for final DataFrame join together by concat:

files=glob.glob(path + '/*.csv')

out = []
for filename in files:
    df=pd.read_csv(filename,sep=',')
    x=df[df.apply(lambda col: col.astype(str).str.contains('myvalue').any(),axis=1)]
    out.append(x)

pd.concat(out).to_csv('result.csv',index=False)