Home Google Colab, loop through google drive folder, read each CSV file in folder into a datframe, then append dataframe

Questions

Google Colab, loop through google drive folder, read each CSV file in folder into a datframe, then append dataframe

byMR

January 17, 2023

I am using Google Colab

I have a folder of CSV files in a google drive directory
'/content/drive/MyDrive/Colab Notebooks/Stocks CSVs/Stocks/'

I want to loop through this folder, read each CSV to a data frame, append this data frame with technical indicators from the pandas_ta library, then export the appended data frame back to another google drive folder with the path
'/content/drive/MyDrive/Colab Notebooks/Stocks CSVs/Stocks/'

With the code below I am receiving the following error:
FileNotFoundError: [Errno 2] No such file or directory: ‘Copy of 138SL.csv’

How can I fix this code, bear in mind I have to use Google Colab as I am using a work laptop and not allowed to download text editors such as VS code.

I have the following code:

def TAfunction(data):
  stock_df = pd.read_csv(data)
  CustomStrategy = ta.Strategy(
    name="RSI Strat",
    description = "RSI",
    ta=[
        {"kind":"rsi"},
        {"kind":"bbands", "length": 20},
        {"kind":"macd", "fast": 8, "slow":21},]
)
stock_df.ta.strategy(CustomStrategy)
stock_df.to_csv(data, encoding = 'utf-8-sig')
files.download(data) 

test_dir = '/content/drive/MyDrive/Colab Notebooks/Stocks CSVs/UpdatedStocks/'

for file in os.listdir(test_dir):
  if file.endswith(".csv"):
    TA_function(file)

>Solution :

You need to pass the full path of the CSV file to TAfunction(), because the script does not have the same working directory as where the files are stored. Simply modify the end of your code to join test_dir and file:

test_dir = '/content/drive/MyDrive/Colab Notebooks/Stocks CSVs/UpdatedStocks/'

for file in os.listdir(test_dir):
    if file.endswith(".csv"):
        TAfunction(test_dir + file)