Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

How we can parse large CSV file and then extract the specific columns in to a list?

There is a CSV file with many rows and 30 columns. What I wanted is to get the data from columns 3,6, and 15 and then save it in a list.

Using Python how can I achieve this so that I dont have to load the entire file into the memory?

Any suggestions?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

I guess in python you can use ‘islice’ function from the ‘itertools’ model.
Here’s an example code for this:

  import csv
from itertools import islice

columns_to_extract = [2, 5, 14]  

with open('filename.csv', 'r') as f:
    reader = csv.reader(f)
    extracted_data = [[] for _ in range(len(columns_to_extract))]
    for row in islice(reader, 1, None):  # Skip header row (if any)
        for i, col_idx in enumerate(columns_to_extract):
            extracted_data[i].append(row[col_idx])
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading