Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

CSV file formatting doesn't work for GCP BlobWriter

I have same data as list of dict and want to import these in a CSV file in a GCP bucket.

I’m importing these object as streaming:

blob = defaults.bucket.blob(filename)
csv_writer = BlobWriter(blob, content_type="text/csv")

for data in result.get('events'):
    _source = data.get('_source', {})
    csv_writer.write(json.dumps(_source, indent=2).encode('utf-8'))  

where _source is dict.

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

The actual result in the CSV file is JSON formatting, not CSV.

For example:

{           
  'key':'value'     
}{          
  'key':'value'         
}{      

>Solution :

import csv

blob = defaults.bucket.blob(filename)
csv_writer = BlobWriter(blob, content_type="text/csv")

# Assuming '_source' is a dictionary and result.get('events') is a list of dictionaries
field_names = ['key1', 'key2', 'key3']  # Replace with actual keys in your '_source' dictionaries

# Write CSV header
csv_writer.write(','.join(field_names).encode('utf-8') + b'\n')

for data in result.get('events'):
    _source = data.get('_source', {})
    
    # Convert each dictionary to a CSV row
    csv_row = [str(_source.get(key, '')) for key in field_names]
    
    # Write CSV row to the file
    csv_writer.write(','.join(csv_row).encode('utf-8') + b'\n')
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading