Python in-memory GZIP on existing file

October 13, 2023

I have a situation where I have an existing file. I want to compress this file using gzip and get the base64 encoding of this file and use this string for latter operations including sending as part of data in an API call.

I have the following code which works fine:

import base64
import gzip


base64_string_to_use_later = None
with open('C:\\test.json', 'rb') as orig_file:
   with gzip.open('C:\\test.json.gz', 'wb') as zipped_file:
        zipped_file.writelines(orig_file)
        
with gzip.open('C:\\test.json.gz', 'rb') as zipped_file:
        base64_string_to_use_later = base64.b64encode(zipped_file.read())

This code will take the existing file, create a compressed version and write this back to the file system. The second block takes the compressed file, opens it and fetches the base 64 encoded version.

Is there a way to make this more elegant to compress the file in memory and retrieve the base64 encoded string in memory?

>Solution :

Use gzip.compress() to compress the data in memory instead of writing to a file.

import base64
import gzip

with open('C:\\test.json', 'rb') as orig_file:
    base64_string_to_use_later = base64.b64encode(gzip.compress(orig_file.read()))