Advertisements I have same data as list of dict and want to import these in a CSV file in a GCP bucket. I’m importing these object as streaming: blob = defaults.bucket.blob(filename) csv_writer = BlobWriter(blob, content_type="text/csv") for data in result.get(‘events’): _source = data.get(‘_source’, {}) csv_writer.write(json.dumps(_source, indent=2).encode(‘utf-8’)) where _source is dict. The actual result in the CSV… Read More CSV file formatting doesn't work for GCP BlobWriter

Advertisements The Google Cloud signed URL documentation says: "Anyone who knows the URL can access the resource until the expiration time for the URL is reached or the key used to sign the URL is rotated." How can I tell which key was used to sign a URL, and when it will be rotated? If… Read More Google Cloud signed URL invalidation

Advertisements I’m using the below code to recursively get all the paths in the folder and push them one by one into the google cloud storage bucket. The problem is, it’s extremely slow. I have around 30-40K files that need to be pushed every day and each one is taking like 0.25 to 0.5 second… Read More Google Cloud Storage pushing the files into bucket very slowly

Advertisements I’ve created a json object to store it in Cloud Storage but I need it to be converted as newline Json standard format so BigQuery can read it. This is my code: items = [] for item in item_list: item = {‘key’: item} items.append(item) The actual current output looks like this: [{‘item’: ‘stuff’}, {‘item’:… Read More create array of object with new line ( newline Json standard format conversion for BigQuery)

Advertisements I am trying to upload some data to firebase storage by using an upload URL instead of the traditional way by using: Storage.storage().reference().child("Name").putFile("file name") The reason that I need the URL instead of going the traditional route is because I need to use URLSession to upload data so my task can finish uploading in… Read More How do you get the upload URL of a backend server so you can upload data to it?