I’m using the below code to recursively get all the paths in the folder and push them one by one into the google cloud storage bucket. The problem is, it’s extremely slow. I have around 30-40K files that need to be pushed every day and each one is taking like 0.25 to 0.5 second to… Read More Google Cloud Storage pushing the files into bucket very slowly

I’ve created a json object to store it in Cloud Storage but I need it to be converted as newline Json standard format so BigQuery can read it. This is my code: items = [] for item in item_list: item = {‘key’: item} items.append(item) The actual current output looks like this: [{‘item’: ‘stuff’}, {‘item’: ‘stuff’},… Read More create array of object with new line ( newline Json standard format conversion for BigQuery)

I am trying to upload some data to firebase storage by using an upload URL instead of the traditional way by using: Storage.storage().reference().child("Name").putFile("file name") The reason that I need the URL instead of going the traditional route is because I need to use URLSession to upload data so my task can finish uploading in the… Read More How do you get the upload URL of a backend server so you can upload data to it?