How to upload large files using POST method in Python?

July 8, 2022

I’m uploading a large file (about 2GB) to an API that accepts POST method using requests module of Python, which results in loading the file to the memory first and increasing memory usage significantly. I believe there will be some other ways to stream the file to the API without burdening the memory. Any suggestions?

>Solution :

Any suggestions?

Use Streaming Upload, as docs put it:

Requests supports streaming uploads, which allow you to send large
streams or files without reading them into memory. To stream and
upload, simply provide a file-like object for your body:
with open('massive-body', 'rb') as f:
    requests.post('http://some.url/streamed', data=f)