Send in-memory bytes (file) over multipart/form-data POST request. Python

August 6, 2022

;TLDR

I want to send a file with requests.send() using multipart/form-data request without storing the file on a hard drive. Basically, I’m looking for an alternative for open() function for bytes object

Hello, I’m currently trying to send multipart/form-data request and pass in-memory files in it, but I can’t figure out how to do that.

My app receives images from one source and sends them to another. Currently it sends get request directly to file, (e.g. requests.get('https://service.com/test.jpeg')), reads image’s bytes and writes them into new file on the hard drive. The sending code that works looks like this:

def send_file(path_to_image: str)
    url = get_upload_link()
    data = {'photo': open(path_to_image, 'rb')}
    r = requests.post(url, files=data)

send_file("test.jpeg")

The main issue I have with this approach is that I have to keep files on my hard drive. Sure, I can use my drive as some sort of a "temporary buffer" and just delete them after I no longer need these files, but I believe there’s much more simple way to do that.

I want my function to receive bytes object and then send it. I actually tried doing that, but the backend doesn’t accept them. Here’s what I tried to do

Attempt 1

def send_file(image: bytes)
    url = get_upload_link()
    data = {'photo': open(image, 'rb')}
    r = requests.post(url, files=data)

I get "ValueError: embedded null byte"

Attempt 2

def upload_photo(image: bytes):
    url = get_upload_link()
    file = BytesIO(image)
    data = {'photo': file}

    r = requests.post(url, files=data)

Backend server doesn’t process my files correctly. It’s like passing files=None, same response

I also tried:

sending the returning value of the methods: file.getbuffer() and file.read()
file.write(image) and then sending file
StringsIO object

etc.

Final notes

I noticed, that open() returns _io.BufferedReader object. I also looked for a way to construct its instance, but couldn’t fund a way. Can someone help me, please?

UPD:
If anyone is interested, the receiving api is this

>Solution :

From the official documentation:

POST a Multipart-Encoded File
…
If you want, you can send strings to
be received as files:
url = 'https://httpbin.org/post'
files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}

r = requests.post(url, files=files)