How to base64 encode a python array, not double-encode it?

January 6, 2022

I want to encode an array using base64 in order to transform it into JSON for sending over websocket. I followed the advice of ssubotin in this SO question in my Python code. On the receiving end, in javascript, I use window.atob() to decode the string. Trouble is, I have to use window.atob() twice. This suggests that I somehow double-encoded my data and thus made the string 33% longer than it needed to be. Some output is shown below the code.

# based on https://websockets.readthedocs.io/en/9.0.1/intro.html
import asyncio
import json
from base64 import b64encode, b64decode
import array

myarray = array.array('H', [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]) 

class Base64Encoder(json.JSONEncoder):
    # ssubotin, https://stackoverflow.com/questions/37225035/serialize-in-json-a-base64-encoded-data
    def default(self, o):
        if isinstance(o, bytes):
            return b64encode(o).decode()
        return json.JSONEncoder.default(self, o)
        

# array_event() creates a message which will be sent over websocket

def array_event():
    bytextnd = bytearray()
    for x in range(len(myarray)):
        bytextnd.extend(myarray[x].to_bytes(2, byteorder='big'))
    print(format(b64decode(b64encode(bytextnd)))) # gives bytes en/de-coded
    print(json.dumps({"type": "array", "array": b64encode(bytextnd)}, cls=Base64Encoder))
    return json.dumps({"type": "array", "array": b64encode(bytextnd)}, cls=Base64Encoder)

Python output:
b’\x00\x01\x00\x02\x00\x03\x00\x04 […] \x00\x0f\x00\x10′

{"type": "array", "array": "QUFFQUFnQURBQVFBQlFBR0FBY0FDQUFKQUFvQUN3QU1BQTBBRGdBUEFCQT0="}

Javascript yields this as the result of the first window.atob():
AAEAAgADAAQABQAGAAcACAAJAAoACwAMAA0ADgAPABA=
which correctly yields the array after a second window.atob() and some DataView and charCodeAt() magic that I borrowed from Nina Scholz.

>Solution :

your doing too much work …

a = array.array('H',range(1,17))
packed_bytes = struct.pack(f">{len(a)}H",*a)
base64_bytes = base64.b64encode(packed_bytes)
print(base64_bytes)
# b'AAEAAgADAAQABQAGAAcACAAJAAoACwAMAA0ADgAPABA='