I need to create a string consisting of a single byte corresponding to an integer of at most 255. It is acceptable that the string is not valid Unicode. Code: import ( "fmt" "strings" ) func main() { n := 255 s := "" s += string(byte(n)) fmt.Printf("Method 1: %x\n", s) sb := strings.Builder{} sb.WriteByte(byte(n))… Read More Why does Go interpret byte as Unicode code point when appending to string?

I have some strings in Python loaded from a file. They look like lists, but are actually strings, for example: example_string = ‘["hello", "there", "w\\u00e5rld"]’ I can easily convert it into an actual list of strings: def string_to_list(string_list:str) -> List[str]: converted = string_list.replace(‘"’, ”).replace(‘[‘, ”).replace(‘]’, ”).split(‘,’) return [s.strip() for s in converted] as_list = string_to_list(example_string)… Read More How to make Python treat literal string as UTF-8 encoded string

Is there any Unicode codepoint that one of the bytes in its utf-8 representation is the ESC byte (0x1b)? Context: The ESC byte is used in ANSI escape codes (in terminals) and I’d like to know whether that byte can appear as part of a utf-8 byte sequence. >Solution : No, all bytes in UTF-8… Read More Does UTF-8 use the ESC byte?

I’m trying to get a title of html document in python, but getting weird symbols. I guess that’s because of encoding, but the html doc in utf-8 encoding. Is there any way I can get normal letters? Here is code and what am I getting: from bs4 import BeautifulSoup with open("index.html") as file: src =… Read More Gettin non-english text from html doc

I have raw data that looks like this: 25023,Zwerg+M%C3%BCtze,0,1,986,3780 25871,red+earth,0,1,38,8349 25931,K4m%21k4z3,90,1,1539,2530 It is saved as a .txt file: https://de205.die-staemme.de/map/player.txt The "characters" starting with % are unicode, as far as I can tell. I found the following table about it: https://www.i18nqa.com/debug/utf8-debug.html Here is my code so far: urllib.urlretrieve(url,pfad + "player.txt") f = open(pfad + "player.txt","r",encoding="utf-8") raw… Read More How to convert utf-8 characters to "normal" characters in string in python3.10?

I got some text encoded to bytes using utf-8 encoding. When processing this text I incautiously used str() to make it a Unicode string because I assumed this would automatically decode the bytes object with the right encoding. This, however, is not the case. For example: a = "عجائب" a_bytes = a.encode(encoding="utf-8") b = str(a_bytes)… Read More Python: What is the string representation of a bytes object really and how to get back the bytes object?