futzing with python encodings.

100%

futzing with python encodings.

✓ Published0🌍 Public

Last edited Mar 11, 2014

Created on Mar 11, 2014

This example demonstrates how Python 2.7 handles Unicode strings when serializing to and from JSON. It shows a dictionary of Ethiopic syllable names being written to a file with `json.dumps`, which escapes the characters into `\u1234`-style sequences, and then reloaded with `json.load` to recover the original Unicode objects. The process highlights the round-trip fidelity of the `json` module, even though the intermediate file appears “escape-y,” and the code also uses `unicodedata.name` to fetch character names.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 3679500

This example demonstrates a Unicode character name lookup error and its resolution through Python’s `unicodedata` module. It shows that passing a two-character string like "ɬʼ" to `unicodedata.name` raises a `TypeError`, while each individual character—"ɬ" and "ʼ"—successfully returns its official Unicode name. The code uses the `unicodedata.name` API interactively in a Python shell, with the traceback illustrating the failure mode and the subsequent successful calls. The rendering approach is plain text output from the Python interpreter, highlighting the module’s requirement for a single-character input.

Aamundo

60% match

Loading thumbnail…

Extra escapey JSON encoding for node.js

This example demonstrates how to encode JavaScript objects into JSON strings with extra escaping for safety and compatibility. The code defines a function, `escapedStringify`, which wraps the native `JSON.stringify` to apply three additional transformations: escaping forward slashes as `\/`, converting angle brackets to uppercase Unicode escape sequences, and converting all characters in the `\u007f` to `\uffff` range into lowercase hex escapes. The implementation uses the `JSON` stringify API and regular expressions with callback functions to perform the character replacements. The example’s data source is the Gist’s own JSON test input, and the rendering approach is purely textual output.

RRandomEtc

59% match

Loading thumbnail…

fussing with UTF-16 surrogate encoding in Javascript

This example demonstrates how JavaScript encodes Unicode code points above U+FFFF into UTF-16 surrogate pairs, using functions like `escapeUTF16` and `makeSurrogate` to convert a code point such as U+1F339 (a rose emoji) into the escaped `\ud83c\udf39` sequence. The code implements the surrogate math from the Wikipedia UTF-16 specification, applying bitwise shifts and masking to derive lead and trail surrogates for supplementary planes. It then verifies the result by parsing the escaped string with `JSON.parse` and comparing it to the literal surrogate pair. The source is a standalone gist with no external libraries or data visualization dependencies.

RRandomEtc

58% match

Loading thumbnail…

Gist 2768d4a5ea98e385b30ffc99390cddb7

This example demonstrates a custom JSON encoder for serializing graph data structures. It shows how Python’s `json.JSONEncoder` is extended to convert `graph.Node` and `graph.Relationship` objects into a plain dictionary format, capturing element IDs, labels, properties, and relationship endpoints. The encoder uses `json.dumps` under the hood but overrides the `default` method to handle graph-specific types. The code relies on the `graph` module’s `Node` and `Relationship` classes, and its output is a JSON-friendly representation suitable for further visualization or data exchange.

Aakollegger

55% match

Loading thumbnail…

letters.py - command line tool to look up the name of Unicode characters in a text

The tool reads UTF-8 text from standard input and prints each character’s Unicode name, its code point in hexadecimal, and a JSON-style escape sequence. It uses Python’s `unicodedata.name` to look up names, `sys.stdin.read` to capture input, and `codecs.getwriter` to ensure UTF-8 output. Characters without a defined name are skipped silently. The output is displayed as a line per character, showing the letter, name, hex code, and escaped form.

Aamundo

55% match

Loading thumbnail…

look up names of characters in text

This example maps the characters of UTF-8 text to their Unicode names. Reading input from standard input, the script decodes the text and iterates through each letter, using Python’s `unicodedata.name` to retrieve the official character name and printing it alongside the character, its code point in hexadecimal, and a Unicode escape sequence. Characters without a name are silently skipped. The output is written to standard output with UTF-8 encoding via `codecs.getwriter`.

Aamundo

54% match

Loading thumbnail…

[print_unicode] python2 下在命令行中正常打印嵌套在 dict/list 内的 unicode 字符 #python #string

This example shows a Python 2 utility function that makes nested Unicode characters inside dictionaries and lists print correctly to the command line. The function `print_uni` takes any string `s`, applies `repr` to capture its escaped form, and then decodes that representation using `"unicode-escape"` to reveal the actual Unicode glyphs. It relies on the built-in `repr` function and the string `decode` method, demonstrating a workaround for Python 2’s default ASCII-based printing of container structures. The code is a standalone snippet from a gist by Shawn91, focusing on a common debugging pain point.

SShawn91

53% match

Loading thumbnail…

ndjson-to-file

This example explores the challenge of writing large JSON datasets to disk. It demonstrates a failing workaround that serializes data in chunks to multiple files, and two broken attempts at using streaming libraries. The code uses `JSON.stringify` with `fs.writeFileSync` for the chunked approach, and attempts to use the `ndjson` and `JSONStream` libraries with `fs.createWriteStream` and piping for streaming output. The example highlights the difficulty of handling big data blobs that crash `JSON.stringify` and the struggle to implement a proper streaming solution.

11wheel

53% match