Gist 3679500

100%

✓ Published0🌍 Public

Last edited Sep 8, 2012

Created on Sep 8, 2012

This example demonstrates a Unicode character name lookup error and its resolution through Python’s `unicodedata` module. It shows that passing a two-character string like "ɬʼ" to `unicodedata.name` raises a `TypeError`, while each individual character—"ɬ" and "ʼ"—successfully returns its official Unicode name. The code uses the `unicodedata.name` API interactively in a Python shell, with the traceback illustrating the failure mode and the subsequent successful calls. The rendering approach is plain text output from the Python interpreter, highlighting the module’s requirement for a single-character input.

AI-generated description

Similar vizzes

Loading thumbnail…

look up names of characters in text

This example maps the characters of UTF-8 text to their Unicode names. Reading input from standard input, the script decodes the text and iterates through each letter, using Python’s `unicodedata.name` to retrieve the official character name and printing it alongside the character, its code point in hexadecimal, and a Unicode escape sequence. Characters without a name are silently skipped. The output is written to standard output with UTF-8 encoding via `codecs.getwriter`.

Aamundo

66% match

Loading thumbnail…

letters.py - command line tool to look up the name of Unicode characters in a text

The tool reads UTF-8 text from standard input and prints each character’s Unicode name, its code point in hexadecimal, and a JSON-style escape sequence. It uses Python’s `unicodedata.name` to look up names, `sys.stdin.read` to capture input, and `codecs.getwriter` to ensure UTF-8 output. Characters without a defined name are skipped silently. The output is displayed as a line per character, showing the letter, name, hex code, and escaped form.

Aamundo

66% match

Loading thumbnail…

futzing with python encodings.

This example demonstrates how Python 2.7 handles Unicode strings when serializing to and from JSON. It shows a dictionary of Ethiopic syllable names being written to a file with `json.dumps`, which escapes the characters into `\u1234`-style sequences, and then reloaded with `json.load` to recover the original Unicode objects. The process highlights the round-trip fidelity of the `json` module, even though the intermediate file appears “escape-y,” and the code also uses `unicodedata.name` to fetch character names.

Aamundo

60% match

Loading thumbnail…

[print_unicode] python2 下在命令行中正常打印嵌套在 dict/list 内的 unicode 字符 #python #string

This example shows a Python 2 utility function that makes nested Unicode characters inside dictionaries and lists print correctly to the command line. The function `print_uni` takes any string `s`, applies `repr` to capture its escaped form, and then decodes that representation using `"unicode-escape"` to reveal the actual Unicode glyphs. It relies on the built-in `repr` function and the string `decode` method, demonstrating a workaround for Python 2’s default ASCII-based printing of container structures. The code is a standalone snippet from a gist by Shawn91, focusing on a common debugging pain point.

SShawn91

57% match

Loading thumbnail…

Gist 288417

This visualization displays the frequency of every character in a text file, sorted from least to most common. The Python script reads the file’s contents, counts each Unicode character’s occurrences in a dictionary, then prints the character alongside its count. It uses the `codecs` module to handle UTF-8 output and standard file reading to load the source text. The result is a plain textual ranking, not a graphical chart, with each line showing a character and its frequency.

Aamundo

54% match

Loading thumbnail…

[注音符号字母转换] #string

This example demonstrates how to normalize Unicode text that contains accented or composite characters into plain ASCII, showing the transformation before and after processing. It uses Python’s `unicodedata` module, specifically the `normalize` function with `NFKD` and `NFD` forms, followed by encoding to ASCII with the `'ignore'` error handler, or by using a dictionary of combining characters with `translate` to strip diacritical marks. The code includes two methods: one that removes non-ASCII characters entirely, and another that preserves them while eliminating accent marks, with comments explaining the behavior for both Python 2 and 3.

SShawn91

54% match

Loading thumbnail…

Gist 285672

This example demonstrates a language identification system by modeling text as bigram frequencies. It shows the most common two-letter sequences from a given input text, sorted by frequency. The Python code builds a bigram model using `ngrams` and `bigrams` functions, counts occurrences with a `defaultdict`, and removes rare bigrams below a threshold. It relies on the `collections` module and `operator.itemgetter` for sorting. The program reads a UTF-8 text file from the command line and prints the top ten bigrams with their counts.

Aamundo

52% match

Loading thumbnail…

unescape hex entities in html

This example demonstrates how to unescape HTML entities using Python's `sgmllib` and `htmlentitydefs` modules. It converts named and numeric character references (like `&`, `©`) into their corresponding Unicode characters, with special handling for common typographic entities (e.g., `’`, `©`) that are mapped to ASCII approximations when the `UNICODE_SNOB` flag is disabled. The code includes support for hex and decimal character references, handles unknown entities gracefully, and integrates entity unescaping into HTML attribute parsing. A companion `openanything.py` module provides a flexible HTTP/file/string input stream opener.

Aamundo

51% match