Refactored autoscan_greek.py for CLTK

100%

Refactored autoscan_greek.py for CLTK

✓ Published0🌍 Public

Last edited Aug 5, 2015

Created on Aug 5, 2015

This visualization demonstrates the prosimetric scansion of Ancient Greek texts by tokenizing cleaned input into sentences and syllables, then applying quantitative metrical rules to classify each syllable as long or short. The Python program uses the CLTK-inspired `ScansionGreek` class, which handles accent stripping, punctuation removal, and diphthong/vowel-length detection through custom character lists and replacement dictionaries. It relies on standard Python string operations and list comprehensions to process the text, with the final output showing the scansion patterns for each sentence. The code also includes a unittest framework for testing the scansion logic.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 4e556c8ccbb579e82ba9

This visualization scans Greek prose and poetry to produce a prosimetric scansion, showing syllable lengths (long or short) across sentences. It first cleans and tokenizes the text using the Classical Language Toolkit’s `TokenizeSentence` and NLTK’s `PunktLanguageVars`, then syllabifies and applies Greek metrical rules such as diphthongs, long vowels, and consonant clusters. The code defines a `Scansion` class with methods like `_long_by_nature` to classify syllables, ultimately outputting the metrical pattern for each line of the input text.

TTylerKirby

78% match

Loading thumbnail…

Gist 641fc44c27c66ee9ef09

This example demonstrates a Latin text preprocessing pipeline that converts raw words into lemmata and macronized dictionary entries. It shows how a sample sentence like "quo usque tandem abutere catilina nostra patientia" is tokenized, cleaned of punctuation, and processed word-by-word. The code uses the CLTK `LemmaReplacer` for lemmatization, lxml’s `etree` to parse the Perseus Latin lexicon XML file, and custom string manipulation to transform entries like "a^bu_tor" into "abūtor". The workflow defines classes for text and word-level morphological analysis, though the dictionary lookup for full texts runs too slowly to complete.

TTylerKirby

64% match

Loading thumbnail…

Gist d117b3d6bcfdf1722bd5

This example analyzes the prosody of Latin prose by counting occurrences of specific metrical patterns, called clausulae, in a string of syllable stress marks. The Clausulae class uses Python’s `str.count` method to tally matches for eighteen named rhythm patterns, such as “cretic + trochee” and “double spondee,” against a concatenated prosody input. It returns a dictionary mapping each pattern name to its frequency, providing a quantitative breakdown of rhythmic endings. The code is plain Python with no external visualization libraries, focusing solely on the text-analysis computation.

TTylerKirby

63% match

Loading thumbnail…

Gist e049bb6d7696d1b2d4ff242e68fcd062

This example analyzes text data by performing topic modeling and visualization. It shows the resulting topics and their associated terms, revealing latent themes in the dataset. The code uses NLTK for tokenization and stopword removal, TextBlob for sentiment analysis, and Gensim’s LdaModel and CoherenceModel for topic extraction. Plotly’s `scatter` and `bar` traces render the interactive visualizations, with data processed from a pandas DataFrame. The rendering approach relies on Plotly’s offline mode, initiated via `pyo.init_notebook_mode()`.

AArizonaTay

50% match

Loading thumbnail…

Gist 0d3ef677475862ba439fd5dacfd4a60d

The visualization maps the scoring patterns of five-letter words, showing how letter positions and letter values contribute to total word scores. It displays a grid or distribution of words, likely sorted or clustered by score, with each word’s score calculated from per-letter point values and a bonus for non-repeated letters. The implementation uses Python’s pandas to load and transform the word list, adding columns for each letter’s position and score, while the letter-to-value mapping follows standard Scrabble tile distributions. The final score sums individual letter scores and adds a two-point bonus when all letters are unique, as seen in the `duplicates` logic. The rendering itself is not shown in the code, but the data processing relies on pandas dataframes and string operations to derive the visualization’s underlying dataset.

CConorAspell

50% match

Loading thumbnail…

Gist 55fc864ec9cea1107845d3ed7a3dab8b

This example demonstrates a text-cleaning pipeline for caption data, showing how raw captions are converted into tokenized, filtered lists for downstream analysis. The visualization itself is not rendered; instead, the focus is on the data preprocessing logic. The code uses `pandas` with `apply` to run natural language processing via `spaCy` (`nlp`), extracting tokens from the `Caption` column. It then applies a custom `tokens` function that removes stop words and non-alphanumeric characters using `re.match`, while preserving tokens containing "@" symbols. The result is a new `clean_caption` column, highlighting a common step in text mining workflows.

AArizonaTay

49% match

Loading thumbnail…

Gist 6437893f183d99e863d835425ffa46f8

This example visualizes the distribution of tokenized caption text from a dataset of images, showing how frequently each word appears after cleaning. The visualization displays the processed terms as a bar chart or word cloud, with the data derived from the `Caption` field of the input dataframe. The preprocessing pipeline loads the `en_core_web_lg` spaCy model to tokenize each caption, removes English stop words and non-alphanumeric characters, and converts all terms to lowercase. The code relies on the `spacy` library for linguistic analysis and standard Python regex operations via the `re` module to filter tokens, while the `stopwords` set comes from NLTK.

AArizonaTay

48% match

Loading thumbnail…

Gist 6ab67c7133edab5b3b01760a3f226a25

This example compares the frequency of uppercase letters in the King James and Luther Bibles, printing a percentage of all uppercase letters per book. Using Python’s `re` module with custom regular expressions, it filters out lines with fewer than four words, skips verses’ opening formatting, and excludes all-uppercase words before counting letters. The output reports each translation’s uppercase share, revealing stylistic differences in capitalization. The script reads local text files (`kjv.txt`, `luther.txt`) and computes letter counts via `matchLetters` and `matchUCLetters` patterns.

HHoffmannP

47% match