Gist 2116a7bda7ef9a6364f3

100%

Gist 2116a7bda7ef9a6364f3

✓ Published0🌍 Public

Last edited Sep 5, 2014

Created on Sep 5, 2014

This example demonstrates how to use NLTK’s frequency distribution object to analyze word frequencies in the pre‑loaded text of Monty Python and the Holy Grail. It first displays the top 50 most frequent words, then guides the viewer through tasks that involve examining the raw text, modifying it, and rerunning the frequency calculation. A cumulative frequency plot is generated, and a list comprehension isolates words longer than five characters that occur at least five times, sorted alphabetically. The code relies on NLTK’s `FreqDist` and the `nltk.book` collection.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 4be75194a32c761f2a20

This example demonstrates how to use NLTK's frequency distribution object to analyze word frequencies in the Monty Python and the Holy Grail text (text6). The code imports the NLTK book module, creates a FreqDist from the text, and displays the top 50 word-frequency pairs, revealing unexpected results. The notebook then poses tasks for modifying the text and generating cumulative frequency plots, while also showing list comprehensions to filter words by length and frequency. The analysis relies on NLTK's FreqDist and the pre-loaded text corpus from the nltk.book module.

MMartiHearst

95% match

Loading thumbnail…

Example from Programming4Humanists

This example visualizes word frequency in a text by processing the poem’s lines and filtering out common stopwords. It shows a ranked list of words by their occurrence count, displayed in descending order. The code uses XQuery 3.0 with functions like `fn:tokenize`, `fn:distinct-values`, and `fn:count` to compute frequencies, while reading the source text and stopword list from gist URLs. The rendering approach is a simple textual output of word elements, each carrying a `frequency` attribute, rather than a graphical chart.

CCliffordAnderson

65% match

Loading thumbnail…

Gist af6ed0a95b476ea7f82b0576a93a2306

This visualization displays the 30 most common words from a dataset of cleaned captions, using a frequency distribution to show which terms appear most often. The code filters out common exclusions like "1," "2," and "3" before processing, then relies on the Natural Language Toolkit’s `FreqDist` to count word occurrences. A bar chart is generated with `matplotlib.pyplot`, where the horizontal axis lists the top words and the vertical axis shows their raw frequencies. The plot is sized at 14 by 6 inches for readability, and the resulting figure is displayed directly via `plt.show()`, with the source being an internal data frame of cleaned captions.

AArizonaTay

60% match

Loading thumbnail…

Knuth/McIlroy word count task using Haskell

This example demonstrates a word-frequency analysis task using Haskell, showing how to compute and display the most common words from standard input. It chains data transformations through a pipeline: converting text to lowercase, replacing non-letter characters with newlines, splitting into words, and tallying counts with a hash map. The code relies on `Data.HashMap.Strict` for efficient key-value aggregation via `HashMap.fromListWith`, and uses `Data.List` for sorting and taking the top *n* results. The final output lists each word with its count, sorted in descending order of frequency. The example highlights functional composition and lazy evaluation in a concise, read-backwards data-flow style.

FFranklin Chen

58% match

Loading thumbnail…

Gist fbc8e33e01b5be01bc6c1ab2d1d08cfc

The visualization displays the 30 most frequently occurring words from a set of cleaned Instagram captions, plotted as a bar chart that ranks terms by raw count. It shows a sharp drop-off in frequency after the top few words, highlighting the long-tail distribution common in text data. The chart is generated using Python’s matplotlib, with the caption data loaded into a pandas DataFrame and tokenized via the Natural Language Toolkit (nltk). Frequencies are computed with `nltk.FreqDist`, and the top 30 entries are selected for plotting, with a filter excluding the characters ‘1’, ‘2’, and ‘3’ from the hashtag tokens. The figure is sized at 14 by 6 inches for readability.

AArizonaTay

57% match

Loading thumbnail…

Gist 18319c4f30bf20182aad

This example uses an iPython Notebook to teach basic Python and Markdown skills. It shows a sequence of tasks for editing text cells, writing code, and manipulating strings, using the NLTK library to process a sentence. The notebook’s cells contain instructional headings, markdown explanations, and Python code inputs with output. The visualization is the notebook itself, rendered as a static document with WebGL, demonstrating its interactive, code-driven structure.

MMartiHearst

56% match

Loading thumbnail…

This set of functions produces word frequencies for any given sequence of text nodes.

This example computes word frequencies from a sequence of text nodes and returns them as an ordered list of word elements. It first shows the frequency-ranked vocabulary by normalizing the input—converting to lowercase, stripping punctuation, and tokenizing—before counting occurrences of each distinct token. The code uses XQuery 3.0 with functions like `fn:tokenize`, `fn:distinct-values`, and `fn:count` to group and sort the results. The final output is an XML `<words>` element containing individual `<word>` elements with a `frequency` attribute, ordered from most to least frequent. The data source is the `//tei:l` elements from a TEI-encoded text.

CCliffordAnderson

55% match

Loading thumbnail…

Gist e049bb6d7696d1b2d4ff242e68fcd062

This example analyzes text data by performing topic modeling and visualization. It shows the resulting topics and their associated terms, revealing latent themes in the dataset. The code uses NLTK for tokenization and stopword removal, TextBlob for sentiment analysis, and Gensim’s LdaModel and CoherenceModel for topic extraction. Plotly’s `scatter` and `bar` traces render the interactive visualizations, with data processed from a pandas DataFrame. The rendering approach relies on Plotly’s offline mode, initiated via `pyo.init_notebook_mode()`.

AArizonaTay

54% match