Understanding word vectors: A tutorial for "Reading and Writing Electronic Text," a class I teach at ITP. (Python 2.7) Code examples released under CC0 https://creativecommons.org/choose/zero/, other text released under CC BY 4.0 https://creativecommons.org/licenses/by/4.0/

100%

Understanding word vectors: A tutorial for "Reading and Writing Electronic Text," a class I teach at ITP. (Python 2.7) Code examples released under CC0 https://creativecommons.org/choose/zero/, other text released under CC BY 4.0 https://creativecommons.org/licenses/by/4.0/

✓ Published0🌍 Public

Last edited Aug 11, 2017

Created on Nov 19, 2017

This tutorial demonstrates how word vectors encode semantic similarity through a progression from simple animal characteristics to high-dimensional text representations. It uses matplotlib to plot animal data points in two-dimensional space, calculating Euclidean distances between them. The notebook also employs Gensim’s Word2Vec model to create and explore word embeddings, including nearest-neighbor queries and analogical reasoning. The code is written in Python 2.7 within a Jupyter Notebook, with all examples released under CC0 and explanatory text under CC BY 4.0.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 6499e03ffdfc7479408da70f20af3625

This visualization maps the semantic relationships between words extracted from a text corpus. It shows a network of nodes and edges, where connected words cluster together based on their contextual similarity. The layout uses a force-directed graph, likely powered by d3.forceSimulation, to position nodes and links. The code loads a spaCy NLP model (`en_core_web_lg`) to compute word vectors, and filters out stop words using NLTK’s stopwords list before processing the raw text.

AArizonaTay

58% match

Loading thumbnail…

news-topics.ipynb

A 2D UMAP projection maps topic vectors extracted from a news article dataset using Top2Vec. Each point represents a topic, labeled with its top word, and colored by topic number. The notebook downloads the NewArticles dataset from Harvard Dataverse, trains a Top2Vec model, and uses UMAP’s `UMAP` class to reduce dimensions. Matplotlib and Seaborn render the scatter plot with labels and a color bar.

CCliffordAnderson

56% match

Loading thumbnail…

Gist e049bb6d7696d1b2d4ff242e68fcd062

This example analyzes text data by performing topic modeling and visualization. It shows the resulting topics and their associated terms, revealing latent themes in the dataset. The code uses NLTK for tokenization and stopword removal, TextBlob for sentiment analysis, and Gensim’s LdaModel and CoherenceModel for topic extraction. Plotly’s `scatter` and `bar` traces render the interactive visualizations, with data processed from a pandas DataFrame. The rendering approach relies on Plotly’s offline mode, initiated via `pyo.init_notebook_mode()`.

AArizonaTay

56% match

Loading thumbnail…

NLP Concepts with spaCy. Code examples released under CC0 https://creativecommons.org/choose/zero/, other text released under CC BY 4.0 https://creativecommons.org/licenses/by/4.0/

This notebook introduces core NLP concepts using the spaCy library, demonstrating how to analyze text structure through part-of-speech tagging and dependency parsing. It explains grammatical abstractions like sentences and word functions while acknowledging their limitations, particularly for poetry. The code example uses spaCy’s built-in pipeline to process English sentences, showing token attributes and linguistic annotations. Written in Python 2.7, it also mentions alternatives like NLTK and resources for other languages. The notebook’s content mixes explanatory prose with executable cells, offering a beginner-friendly tour of how language can be computationally decomposed.

Aabehmiel

55% match

Loading thumbnail…

words TSNE

A scatterplot positions words from a CSV file across a two-dimensional plane using t-SNE coordinates. The visualization maps each word by its x and y values, with axes labeled and a tooltip defined for future interaction. Built with D3 v4, the code uses `d3.csv` to load the data, `d3.scaleLinear` to set the ranges, and `d3.axisBottom` with `d3.axisLeft` to draw the axes. The words shown include abstract nouns, animals, and materials, each placed according to the original t-SNE projection.

Aandrewdblevins

52% match

Loading thumbnail…

A README file for the lecture on 2018-10-20

This example is a lecture README for a 2018 Pandas course, showing how a single gist organizes teaching materials, dependencies, documents, and a post-class survey. It demonstrates the use of Python libraries—Jupyter Lab, NumPy, Pandas, Matplotlib, Scikit-learn, and Graphviz—for data processing, visualization, and a decision-tree machine learning exercise. The text details installation via pip, conda, or pipenv, and links to tutorial notebooks, slides, and cheatsheets. The rendering approach is plain Markdown, with no interactive or programmatic visualization code present.

VVioletVivirand

52% match

Loading thumbnail…

import seenthis into gensim

This example demonstrates how to transform a dump of the social network Seenthis into a format usable by the gensim topic-modeling library, specifically adapting the Wikipedia-corpus script to generate bag-of-words and TF-IDF vector representations. The code shows a pipeline that reads raw posts, strips markup using regex-based cleaning (including removal of comments, templates, and links), tokenizes the text, and then builds a dictionary and sparse matrix outputs in Matrix Market format. It relies on gensim’s `Dictionary`, `TextCorpus`, and `MmCorpus` classes, plus utility functions like `filter_wiki` for preprocessing, mirroring the original `wikicorpus.py` structure. The output files—a word-ID mapping and two corpus files—allow downstream topic modeling with gensim, though the visualization itself is not shown; the code’s focus is on data preparation rather than graphical display.

PPhilippe Rivière

51% match

Loading thumbnail…

Gist e3f5949ff1f99ab4126a

This notebook demonstrates Python string formatting through a series of interactive code cells. It shows how variables like `animal`, `verb`, and `noun` are assigned values and then inserted into template sentences using the `.format()` method from the Python standard library. The examples progress from simple variable printing with `print` statements to building sentences with placeholders like `{noun}` and `{verb}`, which are replaced with actual values, including numbers, demonstrating flexible substitution. The notebook uses standard Python 3 syntax and pandas for data handling, with matplotlib inline plotting enabled for visualization context.

Aakeshavan

50% match