Skip to main content
100%

Bi-term Topic Model implementation in pure Python

✓ Published0🌍 Public
Aabehmiel
Last edited May 2, 2017
Created on Mar 5, 2018

This visualization demonstrates the Bi-Term Topic Model (BTM) in action, showing how topic distributions emerge from short-text co-occurrence patterns. The code implements a pure Python Gibbs sampler that infers latent topics from observed biterms, with the model's parameters visualized as topic-term probability matrices. It uses the `BTMGibbsSampler` class to iteratively update topic assignments via multinomial sampling, relying on the `random` module for stochastic draws and custom normalization functions. The rendering shows the inferred `BTModel`'s topic proportions and per-topic word distributions, highlighting how BTM handles sparse short-text data through biterm aggregation.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist e049bb6d7696d1b2d4ff242e68fcd062

This example analyzes text data by performing topic modeling and visualization. It shows the resulting topics and their associated terms, revealing latent themes in the dataset. The code uses NLTK for tokenization and stopword removal, TextBlob for sentiment analysis, and Gensim’s LdaModel and CoherenceModel for topic extraction. Plotly’s `scatter` and `bar` traces render the interactive visualizations, with data processed from a pandas DataFrame. The rendering approach relies on Plotly’s offline mode, initiated via `pyo.init_notebook_mode()`.

AArizonaTay
64% match
Loading thumbnail…

Gist da5f7a5f8cf01b0874d92349b33ec806

This visualization uses a Latent Dirichlet Allocation (LDA) topic model to classify Instagram captions into eight thematic categories, such as Giveaway, Fitness, and Love. It first shows the derived topics with their top ten keywords per topic, then assigns each caption its dominant topic and displays a random sample of the resulting annotations. The analysis relies on the Gensim library for the LDA model and coherence scoring, while Pandas DataFrames and the Dictionary and doc2bow APIs handle text preprocessing and corpus construction. A custom function maps each caption’s topic distribution to its most likely label, and the final output presents a sample of the enriched data with the assigned topic column.

AArizonaTay
55% match
Loading thumbnail…

Gist d2f2bc6be1716820160ed5183ea07eaa

This example shows a browser-based text analytics pipeline that scrapes and processes web content before visualizing topic models. The code first uses Selenium with Chrome WebDriver to scrape data, then applies NLP libraries including NLTK, spaCy, TextBlob, and gensim’s LdaModel for topic modeling and coherence scoring. The final visualization employs Plotly’s `plotly.express` and `plotly.graph_objs` to render interactive charts, with `pyo.init_notebook_mode()` enabling notebook display. The rendering approach is evident from the use of `go` traces and `px` figures, though the specific chart type is not shown in the code snippet. The data source is dynamic web pages, as indicated by the Selenium scraping logic.

AArizonaTay
53% match
Loading thumbnail…

Gist 7518328

This example computes and displays the most relevant terms (including single words, bigrams, and trigrams) from a pair of documents by calculating term frequency–inverse document frequency (TF-IDF). It first tokenizes Portuguese text using NLTK’s `RegexpTokenizer` and removes stopwords, then builds a vocabulary of unigrams, bigrams, and trigrams using `nltk.bigrams` and `nltk.trigrams`. For each document, the code outputs each term’s raw frequency, TF, IDF, and TF-IDF score, and finally prints a ranked list of the highest scoring terms across all documents. The script relies solely on Python’s `math` module and NLTK, with no external visualization library.

TTheLady
52% match
Loading thumbnail…

import seenthis into gensim

This example demonstrates how to transform a dump of the social network Seenthis into a format usable by the gensim topic-modeling library, specifically adapting the Wikipedia-corpus script to generate bag-of-words and TF-IDF vector representations. The code shows a pipeline that reads raw posts, strips markup using regex-based cleaning (including removal of comments, templates, and links), tokenizes the text, and then builds a dictionary and sparse matrix outputs in Matrix Market format. It relies on gensim’s `Dictionary`, `TextCorpus`, and `MmCorpus` classes, plus utility functions like `filter_wiki` for preprocessing, mirroring the original `wikicorpus.py` structure. The output files—a word-ID mapping and two corpus files—allow downstream topic modeling with gensim, though the visualization itself is not shown; the code’s focus is on data preparation rather than graphical display.

PPhilippe Rivière
51% match
Loading thumbnail…

news-topics.ipynb

A 2D UMAP projection maps topic vectors extracted from a news article dataset using Top2Vec. Each point represents a topic, labeled with its top word, and colored by topic number. The notebook downloads the NewArticles dataset from Harvard Dataverse, trains a Top2Vec model, and uses UMAP’s `UMAP` class to reduce dimensions. Matplotlib and Seaborn render the scatter plot with labels and a color bar.

CCliffordAnderson
51% match
Loading thumbnail…

using-pretrained-abstract-to-tweet-model.ipynb

This example shows how a pre-trained transformer model converts academic abstracts into concise, tweet-sized summaries, demonstrating a text-to-text generation workflow. It loads a Hugging Face `AutoModelForSeq2SeqLM` and `AutoTokenizer` from the `andersoncliffb/abstracts_to_tweet_model` repository, then creates a `text2text-generation` pipeline to produce the output. The notebook’s code uses the Transformers library’s pipeline API, with the model and tokenizer specified directly. It does not include a data visualization, instead focusing on the model’s generative output from input text.

CCliffordAnderson
51% match
Loading thumbnail…

Topic Model Viz

This visualization displays a topic model’s data as a vertical column of small purple squares, with each square’s opacity reflecting the value in the second data column. It reads data from a remote CSV file via `d3.csv`, then renders the first 2,000 records as an SVG using `selectAll('rect')` and `insert('rect')`. Each rectangle is positioned 10 pixels apart, with fixed width and height of 8 pixels. The example uses D3 v5 and demonstrates a simple, minimal approach to encoding quantitative values through opacity.

DDavid Emmanuel Katz
51% match