Skip to main content
100%

Python Utilities for Tweets

✓ Published0🌍 Public
Aabehmiel
Last edited Jan 17, 2015
Created on Jul 11, 2017

This utility module processes raw tweet JSON objects for text analysis. It extracts timestamps via `datetime.strptime`, hashtags, user mentions, and cleaned text by stripping URLs, mentions, and media using entity indices. It further sanitizes punctuation with the `string` module, removes English stopwords from `nltk.corpus`, and applies Lancaster stemming via `nltk.stem.lancaster.LancasterStemmer`. The code demonstrates reusable functions for preparing Twitter data, relying solely on standard Python and NLTK libraries without any visualization or rendering components.

AI-generated description

Similar vizzes

Loading thumbnail…

Tweet collector for the irony-sarcasm dataset

This example collects tweets for the irony-sarcasm dataset, pulling tweet text for eight labeled subtypes (regular, sarcasm, irony, and their test splits) using Tweepy’s `statuses_lookup` API. It reads tweet IDs from tab-separated CSV files provided by Roman Klinger’s dataset, batches them by 100 with sleep delays to respect rate limits, and stores the full JSON metadata for each tweet. The code then minimizes each record to its `text` field and writes both a JSON master file and a CSV version per subtype, using pandas for ID loading and Python’s `json` and `csv` modules for output.

Aabehmiel
65% match
Loading thumbnail…

Gist d3be7947ba494f0cc785ad4ff966f6d5

This example demonstrates a workflow for collecting and structuring Twitter data through the streaming API. It uses the `tweepy` library to authenticate and open a live stream, filtering tweets by specified terms or hashtags, and saving each raw tweet as a JSON line to a file. The code then reads that file back, parses each tweet, and extracts selected fields into a nested list, including the text, user handle, favorite and retweet counts, timestamp, follower count, and language. Finally, it organizes the data into a structured pandas DataFrame with corresponding column names, preparing it for further analysis or visualization.

GGeorgeMcIntire
63% match
Loading thumbnail…

Gist 8044140b08e7e7887e0c7531911d1fa4

This example streams live tweets matching hashtags like #DoctorWho and #DrWho, saving each tweet’s metadata to a CSV file using the Twitter API and the `twitter` Python package. It demonstrates real-time data collection, extracting fields such as user location, follower count, and coordinates. The code handles truncated tweets by fetching extended text, and it cleans nested dictionary values with helper functions. No visualization is rendered; the focus is purely on acquiring and structuring streaming social media data for later analysis.

AAjar06
62% match
Loading thumbnail…

Gist 6c1ae0e4fff5fb8d4704

This example shows a text-parsing utility that cleans and tokenizes a raw string into lowercase words. It first replaces common punctuation sequences like dashes and double quotes with spaces, then uses a regular expression with Unicode flags to extract word tokens, allowing characters such as `@`, `#`, `&`, and hyphens. The code relies on Python’s standard `re` module and basic string methods, with no external visualization libraries or rendering pipeline. The output is a list of normalized words, illustrating a preprocessing step rather than a visual chart or graph.

Aandyreagan
60% match
Loading thumbnail…

URL list from Tuesday, Sep. 15 2015 17:53 PM

This web scraper gathers a list of URLs from a Twitter timeline on a specific date, capturing tweets and their links. The page displays the raw scraped data as a plain text list, with each entry pairing the tweet author and content with the corresponding URL. The code uses Python’s `requests` and `BeautifulSoup` to parse the HTML, extracting the tweet text and hyperlinks. The output is a simple, unstyled text file, revealing the collected social media links without any further visualization or filtering.

SStephen
59% match
Loading thumbnail…

Buzzfeed article scraper for NLP

This example scrapes a Buzzfeed article and converts its text into clean, tokenized sentences for natural language processing. It shows the pipeline from raw HTML to normalized, sentence-segmented data. The scraper uses BeautifulSoup and requests to extract paragraph elements with the class `subbuzz-text`. Text cleaning relies on the `unicodedata` module and regex via `re`, while sentence tokenization uses NLTK’s Punkt tokenizer. The final output is a single string of joined sentences, ready for further NLP analysis.

Aabehmiel
58% match
Loading thumbnail…

ejemplo de un tuit (en formato JSON)

This example visualizes a single tweet as raw JSON data, displaying the full structure of a Twitter post. The code renders the tweet’s content, including metadata such as user information, timestamps, engagement counts, and embedded media URLs. It uses the JSON object directly from the provided `tweet.js` file, presenting the data in a readable format. The visualization relies on standard JavaScript object inspection rather than a specific charting library, making the raw data itself the focal point. No additional libraries or rendering techniques are applied beyond the direct display of the JSON structure.

Aaaizemberg
57% match
Loading thumbnail…

A script to download all of a user's tweets into a csv

This script downloads a user’s entire tweet history into a CSV file, preserving each tweet’s ID, creation timestamp, and text. It demonstrates the Twitter API’s pagination limits by repeatedly fetching batches of 200 tweets via Tweepy’s `user_timeline` method, using the `max_id` parameter to retrieve older tweets until none remain. The final output is written with Python’s built-in `csv` module, encoding text as UTF-8. The example relies on Tweepy for API authentication and data retrieval, and shows a practical pattern for handling rate-limited, cursor-based API responses.

BBenHeubl
57% match