Skip to main content
100%

Tweet collector for the irony-sarcasm dataset

✓ Published0🌍 Public
Aabehmiel
Last edited Jun 23, 2017
Created on Jun 23, 2017

This example collects tweets for the irony-sarcasm dataset, pulling tweet text for eight labeled subtypes (regular, sarcasm, irony, and their test splits) using Tweepy’s `statuses_lookup` API. It reads tweet IDs from tab-separated CSV files provided by Roman Klinger’s dataset, batches them by 100 with sleep delays to respect rate limits, and stores the full JSON metadata for each tweet. The code then minimizes each record to its `text` field and writes both a JSON master file and a CSV version per subtype, using pandas for ID loading and Python’s `json` and `csv` modules for output.

AI-generated description

Similar vizzes

Loading thumbnail…

Python Utilities for Tweets

This utility module processes raw tweet JSON objects for text analysis. It extracts timestamps via `datetime.strptime`, hashtags, user mentions, and cleaned text by stripping URLs, mentions, and media using entity indices. It further sanitizes punctuation with the `string` module, removes English stopwords from `nltk.corpus`, and applies Lancaster stemming via `nltk.stem.lancaster.LancasterStemmer`. The code demonstrates reusable functions for preparing Twitter data, relying solely on standard Python and NLTK libraries without any visualization or rendering components.

Aabehmiel
65% match
Loading thumbnail…

Gist d3be7947ba494f0cc785ad4ff966f6d5

This example demonstrates a workflow for collecting and structuring Twitter data through the streaming API. It uses the `tweepy` library to authenticate and open a live stream, filtering tweets by specified terms or hashtags, and saving each raw tweet as a JSON line to a file. The code then reads that file back, parses each tweet, and extracts selected fields into a nested list, including the text, user handle, favorite and retweet counts, timestamp, follower count, and language. Finally, it organizes the data into a structured pandas DataFrame with corresponding column names, preparing it for further analysis or visualization.

GGeorgeMcIntire
59% match
Loading thumbnail…

A script to download all of a user's tweets into a csv

This script downloads a user’s entire tweet history into a CSV file, preserving each tweet’s ID, creation timestamp, and text. It demonstrates the Twitter API’s pagination limits by repeatedly fetching batches of 200 tweets via Tweepy’s `user_timeline` method, using the `max_id` parameter to retrieve older tweets until none remain. The final output is written with Python’s built-in `csv` module, encoding text as UTF-8. The example relies on Tweepy for API authentication and data retrieval, and shows a practical pattern for handling rate-limited, cursor-based API responses.

BBenHeubl
58% match
Loading thumbnail…

Gist 8044140b08e7e7887e0c7531911d1fa4

This example streams live tweets matching hashtags like #DoctorWho and #DrWho, saving each tweet’s metadata to a CSV file using the Twitter API and the `twitter` Python package. It demonstrates real-time data collection, extracting fields such as user location, follower count, and coordinates. The code handles truncated tweets by fetching extended text, and it cleans nested dictionary values with helper functions. No visualization is rendered; the focus is purely on acquiring and structuring streaming social media data for later analysis.

AAjar06
58% match
Loading thumbnail…

Thinkup Tweet importetr

This example shows a PHP script that imports a large archive of tweets (47,455) into the ThinkUp database. The script reads a text file containing JSON-encoded tweets, parses them line by line, and inserts each tweet into the database via the PostDAO. It extracts fields like tweet ID, text, timestamp, and user ID, hardcoding author information and converting the tweet's created_at string into MySQL-compatible datetime format. The importer runs from the command line and is designed to batch-process the archive, printing a key for each inserted post. The visualization highlights how legacy tweet data can be migrated into ThinkUp's storage layer for later analysis.

Aalper
55% match
Loading thumbnail…

Gist a99d287be62c6d73536521c0e5e74e58

This example scrapes Instagram profile and post data to track follower counts, likes, comments, captions, and post dates for selected users, associating each entry with an industry topic. The script uses Selenium to automate login and search, BeautifulSoup to parse the page source, and the Instagram GraphQL data embedded in `window._sharedData` to extract metrics. It writes the collected records to a CSV via pandas, pausing with random sleeps to avoid detection. The visualization highlights how social-media engagement varies across profiles and industries over time.

AArizonaTay
54% match
Loading thumbnail…

using-pretrained-abstract-to-tweet-model.ipynb

This example shows how a pre-trained transformer model converts academic abstracts into concise, tweet-sized summaries, demonstrating a text-to-text generation workflow. It loads a Hugging Face `AutoModelForSeq2SeqLM` and `AutoTokenizer` from the `andersoncliffb/abstracts_to_tweet_model` repository, then creates a `text2text-generation` pipeline to produce the output. The notebook’s code uses the Transformers library’s pipeline API, with the model and tokenizer specified directly. It does not include a data visualization, instead focusing on the model’s generative output from input text.

CCliffordAnderson
53% match
Loading thumbnail…

JSON concatenate, dump to CSV

This example merges nine years of JSON-format Trump tweets into a single CSV file, consolidating chronological tweet data into one tabular output. It uses Python’s `json` library to load each yearly file and the `csv` module’s `csv.writer` to write rows. The script first writes a header row from the keys of the first tweet, then appends each tweet’s values for every file in sequence. The resulting `trumptweets.csv` preserves all original fields without transformation, enabling further analysis or visualization of the combined dataset.

Aabehmiel
53% match