Gist d3be7947ba494f0cc785ad4ff966f6d5

100%

Gist d3be7947ba494f0cc785ad4ff966f6d5

✓ Published0🌍 Public

Last edited Jun 6, 2017

Created on Jun 6, 2017

This example demonstrates a workflow for collecting and structuring Twitter data through the streaming API. It uses the `tweepy` library to authenticate and open a live stream, filtering tweets by specified terms or hashtags, and saving each raw tweet as a JSON line to a file. The code then reads that file back, parses each tweet, and extracts selected fields into a nested list, including the text, user handle, favorite and retweet counts, timestamp, follower count, and language. Finally, it organizes the data into a structured pandas DataFrame with corresponding column names, preparing it for further analysis or visualization.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 8044140b08e7e7887e0c7531911d1fa4

This example streams live tweets matching hashtags like #DoctorWho and #DrWho, saving each tweet’s metadata to a CSV file using the Twitter API and the `twitter` Python package. It demonstrates real-time data collection, extracting fields such as user location, follower count, and coordinates. The code handles truncated tweets by fetching extended text, and it cleans nested dictionary values with helper functions. No visualization is rendered; the focus is purely on acquiring and structuring streaming social media data for later analysis.

AAjar06

73% match

Loading thumbnail…

Gist 7083cd8264b6aa48ce4bbfaf44546f07

This example builds a profile-scraping pipeline for social media data, turning raw CSV input into a structured DataFrame with fields for user names, follower counts, post dates, likes, comments, captions, and industry tags. The code reads `raw_data.csv` via `pandas.read_csv` and then initializes an empty DataFrame with predefined column names, preparing it for subsequent population. The visualization focuses on the data-preparation stage rather than graphical output, demonstrating how pandas handles tabular ingestion and schema definition. No rendering library like d3 or Three.js appears here; instead, the emphasis is on clean data structuring for later analysis or charting.

AArizonaTay

69% match

Loading thumbnail…

Gist b87ac9ed9766b78a72565a895d3a660f

This example shows a scatterplot-style visualization of audio features for tracks from a Spotify playlist, with each point representing a song’s metadata retrieved via the Spotify Web API. The code pulls playlist tracks using `spotipy`’s `user_playlist` method, extracts track IDs, and calls `audio_features` to gather numeric acoustics attributes. Those features are then loaded into a pandas `DataFrame`, which serves as the dataset for the plotted chart. The visualization relies on Spotify’s API response structure and pandas for tabular organization, but does not include any specific plotting library call in the provided code.

GGeorgeMcIntire

64% match

Loading thumbnail…

Gist d003ece0b6ca171be8e095e0bc90af7e

This notebook example demonstrates how to programmatically scrape, parse, and structure a corpus of New York Times articles into a pandas DataFrame. The code loads a pre-saved pickle file of 50 article links, iterates through each URL using the `newspaper` library's `Article` class, and extracts the article’s text, title, publication date, keywords, and summary. It then stores the results in a list of dictionaries and converts that list into a structured DataFrame. The script includes error handling for unparseable articles, inserting `NaN` values, and prints the total execution time, which took roughly 38.5 seconds for this run. The displayed table shows the resulting columns: date, keywords, link, summary, text, and title.

GGeorgeMcIntire

64% match

Loading thumbnail…

Gist dfbc4b41961e5b098fffce6af3828f57

This visualization scrapes Instagram profile data to compare follower counts, post engagement, and captions across influencers in different industries. Using Selenium to automate Chrome and BeautifulSoup to parse page JSON, the script extracts post dates, like counts, comments, and partnership flags from each profile’s embedded `_sharedData` blob. It then compiles the results into a pandas DataFrame and exports them to CSV. The example highlights the workflow of collecting social-media metrics at scale, with the final chart likely plotting follower growth or engagement trends per industry.

AArizonaTay

63% match

Loading thumbnail…

Gist fa05e9e0684e37e537c08085e5900435

This example shows a data-cleaning pipeline for an Instagram influencer dataset, highlighting the number of unique users and missing values across key columns. The code uses pandas to inspect and transform the data, filling missing industry labels with "General," dropping rows without captions, removing duplicate entries, and converting the post date column to datetime format for time-series indexing. The visualization relies on pandas' built-in data manipulation functions, such as `fillna`, `drop_duplicates`, and `set_index`, rather than a plotting library. The final output would be a cleaned dataframe ready for trend analysis, though no chart is generated in the provided script.

AArizonaTay

62% match

Loading thumbnail…

Gist a99d287be62c6d73536521c0e5e74e58

This example scrapes Instagram profile and post data to track follower counts, likes, comments, captions, and post dates for selected users, associating each entry with an industry topic. The script uses Selenium to automate login and search, BeautifulSoup to parse the page source, and the Instagram GraphQL data embedded in `window._sharedData` to extract metrics. It writes the collected records to a CSV via pandas, pausing with random sleeps to avoid detection. The visualization highlights how social-media engagement varies across profiles and industries over time.

AArizonaTay

61% match

Loading thumbnail…

A script to download all of a user's tweets into a csv

This script downloads a user’s entire tweet history into a CSV file, preserving each tweet’s ID, creation timestamp, and text. It demonstrates the Twitter API’s pagination limits by repeatedly fetching batches of 200 tweets via Tweepy’s `user_timeline` method, using the `max_id` parameter to retrieve older tweets until none remain. The final output is written with Python’s built-in `csv` module, encoding text as UTF-8. The example relies on Tweepy for API authentication and data retrieval, and shows a practical pattern for handling rate-limited, cursor-based API responses.

BBenHeubl

61% match