Gist d5f9fd4d851d95e8815ba77e1364eb5d

100%

Gist d5f9fd4d851d95e8815ba77e1364eb5d

✓ Published0🌍 Public

Last edited Jan 16, 2022

Created on Jan 16, 2022

This example demonstrates how to update a dataset of speeches by appending new entries and committing the changes to a data repository. It shows the process of merging delta records with existing speech data using pandas, then saving the combined result to a CSV file. The code uses `pd.concat` to combine the DataFrames, `to_csv` to persist the updated file, and the `api.dataset_create_version` method from the data platform’s API to create a new version with a timestamped note. This workflow highlights automated data maintenance and version control for evolving datasets.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 541f344f0b112c3ace68d676a41682ee

This example fetches a dataset of Indian Prime Minister Narendra Modi’s speeches from Kaggle using the `kaggle_api_extended.KaggleApi` class, specifically calling `dataset_download_files` to download and unzip the “modi-speeches” dataset. The code then reads the downloaded CSV file into a pandas DataFrame via `pd.read_csv` and extracts the first speech title from the `title` column. The visualization implicitly relies on the Kaggle API and pandas to retrieve and display the dataset’s content, but no charting or rendering library is shown beyond these data-access steps.

Aadiamaan92

63% match

Loading thumbnail…

Gist 575a4d1a506d4f39d35f

This example presents a collection of Python code snippets for data manipulation with the pandas library, focusing on common DataFrame operations. It demonstrates techniques for filtering rows based on column values, handling missing data, renaming columns, and performing data type conversions. The snippets also cover more advanced operations like pivot tables, merging delimited values, and using `pd.melt` for reshaping data. The code includes functions like `df.column_name.unique()`, `pd.pivot_table`, and `df.assign` to illustrate practical workflows for cleaning and transforming tabular data.

SSav10

60% match

Loading thumbnail…

Gist 80444441f83babe9e45d8a3166c837ce

This example demonstrates a small automated data-update workflow that refreshes a dataset by running a Python script. It shows how environment variables supply credentials and user identification, which are then passed to an asynchronous update function. The code uses Python’s `asyncio` library to manage the asynchronous task, while `os.environ` retrieves the required secrets from the system environment. The visualization itself is not rendered here; instead, the example focuses on the programmatic data-fetching and update logic, with the actual charting or rendering handled separately by an unspecified front-end tool.

CConorAspell

59% match

Loading thumbnail…

Fetch Delta using Direct API Calls

This example shows how to incrementally fetch newly published speech titles from the Narendra Modi website using direct API calls. It paginates through the site’s `loadspeeche` endpoint, parsing the returned HTML with `lxml`’s `etree` and XPath queries. The script stops fetching pages once it encounters a title matching the previously saved latest speech, avoiding redundant downloads. It then calls `get_article_data(tree)` on new entries, demonstrating a simple delta-extraction pattern for tracking content updates over time.

Aadiamaan92

56% match

Loading thumbnail…

Gist 7014a4039307a88d7d10e9e4adce0549

This example demonstrates an automated GitHub Actions workflow that scrapes speeches by Indian Prime Minister Narendra Modi on a daily schedule. The workflow, defined in a YAML file, runs a Python script named `scrapper.py` at 8 AM UTC using a cron trigger. It sets up Python 3.8, installs dependencies via Poetry, and caches the virtual environment to speed up subsequent runs. The scraper uploads data to Kaggle using credentials stored as GitHub secrets, with the workflow supporting both scheduled and manual execution through `workflow_dispatch`.

Aadiamaan92

55% match

Loading thumbnail…

Gist 7083cd8264b6aa48ce4bbfaf44546f07

This example builds a profile-scraping pipeline for social media data, turning raw CSV input into a structured DataFrame with fields for user names, follower counts, post dates, likes, comments, captions, and industry tags. The code reads `raw_data.csv` via `pandas.read_csv` and then initializes an empty DataFrame with predefined column names, preparing it for subsequent population. The visualization focuses on the data-preparation stage rather than graphical output, demonstrating how pandas handles tabular ingestion and schema definition. No rendering library like d3 or Three.js appears here; instead, the emphasis is on clean data structuring for later analysis or charting.

AArizonaTay

55% match

Loading thumbnail…

Gist 740405b6405a6644fd16d3a50588de51

This example shows a step-by-step transformation of the classic wine dataset, progressively filtering and selecting columns to isolate alcohol content, color intensity, and hue before randomly sampling five rows. The code chains pandas operations—including `rename`, `assign`, `query`, and `sort_values`—through a custom `csnap` helper that snapshots the intermediate state after each step. It uses NumPy’s `np.where` to create a binary `color_filter` column based on hue and color intensity thresholds, then resets the index and samples five rows with `sample(5)`. The visualization highlights how pipelined data wrangling can be inspected at each stage.

Aadiamaan92

54% match

Loading thumbnail…

Gist 6ac95622ce9dd73d237ca6b17b2d5543

This visualization demonstrates a method-chaining utility for pandas DataFrames, allowing users to inspect intermediate data states during complex transformations. The `csnap` function takes a DataFrame, an optional transformation function, and a message, displaying the result of applying that function to the data while returning the original DataFrame for continued chaining. It uses the `display` API from IPython to render the snapshot inline, with a default function that shows the DataFrame’s shape. The example highlights a practical debugging pattern for exploratory data analysis workflows.

Aadiamaan92

54% match