Fetch Delta using Direct API Calls

100%

Fetch Delta using Direct API Calls

✓ Published0🌍 Public

Last edited Jan 16, 2022

Created on Jan 16, 2022

This example shows how to incrementally fetch newly published speech titles from the Narendra Modi website using direct API calls. It paginates through the site’s `loadspeeche` endpoint, parsing the returned HTML with `lxml`’s `etree` and XPath queries. The script stops fetching pages once it encounters a title matching the previously saved latest speech, avoiding redundant downloads. It then calls `get_article_data(tree)` on new entries, demonstrating a simple delta-extraction pattern for tracking content updates over time.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 541f344f0b112c3ace68d676a41682ee

This example fetches a dataset of Indian Prime Minister Narendra Modi’s speeches from Kaggle using the `kaggle_api_extended.KaggleApi` class, specifically calling `dataset_download_files` to download and unzip the “modi-speeches” dataset. The code then reads the downloaded CSV file into a pandas DataFrame via `pd.read_csv` and extracts the first speech title from the `title` column. The visualization implicitly relies on the Kaggle API and pandas to retrieve and display the dataset’s content, but no charting or rendering library is shown beyond these data-access steps.

Aadiamaan92

64% match

Loading thumbnail…

Gist d5f9fd4d851d95e8815ba77e1364eb5d

This example demonstrates how to update a dataset of speeches by appending new entries and committing the changes to a data repository. It shows the process of merging delta records with existing speech data using pandas, then saving the combined result to a CSV file. The code uses `pd.concat` to combine the DataFrames, `to_csv` to persist the updated file, and the `api.dataset_create_version` method from the data platform’s API to create a new version with a timestamped note. This workflow highlights automated data maintenance and version control for evolving datasets.

Aadiamaan92

56% match

Loading thumbnail…

Gist 7ef9c17d99cb5a83e7312992e4ec683d

This visualization maps the geographic and thematic reach of Narendra Modi’s speeches, sourced from a Kaggle dataset of text speeches starting in 2018. It displays an interactive choropleth of India, with state boundaries drawn using `d3.geoPath` and a `d3.forceSimulation` layout that positions speech-related keywords as nodes. Hovering over a state reveals its name, while colored circles represent speech themes, with sizes scaled to frequency. The code uses D3.js for SVG rendering and transitions, and it loads the dataset from a JSON file, updating the visualization dynamically.

Aadiamaan92

55% match

Loading thumbnail…

Gist 7014a4039307a88d7d10e9e4adce0549

This example demonstrates an automated GitHub Actions workflow that scrapes speeches by Indian Prime Minister Narendra Modi on a daily schedule. The workflow, defined in a YAML file, runs a Python script named `scrapper.py` at 8 AM UTC using a cron trigger. It sets up Python 3.8, installs dependencies via Poetry, and caches the virtual environment to speed up subsequent runs. The scraper uploads data to Kaggle using credentials stored as GitHub secrets, with the workflow supporting both scheduled and manual execution through `workflow_dispatch`.

Aadiamaan92

55% match

Loading thumbnail…

Gist d2f2bc6be1716820160ed5183ea07eaa

This example shows a browser-based text analytics pipeline that scrapes and processes web content before visualizing topic models. The code first uses Selenium with Chrome WebDriver to scrape data, then applies NLP libraries including NLTK, spaCy, TextBlob, and gensim’s LdaModel for topic modeling and coherence scoring. The final visualization employs Plotly’s `plotly.express` and `plotly.graph_objs` to render interactive charts, with `pyo.init_notebook_mode()` enabling notebook display. The rendering approach is evident from the use of `go` traces and `px` figures, though the specific chart type is not shown in the code snippet. The data source is dynamic web pages, as indicated by the Selenium scraping logic.

AArizonaTay

48% match

Loading thumbnail…

How to use The Guardian's API to download article data for content analysis (in Python 3.x)

This example demonstrates how to perform content analysis on Guardian news articles by programmatically downloading article data through The Guardian’s open API. It shows the entire workflow, from constructing API queries with parameters like `from-date`, `to-date`, and `show-fields`, to parsing the returned JSON metadata and article bodies. The Python script uses the `requests` library to fetch paginated results into day-sized chunks, saving the raw JSON files locally for further analysis. The example also highlights the API’s response structure, including fields such as `wordcount`, `byline`, and full article HTML.

GGeorgeMcIntire

48% match

Loading thumbnail…

ba 401 Data loading Data

This example demonstrates a minimal data-loading workflow using the browser’s native Fetch API to retrieve a remote dataset from a GitHub Gist. The page consists of a single HTML file with inline JavaScript that calls `fetch(url)` and logs the resulting Promise to the console. No rendering, styling, or data processing occurs; the example solely illustrates the first step of asynchronously fetching external resources in a plain HTML/JavaScript context, without any additional libraries or frameworks.

Ddeven1998

46% match

Loading thumbnail…

fetch

This example demonstrates how to load data from a remote CSV URL using the browser’s built-in `fetch` API. It displays the static text “Hola!” in large type, while logging the raw `fetch` response object to the console—no parsing or rendering of the data occurs. The page is a bare HTML file with inline CSS and JavaScript, providing a minimal starting point for fetching external datasets, such as the Argentine government’s COVID-19 vaccination personnel statistics. The example uses no external libraries, relying solely on native web APIs.

Aaleoncavallo

46% match