Script for updating a database of the Citibike JSON feed

100%

Script for updating a database of the Citibike JSON feed

✓ Published0🌍 Public

Last edited Apr 8, 2015

Created on Mar 28, 2015

This example demonstrates a script that keeps a Citibike station and bike database current by parsing the live Citibike JSON feed. It shows how to update PostgreSQL records through Django models, capturing the feed’s execution time as separate date and time fields. The code uses the `requests` library to fetch the JSON, `datetime` to reformat timestamps, and Django’s `get_or_create` to add or retrieve station and bike entries. It also handles station ID changes by matching on station names, ensuring consistent data linkage across updates.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist e9eac02df35654daa5d0

This example demonstrates a minimal data pipeline that reads JSON data and inserts it into a local SQLite database. The script, `jsonToSqlite.py`, loads a JSON file containing a list of objects, extracts the first object’s `"foo"` and `"bar"` fields, and writes those two values as a single row into a table named `table_name`. It uses the standard `json` module for parsing and `sqlite3` for database operations, creating a table and inserting the data with a parameterized `execute` call. The visualization shows only the code structure, not a graphical output, highlighting the simple ETL pattern of extracting, transforming, and loading data.

CCBasis

56% match

Loading thumbnail…

Gist 80444441f83babe9e45d8a3166c837ce

This example demonstrates a small automated data-update workflow that refreshes a dataset by running a Python script. It shows how environment variables supply credentials and user identification, which are then passed to an asynchronous update function. The code uses Python’s `asyncio` library to manage the asynchronous task, while `os.environ` retrieves the required secrets from the system environment. The visualization itself is not rendered here; instead, the example focuses on the programmatic data-fetching and update logic, with the actual charting or rendering handled separately by an unspecified front-end tool.

CConorAspell

53% match

Loading thumbnail…

fresh block

This example demonstrates a data-processing sketch for a bike-sharing dataset, computing a rough estimate of future bike availability at each station. The code loads station information from a `stations.json` file using `d3.json`, then iterates through the records to calculate a random correction factor based on the number of total and available bike stands. It outputs the intermediate values and the corrected estimate to the console, but does not render any visual marks on the SVG canvas. The SVG is created with `d3.select`, yet it only contains a static text prompt, as no actual visualization is produced.

CChmits

53% match

Loading thumbnail…

Autoupdating graphite timeseries in d3

This example shows live-updating time-series graphs that automatically refresh their data at a regular interval. It renders each graph as an SVG with D3 scales and axes, drawing lines from Graphite’s JSON endpoint. The code uses `d3.json` for data fetching, `d3.svg.axis` and `d3.svg.line` for rendering, and a clip path to constrain the plotted area. A real-time mode triggers repeated requests using a dynamic URL that updates the time window based on the first payload’s interval.

CCaged

52% match

Loading thumbnail…

fresh block

This visualization displays bicycle and docking station availability in Lyon, France. The data includes station locations from a JSON file and time-stamped availability counts from CSV files. The code uses D3.js (version 4) to render an SVG visualization. It loads the data files and creates an interactive display showing the dynamic changes in bike and slot availability over time.

MMarineRu

52% match

Loading thumbnail…

Gist dca53ad15bb437468b52eb951a09f356

This visualization shows the data pipeline for managing Fantasy Premier League player data, fetching gameweek deadlines and team rosters from the official FPL API. It retrieves data using Python's `requests` library, converts JSON responses into DataFrames with `pandas`, and then persists the processed data to Amazon S3 through the `boto3` client. The code also reads configuration parameters from AWS SSM and sends notifications via SNS topics, demonstrating an automated workflow for updating and storing fantasy football statistics.

CConorAspell

51% match

Loading thumbnail…

Gist a96c926e932dc1beb8054857775aae2d

This example shows a Python script that automatically manages a Fantasy Premier League team by calculating optimal transfers and lineups. It retrieves player data, evaluates potential transfers, and submits changes to the FPL API. The code uses the `requests` library to handle API calls and `pandas` for data manipulation, including pivot tables and filtering based on team constraints. It also demonstrates how to update team sheets and set captaincy through HTTP POST requests. The visualization focuses on the logic and data flow rather than graphical output.

CConorAspell

51% match

Loading thumbnail…

Created on Cognitive Class Labs

This example demonstrates how to load three Chicago public datasets—socioeconomic indicators, public school performance, and crime reports—from CSV files into a Db2 database using the `ibm_db` API, then query them with SQL. It uses the `pandas` library to read the CSV data and the `ibm_db` connection functions to create and populate tables. The notebook guides through understanding the datasets, downloading them, and storing them in database tables, with the SQL queries highlighting relationships between community areas, schools, and crime incidents.

DDAVINDER

50% match