quick example of using the batch api over python

100%

quick example of using the batch api over python

✓ Published0🌍 Public

Last edited May 3, 2016

Created on May 3, 2016

This example shows how to submit asynchronous SQL jobs to CartoDB’s Batch API from Python, using a query that updates geometries by replacing them with centroids. It demonstrates the workflow of starting a job with a POST request, then polling its status via a GET request, and includes a helper to register new tables with CartoDB’s `CDB_Cartodbfytable` function. The code uses the `requests` library to handle HTTP calls, along with `urllib.quote_plus` for query encoding, and relies on the `cartodb.com` API endpoints for job creation and status retrieval.

AI-generated description

Similar vizzes

Loading thumbnail…

Batch API uploads to CARTO

This example shows how to upload large datasets to CARTO by splitting them into smaller chunks and using the SQL Batch API to merge them. It demonstrates three chained batch queries: creating a combined table with `CREATE TABLE ... UNION ALL`, dropping the `cartodb_id` column with `ALTER TABLE`, and finalizing with `cdb_cartodbfytable`. The code references CARTO’s SQL API and batch query chaining, alongside a Python script from `carto-etl` and GDAL/ogr2ogr alternatives for record-by-record inserts.

Aandrewbt

78% match

Loading thumbnail…

command-line python interface for manipulating data on CartoDB

This example demonstrates a command-line Python tool for managing and manipulating data stored on CartoDB, allowing users to upload files, overwrite or append rows, add columns, drop tables, and export datasets. It first shows the workflow of interacting with CartoDB’s SQL and Import APIs, then explains the underlying implementation. The script uses the `requests` library for HTTP calls, `urllib2` for polling import status, and `argparse` for command-line options, translating between CartoDB’s data types and PostgreSQL types during column alterations and insertions.

Aandrewxhill

71% match

Loading thumbnail…

CartoDB CSV Bulkloader for Python

This example demonstrates a Python script that bulk-loads CSV data into a CartoDB table via the CartoDB SQL API, converting latitude and longitude columns into geometry using the CDB_LatLng function. It reads rows from a local CSV file, batches inserts of up to 25,000 rows per request, and skips records already present by checking a unique identifier against the table’s latest entry. The code uses Python’s csv, urllib2, and json modules to parse data and send HTTP POST requests, with error handling that writes failed SQL statements to a log file.

Aandrewxhill

68% match

Loading thumbnail…

Merging an entire directory of data and loading it to CartoDB

This example shows a shell script that uploads every GPX file in a directory to an existing CartoDB table. It demonstrates how to merge an entire folder of GPS track data into a single cloud-hosted dataset. The script relies on a Python utility called `cartodb-utils.py` to handle the upload process. For each `.gpx` file, the script calls the `append` command, passing the file path, table name, API key, and CartoDB account name as arguments. This approach automates the bulk ingestion of geospatial data without manual intervention.

Aandrewxhill

59% match

Loading thumbnail…

Import CartoDB file with cURL

This example demonstrates how to upload a file into a CartoDB account using the command-line tool cURL, showing the complete import workflow from submission to completion. The shell script sends the file to CartoDB’s `/api/v1/imports/` endpoint, extracts the job ID from the JSON response, and then polls the import status until it reports “complete.” It uses cURL for HTTP requests and `sed` for parsing the JSON, with platform-specific regex variations for macOS (Darwin) and Linux. The script takes three arguments: the CartoDB username, API key, and the filename to import.

Aandrewxhill

59% match

Loading thumbnail…

cartodb-python SDK Beta 1.0

The cartodb-python SDK Beta 1.0 example demonstrates the programmatic management of CARTO accounts and datasets through Python. Using the APIKeyAuthClient for authentication, the script creates a new user in a CARTO organization and then creates a dataset by importing a publicly available tornado dataset from an external ZIP file URL. It showcases the SDK’s object model, including UserManager and DatasetManager, and prints the resulting metadata for the new user and dataset—details like quota, table counts, geometry types, and row counts. Finally, it shows deleting the user and demonstrates the full CRUD cycle of the SDK’s high-level API for managing both users and datasets programmatically.

Aandrewbt

59% match

Loading thumbnail…

Fun SQL statements for CartoDB

This example shows how to explore and analyze tornado and county data in CartoDB using SQL statements. It demonstrates basic queries for filtering, sorting, and aggregating records, followed by spatial functions that inspect geometry types, compute areas, distances, and intersections. The code uses PostGIS functions such as `ST_Area`, `ST_Distance`, `ST_Intersects`, and `ST_DWithin`, alongside CartoDB-specific helpers like `CDB_LatLng`. It also includes updates to add tornado counts and density per county, highlighting how SQL can drive interactive map visualizations.

Aandrewxhill

57% match

Loading thumbnail…

Carto

This example demonstrates how to move data from Amazon Redshift into CARTO for visualization. Using SQLAlchemy to connect to a Redshift cluster, the notebook reads an `addresses` table into a pandas DataFrame, then writes it to a CARTO account with the `cartoframes` library. The result is a CARTO dataset (`redshift_example`) that can be explored on an interactive map, enabling geospatial analysis of address-based data such as inhabitant counts.

Aandrewbt

56% match