Skip to main content
SimilarExport ZIP
100%

Bash / JQ for Cloudant Bulk Upload & Download

✓ Published0🌍 Public
CCliffordAnderson
Last edited Mar 12, 2018
Created on Nov 15, 2014

This example demonstrates a command-line workflow for bulk uploading and downloading JSON documents to and from a Cloudant database, using bash scripts that combine HTTPie and JQ. It shows how to aggregate multiple local JSON files into a single `_bulk_docs` payload via `jq -s`, upload that payload with an HTTP POST request, and retrieve data from a Cloudant view—transforming the response into either a plain document array or a GeoJSON FeatureCollection. The scripts also convert view output back into a file, enabling round-trip data migration and export without a custom application.

AI-generated description

Similar vizzes

Loading thumbnail…

Identify GeoJSON errors in Mapping Berlin

This example identifies errors in GeoJSON data for a map of Berlin by piping a Cloudant view response through `jq` to extract point features and run `geojsonhint`. The command constructs a FeatureCollection from the database rows, then validates the resulting JSON structure and geometry. The visualization shows which features fail validation, using the command-line tool to highlight malformed coordinates or properties. The process relies on a gist-hosted script that queries the Cloudant database and pipes the output through `jq` before passing it to `geojsonhint` for error detection.

CCliffordAnderson
57% match
Loading thumbnail…

CVS to JSONL example

This example shows a practical data-transformation workflow that converts a CSV file into JSONL format, producing one JSON object per line. It fetches a remote CSV file from a GitHub gist using the `fetch:text` function, parses it into records with `csv:parse`, and iterates through each record to build key-value pairs. The code distinguishes between integer and string values using `castable as xs:integer`, then constructs JSON maps with `xml-to-json` using XQuery 3.1. The output is unindented JSONL, suitable for streaming or line-oriented processing.

CCliffordAnderson
56% match
Loading thumbnail…

Cloudant Post

This example visualizes the geographic distribution of user-uploaded content from a Cloudant database, rendered as an interactive map. It first retrieves raw GeoJSON data from the Cloudant API, then uses `d3.geoPath` to project and draw country outlines, while `d3.forceSimulation` positions individual point markers based on their latitude and longitude. The map’s tile layer is generated with Canvas `getImageData` to apply a custom color scale to population density, and user interactions trigger `transition.attrTween` for smooth panning and zooming. The code relies on Three.js for the underlying WebGL rendering of the base terrain.

CCliffordAnderson
55% match
Loading thumbnail…

{$doc/RecordTitle/text()}

This example extracts bibliographic records from a collection of quarterly publications and serializes them into CSV format. It queries a local XML document store containing full-text articles, capturing fields such as title, publication, volume, issue, date, pagination, and a document link. The XQuery 3.1 script uses the `csv:serialize` function to convert the generated `element csv` tree into a delimited text file, demonstrating a straightforward data-wrangling pipeline for export and downstream analysis.

CCliffordAnderson
53% match
Loading thumbnail…

A brief introduction to JSONiq for NashFP

This example demonstrates the JSONiq query language by showing code snippets for tasks like object navigation, FLWOR expressions, JSON-to-XML conversion, and the Luhn algorithm. The sample queries use JSONiq’s map operator, function libraries like `fn:upper-case` and `fn:fold-left`, and the `keys()` and `members()` functions. Each snippet includes a link to run it in the Zorba engine’s online sandbox, referencing the NashFP presentation context.

CCliffordAnderson
53% match
Loading thumbnail…

Power Query export data to SQL Server

This example demonstrates exporting data from a SQL Server database by running a large SQL script stored in a local file. It shows how Power Query can leverage native database connectivity to execute a bulk insert operation with high parallelism. The code uses `File.Contents` to read the SQL file, then passes its binary content to `Value.NativeQuery`, which targets a SQL Server instance via `Sql.Database`. Key parameters include `EnableBulkInsert` for efficient data loading, `MaxDegreeOfParallelism` to speed up execution, and explicit timeouts for command and connection. The rendered output is the result set from the query, effectively transferring data into Power BI’s data model.

HHugoberry
51% match
Loading thumbnail…

Append some new json to some existing json

This example shows how to merge two JSON files into a single, updated JSON object. The command-line tool `jq` uses its `-s` flag to read both input files into a single array, then applies the `.[0] + .[1]` filter to concatenate the two objects, writing the result to `updated.json`. The process relies entirely on `jq`’s stream editor and its built-in addition operator for objects. The source data originates from the two input files, and the output is a new file.

GGeoWill
50% match
Loading thumbnail…

Gist e9eac02df35654daa5d0

This example demonstrates a minimal data pipeline that reads JSON data and inserts it into a local SQLite database. The script, `jsonToSqlite.py`, loads a JSON file containing a list of objects, extracts the first object’s `"foo"` and `"bar"` fields, and writes those two values as a single row into a table named `table_name`. It uses the standard `json` module for parsing and `sqlite3` for database operations, creating a table and inserting the data with a parameterized `execute` call. The visualization shows only the code structure, not a graphical output, highlighting the simple ETL pattern of extracting, transforming, and loading data.

CCBasis
50% match