XQuery for Complex Migration

100%

XQuery for Complex Migration

✓ Published0🌍 Public

Last edited Apr 24, 2019

Created on Apr 16, 2019

This example demonstrates XQuery scripts for cleaning and restructuring VRA Core XML metadata from the DIMLI collection, focusing on consolidating duplicate work records and image records by refid. The code identifies duplicate work entries, merges their associated images while removing redundant id attributes, and normalizes image records. It also runs diagnostic queries to count built works, locations without refids, and image records (targeting 40,420 image records), while filtering location names that reference museums, libraries, or geographic terms. The visualization highlights data-quality issues in cultural-heritage metadata and the programmatic transformations needed to reconcile legacy cataloging inconsistencies.

AI-generated description

Similar vizzes

Loading thumbnail…

XQuery Conversion for MDID

This example demonstrates an XQuery conversion that transforms a collection of XML records into a structured CSV file, preserving field mappings for VRA and VU metadata. The code iterates over all records in a collection, extracts specific child elements and attributes, and constructs new `<record>` elements with standardized tags. It then wraps the records in a `<csv>` root element and uses the `csv:serialize` function with a map specifying `'header': true()` to generate a CSV with column headers. Finally, it writes the output to a file named "vrc.csv". The process relies on XQuery functions like `fn:collection()`, `fn:data()`, and `file:write()`, along with the `csv` module’s serialization API.

CCliffordAnderson

66% match

Loading thumbnail…

VRA website example

This example demonstrates a focused query of a VRA XML metadata record, extracting and displaying only the `locationSet` element from the source document. The code uses the eXist-db HTTP Client module (`http:send-request`) to fetch the remote XML file from GitHub, then applies an XQuery path expression to isolate the location data. The visualization shows how structured art metadata can be programmatically filtered to reveal just geographic or provenance-related information. By leveraging the VRA Core 4 namespace declaration, the script ensures precise element selection, making it a practical tool for targeted data extraction in digital humanities projects.

CCliffordAnderson

62% match

Loading thumbnail…

Tricky Metadata Problem

This visualization demonstrates the geographic distribution of artists' books by filtering MARCXML records according to publication location codes, then organizing them into a simple table. It shows which US states and territories are represented in the source collection, with each row containing the record’s leader, normalized title, and three-letter location code. The XQuery script uses the `fn:doc()` function to fetch remote XML, the `marc` namespace to parse MARC fields, and `csv:serialize()` from the BaseX or eXist-db library to generate a CSV output. Titles are cleaned by stripping trailing slashes and normalizing whitespace via string functions.

CCliffordAnderson

58% match

Loading thumbnail…

XQuery Blue Mountain Project - Part III

This example shows a pipeline for transforming a single TEI XML document from the Blue Mountain Project—a digitized 1920 issue of the French journal *Action*—into clean, article-level TEI files. The first script removes facsimile elements, end-hyphenation markers, and line breaks, replacing `<lb>` elements with spaces. A second script extracts metadata and article content into a fresh TEI template, while a third builds individual TEI files for each article. The code uses XQuery 3.1 with the `fn:doc` API for document access and node operations like `copy`, `modify`, and `replace node`.

CCliffordAnderson

57% match

Loading thumbnail…

XQuery for Partitioning the Blue Mountain Project

This example demonstrates a pipeline for cleaning, partitioning, and exporting articles from the Blue Mountain Project’s TEI transcriptions into language-specific databases. It shows how XQuery 3.1 scripts transform raw periodical XML by removing facsimile and hyphenation artifacts, splitting documents into individual articles, and grouping them by `mainLang` attributes into new databases. The workflow then exports French-language texts as plain .txt files, and R code using the `tm` and `SnowballC` libraries performs text mining, including term-document matrices and stemming.

CCliffordAnderson

57% match

Loading thumbnail…

Sample code for extracting, transforming, and loading IA metadata into BaseX

This example demonstrates an XQuery 3.1 pipeline for harvesting Open Library search results and persisting them as structured XML in a BaseX database. It first fetches JSON via `fetch:text` and parses it with `json:parse`, then converts each record into a canonical `book` element with attributes for key and full-text availability, and child elements for title, authors, dates, publishers, and subjects. Subsequent scripts extend the process to paginate through large result sets, and a final query retrieves full book texts from the Internet Archive, storing them alongside the metadata.

CCliffordAnderson

57% match

Loading thumbnail…

Code snippets for XQuery Working Group (Text Mining at Scale)

This example demonstrates a text-mining workflow for querying the Victorian Women Writers Project corpus stored in a BaseX database, using XQuery and XPath Full Text 1.0. It walks through creating structured records from TEI XML, saving them to a new database, and running full-text searches with scoring, stemming, and thesaurus-based matching. The code exercises functions like `db:replace`, `ft:mark`, and `ft:extract` to highlight matching text, while also showing stop-word filtering and boolean operators for queries.

CCliffordAnderson

56% match

Loading thumbnail…

XQuery Samples for Blue Mountain Project

This example shows a sequence of XQuery scripts that analyze the language metadata of TEI-encoded journals from the Blue Mountain Project, a digital archive of avant-garde publications. The scripts query a collection called `bluemountain-transcriptions` using the `fn:collection` function, counting documents, extracting distinct languages from `tei:langUsage`, and identifying journals with multiple `tei:textLang/@mainLang` attributes. Later scripts partition documents by language, using `db:create` to store French texts in a new database, and verify the results by listing the URIs of the partitioned files. The code relies on the XQuery 3.1 standard and the eXist-db database API for collection access and database creation.

CCliffordAnderson

56% match