Use `sqlfluff` to build a lineage tree in D2

100%

Use `sqlfluff` to build a lineage tree in D2

✓ Published0🌍 Public

Last edited Oct 7, 2024

Created on Oct 7, 2024

This visualization parses T-SQL view definitions from `vwMarket.sql` and renders a lineage tree showing table and column dependencies, using `sqlfluff` to generate a parse tree and `D2` to draw the diagram. The code extracts aliases from `from_clause` nodes to list source tables as SQL-table shapes, and pulls column-to-column mappings from `expression` segments where comparisons use the `=` operator, outputting edges like `alias.column -> other.column`. It relies on `sqlfluff.parse` for T-SQL grammar analysis and recursive JSON traversal via `get_json_segment` to locate `from_expression_element` and `column_reference` structures, printing D2 syntax directly to standard output. The example demonstrates how to bridge SQL parsing with declarative diagramming for dependency visualization.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist be32a2ac117700d3dbde6b0824367ba5

This example demonstrates how to create a simple database table visualization from a SQL schema definition. The code defines a `Demo` table with three columns—`ID`, `Key`, and `Value`—and shows how to map those column types and constraints into a visual entity-relationship style diagram. Using the provided `CREATE TABLE` statement as the data source, the visualization renders each column as a node, with the primary key and nullable attributes indicated through distinct visual encodings. The implementation leverages `d3.forceSimulation` for the node layout and `d3.geoPath` for any geometric rendering, while relying on standard DOM elements for interactivity. No additional behavior beyond the table’s structural representation is implied.

HHugoberry

64% match

Loading thumbnail…

Gist 7fcfddfdb64357791cf2524677924c12

This visualization parses and displays the hierarchical structure of a DAX query plan, showing operators and their parameters as an indented tree. The code uses the Moo lexer to tokenize the plan text, then a custom parser builds a tree from indentation and operator tokens. It renders the tree using D3.js force simulation, with nodes representing operators and edges connecting parent-child relationships, while leaf nodes display property values like types and ranges. The visualization makes the query plan’s execution flow and data lineage immediately apparent.

HHugoberry

64% match

Loading thumbnail…

Gist 4905f6e091305c6a3ff3c1fe2c0a50b2

This example visualizes the structure of a DAX query plan by parsing a text-based log file into a hierarchical tree. It uses the Moo lexer to tokenize lines, identifiers, column references, and operator types, then builds a nested representation of the plan. The visualization likely renders this tree to show the flow of logical and physical operations, with each node displaying its operator type and associated properties such as filters, joins, or spooling details. The code defines custom token patterns for DAX-specific syntax, including column references like `'Table'[Column]` and measure references like `[Measure]`, and parses them into structured objects for display.

HHugoberry

63% match

Loading thumbnail…

Gist 8bb4970e2700b40a3dce1d785654d907

This visualization uses a force-directed graph to map the relationships between identifiers extracted from a Power Query M codebase. The data originates from a gist by Hugoberry, showing dependencies among functions, modules, and error-handling patterns. The rendering relies on D3.js’s `forceSimulation` with custom link and node positioning, while each node’s label is drawn using SVG text. The code specifically highlights groupings like `TableAction.*`, `ValueAction.*`, and `Pattern.*`, revealing hierarchical clusters of related API calls. The graph’s layout is animated via D3’s tick handler, and the visible node sizes suggest a metric like degree centrality, though the exact encoding is not explicit in the snippet.

HHugoberry

62% match

Loading thumbnail…

Gist 0ae5e1a27ffc3cfab328c1895ac272cf

This example visualizes the structure of a DAX query plan by parsing its text into a hierarchical tree. It uses the `moo` lexer to tokenize the query plan, recognizing operators, column references, measure references, and property assignments. The parser builds a nested tree from indentation and line breaks, while a custom `parseProperties` function extracts key-value pairs from tokens, including hash-prefixed attributes and parenthesized groups. The visualization reveals the query plan’s operator flow and parameter details, though the rendering itself is not implemented in the shown code.

HHugoberry

60% match

Loading thumbnail…

Gist aab86eb5543eee75b540e8e173212007

This dendrogram visualization displays the Flare class hierarchy as a node-link tree diagram, with all leaf nodes aligned along the right edge and internal package nodes branching to the left. The example uses the classic Flare dataset (courtesy Jeff Heer) and employs D3's tree layout with SVG rendering. Leaf nodes are positioned at a uniform depth, creating a clean right-aligned column of class names, while curved links connect them to their parent packages. This layout makes it easy to compare hierarchical depth across different branches at a glance. The visualization is implemented with D3 v3 and uses the standard dendrogram layout algorithm.

AAdlopez2016

60% match

Loading thumbnail…

Dendrogram

This dendrogram visualizes a hierarchical clustering of the Flare data set, showing the nested structure of its analytics, animation, and visualization classes. Internal nodes are rendered as small circles, with connecting links drawn as smoothed curves, while leaf nodes display labels. The example uses D3 v4 and SVG rendering, employing the `d3.cluster` and `d3.stratify` layouts to position nodes based on their hierarchical depth and computed height. Data is loaded from a CSV file, and the chart’s orientation places the root at the left, expanding rightward.

Mmbostock

59% match

Loading thumbnail…

D3 + Vue.js

This cluster dendrogram visualizes the Flare dataset, showing hierarchical relationships between software classes and modules as a tree of nodes and links. Built with Vue.js, the SVG rendering is driven by Vue’s template syntax, replacing D3’s `enter`/`exit`/`update` with reactive computed properties that bind data changes directly to the chart. This approach highlights Vue’s strength in building re-usable, reactive components for data-driven visualizations, while D3 handles the layout and scales. The example demonstrates how Vue can manage the DOM bindings and updates, making the chart more maintainable in complex applications.

FFrissAnalytics

59% match