DAG dynamically generating tasks

100%

DAG dynamically generating tasks

✓ Published0🌍 Public

Last edited Jul 14, 2018

Created on Jul 14, 2018

This example shows an Airflow DAG that dynamically generates a variable number of tasks at runtime, using a `random.choice` call to select between one and six `BashOperator` tasks. The DAG links a starting `print_date_1` task to each generated task, and each generated task to a shared `final` task. The code constructs these dependencies with the `>>` and `<<` operators, while the Airflow `BashOperator` and `DAG` classes manage task execution. The visualisation displays the dynamically created task graph, highlighting how Airflow's scheduler can struggle when task topology changes mid-run.

AI-generated description

Similar vizzes

Loading thumbnail…

etl

This example visualizes an ETL workflow as an animated, interactive graph. It shows the dependencies between tasks in an Apache Airflow DAG, with colored nodes representing different task types (e.g., BashOperator, DummyOperator) and edges indicating execution order. The visualization uses the D3.js library to construct the node-link diagram within an SVG, with CSS transitions that highlight and fade elements to emphasize the flow. The underlying data, including task states like “success” and “skipped,” is sourced from a simulated Airflow run, and the graph is rendered using a force-directed layout.

MMing LIU

56% match

Loading thumbnail…

random task =)

This example visualizes the output of a Ruby `Task` class’s `reccurent` method, which generates an array of recurring month-day strings from a starting date. The code implements date arithmetic via the `>>` operator, using `strftime` for formatting, and demonstrates edge-case handling when the target day does not exist in a month (e.g., February 28). The visualization, driven by RSpec test expectations, highlights the method’s behavior for valid recurrence counts (1–12) and its argument validation. The data appears as a static set of expected date lists derived from the spec file’s assertions, showing the recurring dates for five and eight month intervals.

66ewis

54% match

Loading thumbnail…

Gist 7facf6eee4cdc2d78b333fb54eafc2c9

This example demonstrates a dynamic task-management interface that updates in response to simulated HTTP polling data. The visualization shows a table of devices and their associated tasks, where each periodic response adds new tasks to existing devices or introduces entirely new device rows. Users can click on individual tasks to complete them, removing them from the display to free up space for future updates. The implementation uses plain JavaScript with a custom `getResponse` function to generate random device-task assignments, and the UI is built directly via the DOM API without external libraries, featuring a clean two-column layout for professional presentation.

WWill Carter

50% match

Loading thumbnail…

Gist 68cc98468a08d322f2da

This example transforms a GitHub Gist into a visual to-do list, rendering each unchecked task as a draggable node. The code uses the D3.js library’s force-directed graph layout (`d3.forceSimulation`) to arrange the items and apply interactive physics. Each task is represented by a circle, and the list’s markdown structure is parsed directly from the Gist’s raw content. The visualization allows repositioning of nodes while preserving the original task order, offering a playful, spatial interaction with a simple text-based checklist.

EEleonore9

48% match

Loading thumbnail…

Force Directed Graph For TaskGraph #2

This example displays a force-directed graph of a small task hierarchy, using nodes named albert, bettie, carly, donna, echo, and father, with links that are either solid blue or dashed black. The visualization is rendered as an SVG with D3 v5, using `d3.forceSimulation` to compute node positions with link, charge, collision, and centering forces. The code builds nodes dynamically from the link data and supports interactive dragging of nodes, with links styled according to a binary value attribute that dictates their color and dash pattern.

Aantoniojl@gmail.com

48% match

Loading thumbnail…

Task Graph With Force Simulation

This example visualizes a task graph as a node-link diagram with a force-directed layout. It shows how parent-child relationships among tasks (such as “albert” and “betty”) are inferred from a data structure that records each node’s parent, then rendered as an SVG with circles and connecting paths. The code uses d3.forceSimulation with forceLink, forceManyBody, forceCenter, and forceCollide to position nodes dynamically, while tick events update the paths and node positions. Drag behavior is enabled via d3.drag, and links are styled with dashed strokes or colors based on link values.

Aantoniojl@gmail.com

48% match

Loading thumbnail…

Timeline with Procedurally Generated Data

This animated example shows a timeline visualization that procedurally generates random bands of data, with each band’s start and end values falling between 1 and 40. Every 500 milliseconds, a new band is added, and the timeline updates over a 400-millisecond transition, ultimately generating 100 bands. Earlier bands are rendered with greater transparency than later ones, creating a sense of depth. The d3.layout.timeline library’s `extent` and `maxBandHeight` settings keep the axes fixed so the chart does not resize as data is appended.

DDinoJay

47% match

Loading thumbnail…

task算子柱状图

This bar chart visualizes the execution duration of task operators, segmenting each task’s timeline into colored bands that represent individual metrics like input read time, shuffle write bytes, and peak execution memory. The SVG rendering uses d3.scaleLinear and d3.scaleBand to map durations and task IDs, while d3.axisLeft and d3.axisBottom display the axes. Metrics are layered as stacked rectangles within each bar, with data loaded from a JSON file via d3.json.

Wwyh1029

47% match