Skip to main content
100%

Test kwargs for Airflow function call

✓ Published0🌍 Public
Aadkdev
Last edited Sep 21, 2021
Created on Sep 21, 2021

This example demonstrates how to test keyword arguments for an Airflow function call by passing an `execution_date` and `next_execution_date` to a test function. The code uses `pendulum.today()` to compute yesterday’s and today’s dates, which are then assigned to the `kwargs` dictionary and unpacked into `test_fn(**kwargs)`. It highlights a simple pattern for validating that a function accepts the expected Airflow context parameters, with `exit()` indicating the script ends after the call.

AI-generated description

Similar vizzes

Loading thumbnail…

A gist illustrating testing in Python using unittest

This example demonstrates how to structure and execute a Python test suite using the built-in `unittest` framework, alongside `doctest` for embedded documentation checks. It shows the definition of a `factorial` function and a `TestFactorial` class that validates expected outputs, error handling, and edge cases. The code dynamically generates test methods via `setattr` and uses `subTest` for granular failure reporting. It also illustrates skipping tests with `@unittest.skip`, suppressing console logging via `logging.NullHandler`, and controlling output with `unittest.main(buffer=True)`. The visible data source is the hardcoded test case list and doctest examples within the function docstring.

IIanHopkinson
46% match
Loading thumbnail…

Gist 9d651a8ae89dc0d88db634886a5e5588

This example shows a serverless deadline checker for Fantasy Premier League match deadlines, using a Python AWS Lambda function. It fetches fixture data from the Fantasy Premier League API and determines whether the next upcoming deadline falls within the next 24 hours. The code converts current and future timestamps, filters fixtures by deadline time, and returns `True` if the nearest deadline is tomorrow, otherwise it exits without a response. It relies on the `requests` library for API calls, `pandas` for dataframe filtering, and the `datetime` module for time calculations, with no visual rendering involved.

CConorAspell
46% match
Loading thumbnail…

Gist 4ae41690dec489b7a725

This example shows a Django application whose settings and database configuration change depending on whether a `test` argument is passed on the command line. It demonstrates a conditional import that selects between a production settings module and a test-specific one, while also adjusting the installed apps list through a `fix_apps` function. The code relies on Python’s `sys.argv` for argument parsing and uses Django’s `DATABASES` and `INSTALLED_APPS` configuration dictionaries. The visualization highlights how environment-specific setup can be cleanly separated and toggled.

Aandybak
44% match
Loading thumbnail…

Gist a4e9165ec14c57fbdc0ca88f9a23ba86

This example shows a serverless health check for a fantasy football league deadline. It compares the next fixture deadline to the current time, returning a boolean that indicates whether the deadline is within the next 24 hours. The code fetches fixture data from the Fantasy Premier League’s `bootstrap-static` API, converts the response to a pandas DataFrame, and filters records by Epoch timestamps. If the deadline is too far away, it calculates the time remaining (days, hours, minutes) and sends a failure notification via an AWS Lambda handler using the `fpl_utils` library’s SNS topic.

CConorAspell
43% match
Loading thumbnail…

get step function status

This example shows the status of a Step Functions execution by extracting the departure forecast from its event history. It queries the execution history using the AWS SDK’s `StepFunctions.getExecutionHistory` method, then filters for `PassStateEntered` events to parse the flight and forecast data embedded in the state input. The handler, written with `co.wrap` and Bluebird promises, returns the execution ARN and the parsed departure object as a JSON response via an API Gateway Lambda trigger.

Aandresmijares
43% match
Loading thumbnail…

DAG dynamically generating tasks

This example shows an Airflow DAG that dynamically generates a variable number of tasks at runtime, using a `random.choice` call to select between one and six `BashOperator` tasks. The DAG links a starting `print_date_1` task to each generated task, and each generated task to a shared `final` task. The code constructs these dependencies with the `>>` and `<<` operators, while the Airflow `BashOperator` and `DAG` classes manage task execution. The visualisation displays the dynamically created task graph, highlighting how Airflow's scheduler can struggle when task topology changes mid-run.

RRobinL
43% match
Loading thumbnail…

Kill all airflow running task

This example shows a shell command that forcefully terminates every running Airflow task by killing their processes. It first displays the command, then explains its mechanism: `ps aux` lists all system processes, `grep "airflow run"` filters for Airflow task executions, `grep -v grep` removes the grep process itself, `awk '{print $2}'` extracts process IDs, and `xargs kill -9` sends a forceful termination signal. The code relies on standard Unix utilities—`ps`, `grep`, `awk`, and `xargs`—rather than a visualization library, presenting the operation as a text-based data flow.

Aadkdev
42% match
Loading thumbnail…

Gist ae727f5f05cbc1b03932

This example demonstrates how to construct and execute a simple Nipype workflow using the pipeline engine. It defines a Python function node that passes inputs through unchanged, connects it to an identity node with iterable parameters, and runs the workflow serially. The visualization shows the workflow's execution log, tracking how multiple parameter combinations (input1 values 1–7 and input2 values 5–9) trigger separate function node runs in distinct output directories. The code uses Nipype's `pe.Workflow`, `pe.Node`, `Function`, and `IdentityInterface` APIs to manage the dataflow.

Aakeshavan
42% match