Airflow - ShortCircuitOperator

 

"""Example DAG demonstrating the usage of the ShortCircuitOperator."""

from __future__ import annotations

import pendulum

from airflow.providers.standard.operators.empty import EmptyOperator
from airflow.providers.standard.operators.python import ShortCircuitOperator
from airflow.sdk import DAG, chain

try:
    from airflow.sdk import TriggerRule
except ImportError:
    # Compatibility for Airflow < 3.1
    from airflow.utils.trigger_rule import TriggerRule  # type: ignore[no-redef,attr-defined]

with DAG(
    dag_id="example_short_circuit_operator",
    schedule=None,
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False,
    tags=["example"],
) as dag:
    # [START howto_operator_short_circuit]
    cond_true = ShortCircuitOperator(
        task_id="condition_is_True",
        python_callable=lambda: True,
    )

    cond_false = ShortCircuitOperator(
        task_id="condition_is_False",
        python_callable=lambda: False,
    )

    ds_true = [EmptyOperator(task_id=f"true_{i}") for i in [1, 2]]
    ds_false = [EmptyOperator(task_id=f"false_{i}") for i in [1, 2]]

    chain(cond_true, *ds_true)
    chain(cond_false, *ds_false)
    # [END howto_operator_short_circuit]

    # [START howto_operator_short_circuit_trigger_rules]
    [task_1, task_2, task_3, task_4, task_5, task_6] = [
        EmptyOperator(task_id=f"task_{i}") for i in range(1, 7)
    ]

    task_7 = EmptyOperator(task_id="task_7", trigger_rule=TriggerRule.ALL_DONE)

    short_circuit = ShortCircuitOperator(
        task_id="short_circuit", ignore_downstream_trigger_rules=False, python_callable=lambda: False
    )

    chain(task_1, [task_2, short_circuit], [task_3, task_4], [task_5, task_6], task_7)
    # [END howto_operator_short_circuit_trigger_rules]

 

"""Example DAG demonstrating the usage of the `@task.short_circuit()` TaskFlow decorator."""

from __future__ import annotations

import pendulum

from airflow.providers.standard.operators.empty import EmptyOperator
from airflow.sdk import chain, dag, task

try:
    from airflow.sdk import TriggerRule
except ImportError:
    # Compatibility for Airflow < 3.1
    from airflow.utils.trigger_rule import TriggerRule  # type: ignore[no-redef,attr-defined]


@dag(schedule=None, start_date=pendulum.datetime(2021, 1, 1, tz="UTC"), catchup=False, tags=["example"])
def example_short_circuit_decorator():
    # [START howto_operator_short_circuit]
    @task.short_circuit()
    def check_condition(condition):
        return condition

    ds_true = [EmptyOperator(task_id=f"true_{i}") for i in [1, 2]]
    ds_false = [EmptyOperator(task_id=f"false_{i}") for i in [1, 2]]

    condition_is_true = check_condition.override(task_id="condition_is_true")(condition=True)
    condition_is_false = check_condition.override(task_id="condition_is_false")(condition=False)

    chain(condition_is_true, *ds_true)
    chain(condition_is_false, *ds_false)
    # [END howto_operator_short_circuit]

    # [START howto_operator_short_circuit_trigger_rules]
    [task_1, task_2, task_3, task_4, task_5, task_6] = [
        EmptyOperator(task_id=f"task_{i}") for i in range(1, 7)
    ]

    task_7 = EmptyOperator(task_id="task_7", trigger_rule=TriggerRule.ALL_DONE)

    short_circuit = check_condition.override(task_id="short_circuit", ignore_downstream_trigger_rules=False)(
        condition=False
    )

    chain(task_1, [task_2, short_circuit], [task_3, task_4], [task_5, task_6], task_7)
    # [END howto_operator_short_circuit_trigger_rules]


example_dag = example_short_circuit_decorator()

 

 

What's the behavior of Airflow ShortCircuitOperator ?

 

The Airflow ShortCircuitOperator is used to conditionally stop downstream tasks based on a Boolean condition.

Here’s its behavior:

---

✅ What it does

ShortCircuitOperator evaluates a Python callable.

• If the callable returns True → the task is marked success, and downstream tasks run normally.

• If it returns False → the task is marked success, but all direct downstream tasks are skipped.

This makes it useful for conditional DAG branching without failing the DAG.

---

⚙️ Key points

1. Does NOT fail the DAG

Returning False does not cause failure; it simply short-circuits downstream tasks by marking them as skipped.

2. Applies only to direct downstreams

Only immediately downstream tasks get skipped.
Other branches in the DAG proceed normally.

3. Works under TaskFlow API and classic operators

You provide a Python function that returns a Boolean.

---

🧩 When to use it

• Skip downstream tasks when a condition is not met

• Prevent unnecessary computations

• Dynamically pause a branch based on upstream data

• Replace simple branching without using BranchPythonOperator