Conditional and dynamic workflows

Flytekit provides two primary mechanisms for introducing non-linear logic into workflows: Conditional Sections and Dynamic Workflows. While both allow for branching, they operate at different stages of the workflow lifecycle and have distinct constraints.

Conditional Sections

Conditional sections allow you to define branching logic that is evaluated at runtime by the Flyte engine. Because these branches are defined using the conditional function, they are fully visible to the compiler, allowing Flyte to visualize the entire graph (including all possible branches) before execution begins.

Usage and Syntax

You define a conditional block using the conditional function, which returns a ConditionalSection. This section uses a fluent API (if_, elif_, else_, then, and fail) to build the logic.

from flytekit import task, workflow, conditional

@task
def double(n: float) -> float:
    return n * 2.0

@task
def square(n: float) -> float:
    return n * n

@workflow
def my_workflow(my_input: float) -> float:
    return (
        conditional("fractions")
        .if_((my_input > 0.1) & (my_input < 1.0))
        .then(double(n=my_input))
        .elif_((my_input >= 1.0) & (my_input < 10.0))
        .then(square(n=my_input))
        .else_()
        .fail("Input out of range")
    )

Compilation and Execution Semantics

When a workflow is compiled, flytekit creates a BranchNode (found in flytekit/core/condition.py). This node contains an IfElseBlock that encapsulates all possible execution paths.

1. Static Analysis: Unlike standard Python if statements, conditional blocks are evaluated during workflow registration. The compiler visits every branch to ensure type safety across the entire structure.
2. Output Consistency: Every branch in a conditional section must return the same type. ConditionalSection.compute_output_vars() determines the intersection of outputs across all branches to ensure the workflow remains valid regardless of which path is taken.
3. Local Execution: During local execution, flytekit uses LocalExecutedConditionalSection. It evaluates the expressions immediately and uses ctx.execution_state.take_branch() to execute only the selected path, effectively short-circuiting the other branches.

Constraints on Expressions

Expressions in if_ and elif_ must use Flyte-compatible operators. The Case class in flytekit/core/condition.py enforces these rules:

• No Unary Promises: You cannot use if_(my_input). You must use a comparison like if_(my_input == True).
• Supported Operators: Only Comparison (<, <=, >, >=, ==, !=) and Conjunction (&, |) expressions are allowed.
• No Python and/or/not: Standard Python logical operators cannot be overloaded for Flyte promises; you must use bitwise operators (&, |) for conjunctions.

Dynamic Workflows

Dynamic workflows are used when the structure of the workflow (the number of tasks or the specific dependencies) depends on the value of a runtime input. While a conditional section has a fixed set of branches, a @dynamic workflow can generate a completely new graph at runtime.

Defining Dynamic Logic

A dynamic workflow is defined using the @dynamic decorator (a partial application of @task with ExecutionBehavior.DYNAMIC).

from flytekit import dynamic, task

@task
def process_item(item: int) -> int:
    return item * 2

@dynamic
def my_dynamic_subwf(count: int) -> list[int]:
    results = []
    # In a @dynamic task, you can use native Python logic like range()
    # which is forbidden in a standard @workflow.
    for i in range(count):
        results.append(process_item(item=i))
    return results

How Dynamic Workflows Work

Internally, a @dynamic function is treated as a task by the parent workflow. However, when the Flyte engine executes this task:

1. It runs the function body.
2. Instead of returning a final value, the function returns a set of task executions (a compiled sub-workflow).
3. The Flyte engine then executes this generated sub-workflow.

Choosing Between Conditional and Dynamic

Feature	Conditional Section	Dynamic Workflow
Visibility	Full graph visible at registration.	Graph is hidden until runtime.
Python Logic	Limited to Flyte expressions (&, `	, ==`).
Performance	Low overhead; handled by the engine.	Higher overhead; requires a task execution to generate the graph.
Use Case	Simple branching based on task outputs.	Data-parallelism, loops, or complex logic.

Use conditional when you have a fixed set of alternative paths. Use @dynamic when the number of tasks or the workflow structure itself is determined by the data.