Task Decomposition — Fixed Pipeline vs Dynamic Adaptive

Core 8 min +30 XP

💡

THE ANALOGY

Following a recipe vs cooking by feel. A recipe (fixed pipeline) tells you exactly what to do in what order — great for consistency. Cooking by feel (dynamic decomposition) means you taste as you go and adjust — great for complex dishes where you can't know every detail in advance.

⚠️ EXAM TRAP — The Wrong Answer People Choose

Applying fixed pipeline decomposition to open-ended tasks where you don't know what you'll find. If your pipeline assumes Step 3 will always produce X, but the actual data doesn't match that assumption, the pipeline breaks silently.

KEY POINTS

1 Fixed pipeline: predefined sequence of steps with known inputs and outputs. Use when the task structure is fully known in advance.

2 Dynamic decomposition: coordinator discovers what needs to be done and creates a plan based on findings. Use when the right approach depends on what's found.

3 The coordinator trap: decomposing a broad topic into subtasks that are too narrow — missing entire domains of relevant content.

4 Granularity decisions: subtasks should be large enough to be meaningful but small enough to fit in a subagent context window.

5 Dependency analysis: identify which subtasks can start immediately and which must wait for prior results.

Fixed Pipeline Decomposition

Use when you know the exact steps before you start.

Characteristics:

Same steps in the same order every time
Each step’s output type is predictable
The task is well-understood and repeatable
Reliability and consistency are priorities

Example: Customer Support Resolution Pipeline

# Fixed pipeline — always these steps, always this order
SUPPORT_PIPELINE = [
    {"step": "verify_identity",    "tool": "get_customer",     "required": True},
    {"step": "get_context",        "tool": "get_order_history", "required": True},
    {"step": "check_policy",       "tool": "get_policy",       "required": True},
    {"step": "determine_action",   "tool": None,                "required": True},  # Claude decides
    {"step": "execute_resolution", "tool": "process_refund",   "required": False}, # Optional
    {"step": "send_confirmation",  "tool": "send_email",       "required": True},
]

Every customer support interaction follows this pipeline. The structure never changes even though the data does.

Dynamic Adaptive Decomposition

Use when you need to discover what to do before you can plan how to do it.

Characteristics:

Coordinator first explores the problem space
Subtasks are created based on what’s found
Different runs produce different execution graphs
Adaptability to unexpected findings is required

Example: Legacy Codebase Test Coverage

# Dynamic — coordinator decides structure based on exploration
coordinator_prompt = """
Your task: Add comprehensive test coverage to this legacy codebase.

Phase 1 (do this first):
- Explore the codebase structure
- Identify which modules have zero or minimal test coverage
- Assess which untested modules have the highest business impact
- Identify dependencies and test infrastructure

Based on Phase 1 findings, create a prioritized test implementation plan.
The number and type of subagents you spawn in Phase 2 should match
what Phase 1 reveals — not a fixed structure decided in advance.
"""

The coordinator might discover 3 high-impact modules needing tests, or 15. It adapts based on what it finds.

The Coordinator Trap (Exam Tested)

The coordinator trap is a specific type of decomposition failure: subtasks that are internally coherent but miss entire categories of the original scope.

The scenario:

Task: “Analyze the impact of AI on creative industries”

Trapped decomposition:

“AI in digital art creation”
“AI in graphic design tools”
“AI in photography editing”
“AI in illustration”

Result: A comprehensive report on AI in visual arts. Music, writing, film, game design, architecture — all missing.

Why it happens: The coordinator recognized a coherent category (visual arts) and stayed within it, missing that “creative industries” is a much broader scope.

Prevention:

coordinator_prompt = """
Task: Analyze the impact of AI on creative industries.

SCOPE CHECK: Before creating your decomposition plan, explicitly list ALL 
categories that fall under 'creative industries.' Your subtask plan must 
cover every category you identify. If you identify music, literature, film,
visual arts, architecture, and game design — all six need coverage.

Do not create a plan that addresses only a subset of the categories you identify.
"""

Granularity Decisions

Too fine-grained:

Subagent overhead (spawning, context setup) exceeds the value of the task
Example: “Analyze this one function’s complexity” — a subagent for one function is wasteful

Too coarse-grained:

Subtask is too large to fit in the subagent’s context window
Example: “Analyze the entire authentication system” — might be 50 files

Right granularity:

Meaningful unit of work (logical module, document section, related group of files)
Fits comfortably within context window with room for reasoning
Produces a result that stands alone (can be understood without the full context)

# Example: Code review with right granularity
# Bad: one subagent for all 200 files
# Bad: one subagent per file (200 subagents for 200 files — overhead)
# Right: group related files by module (8-12 subagents for a 200-file codebase)

def group_files_by_module(files: list) -> dict:
    """Group files into cohesive modules for subagent assignment."""
    modules = {}
    for file in files:
        module = detect_module(file.path)  # auth, payments, orders, etc.
        modules.setdefault(module, []).append(file)

    # If a module is very large, split it
    result = {}
    for module, module_files in modules.items():
        if len(module_files) > 20:
            # Split large modules into chunks
            for i, chunk in enumerate(chunks(module_files, 15)):
                result[f"{module}_part{i}"] = chunk
        else:
            result[module] = module_files

    return result

Key Takeaways

Fixed pipeline for known, repeatable, well-understood workflows
Dynamic decomposition for open-ended investigation
Coordinator trap: always verify your decomposition covers the full scope
Granularity: meaningful unit, fits in context, produces standalone result
Discovery first: for dynamic tasks, Phase 1 explores, Phase 2 acts on findings