> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cascadeflow.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# How It Works

> Architecture of cascadeflow's two engines — Cascade for speculative model routing and Harness for agent runtime intelligence.

cascadeflow ships two complementary engines that can be used independently or together.

## Cascade Engine

The Cascade Engine optimizes model selection through **speculative execution with quality validation**:

1. **Speculatively executes** small, fast models first — optimistic execution (\$0.15-0.30/1M tokens)
2. **Validates quality** of responses using configurable thresholds (completeness, confidence, correctness)
3. **Dynamically escalates** to larger models only when quality validation fails (\$1.25-3.00/1M tokens)
4. **Learns patterns** to optimize future cascading decisions and domain-specific routing

In practice, 60-70% of queries are handled by small, efficient models without escalation.

**Result:** 40-85% cost reduction, 2-10x faster responses, zero quality loss.

```
Query → Domain Detection → Try Draft Model → Quality Check
                                                  │
                                          Pass ───┘─── Fail
                                           │            │
                                        Return      Escalate to
                                        Result      Verifier Model
```

## Harness Engine

The Harness Engine provides **agent runtime intelligence** — budget enforcement, compliance gating, KPI-weighted routing, energy tracking, and decision traces.

Unlike the Cascade Engine which routes between models, the Harness Engine wraps existing agent execution and makes decisions at every step:

```
Agent Step → Harness Decision → allow / switch_model / deny_tool / stop
                 │
                 ├── Check budget remaining
                 ├── Check compliance allowlist
                 ├── Score KPI dimensions
                 ├── Check tool call cap
                 ├── Check latency cap
                 └── Check energy cap
```

### Decision Flow

For each LLM call or tool execution inside an agent loop, the harness:

1. **Records** the model, step number, and cumulative metrics
2. **Evaluates** all configured constraints (budget, compliance, tool calls, latency, energy)
3. **Scores** the call against KPI weights if configured
4. **Decides** an action: `allow`, `switch_model`, `deny_tool`, or `stop`
5. **Enforces** the action if in `enforce` mode (logs only in `observe` mode)
6. **Appends** a trace record for auditability

### HarnessConfig

All harness behavior is configured through a single dataclass:

```python theme={null}
HarnessConfig(
    mode="enforce",           # off | observe | enforce
    budget=0.50,              # Max USD for the run
    max_tool_calls=10,        # Max tool/function calls
    max_latency_ms=5000.0,    # Max wall-clock ms per call
    max_energy=100.0,         # Max energy units
    compliance="gdpr",        # gdpr | hipaa | pci | strict
    kpi_weights={"quality": 0.6, "cost": 0.3, "latency": 0.1},
    kpi_targets={"quality": 0.9},
)
```

## Combined Usage

When both engines are active, the Cascade Engine handles model selection while the Harness Engine enforces constraints:

```python theme={null}
import cascadeflow
from cascadeflow import CascadeAgent, ModelConfig

# Harness: enforce budget and compliance
cascadeflow.init(mode="enforce")

# Cascade: speculative model routing
agent = CascadeAgent(models=[
    ModelConfig(name="gpt-4o-mini", provider="openai", cost=0.000375),
    ModelConfig(name="gpt-4o", provider="openai", cost=0.00625),
])

with cascadeflow.run(budget=1.00) as session:
    result = await agent.run("Analyze this contract for GDPR compliance")
    print(session.summary())
```

## Provider Abstraction

cascadeflow supports 17+ providers through a unified interface:

| Provider      | Type       | Package                    |
| ------------- | ---------- | -------------------------- |
| OpenAI        | API        | `cascadeflow[openai]`      |
| Anthropic     | API        | `cascadeflow[anthropic]`   |
| Groq          | API        | `cascadeflow[groq]`        |
| Together      | API        | `cascadeflow[together]`    |
| Hugging Face  | API        | `cascadeflow[huggingface]` |
| Ollama        | Local      | Built-in (HTTP)            |
| vLLM          | Local      | `cascadeflow[vllm]`        |
| Vercel AI SDK | TypeScript | `@cascadeflow/vercel-ai`   |

<Tip>
  **Go deeper:** [Agent Harness](/get-started/agent-harness) | [Agent Loop](/get-started/agent-loop) | [Harness Overview](/harness/overview) | **Example:** [examples/basic\_usage.py](https://github.com/lemony-ai/cascadeflow/blob/main/examples/basic_usage.py)
</Tip>