Governing Agentic Loops: Preventing Unbounded Iteration in LLM-Driven Systems

Department of Computer Science | Level: Intermediate | Duration: 25 minutes

Objectives

Distinguish productive iteration (convergent) from pathological loops (divergent) in LLM-driven agents
Understand why termination conditions must be enforced externally rather than self-reported
Apply the six required properties of a governed loop to real system designs
Map loop governance to Default Article 9 (Bounded Autonomy) of the Demerzel constitution

1. The Loop Problem

An LLM agent given a goal will iterate toward it. This is useful — iterative refinement is how complex tasks get done. But it creates a structural hazard: the same reasoning that produced the loop can produce the “I’ve converged” assessment.

This is not a bug in any specific model. It is an intrinsic property of autoregressive generation: the model can’t observe its own behavior from outside. It can describe convergence, but cannot guarantee it. The guarantee must come from the framework.

The Halting Parallel

Alan Turing proved (1936) that no algorithm can decide for all programs whether they will halt. No general procedure can settle it for an arbitrary agent loop either. So termination is imposed rather than detected: a framework-enforced cap bounds the number of iterations, and a framework-enforced deadline on each whole iteration stops it when it expires, if need be by killing the process that runs it. Only the two together make the loop halt by construction: an iteration that never ends never reaches the counter, and a deadline on each call is not enough, since an iteration can make any number of calls or compute between them.

Implication: Any agentic framework that relies on the model to declare its own completion is unsound by construction.

2. Taxonomy: Productive vs. Pathological Loops

Property	Productive	Pathological
Each iteration produces a distinct state	Yes	No — outputs repeat or drift
Termination criterion is definable pre-loop	Yes	No — criterion is generated in-loop
Progress is measurable externally	Yes	No — only self-reported
A human can inspect intermediate state	Yes	No — internal only
Loop can be paused and resumed	Yes	No — state is not serializable

A loop is productive when every iteration moves the system measurably closer to a definable terminal state. It is pathological when it generates tokens without generating state transitions.

3. Six Required Properties of a Governed Loop

These six properties make an iteration bounded and auditable. A loop that must be paused and resumed also needs serializable state (sections 2 and 6):

Property 1: Hard Iteration Cap

A maximum iteration count enforced by the framework, not the model. When reached: halt, log the cap, escalate to human review. The counter advances only when an iteration ends, so the cap also needs a deadline around each whole iteration, enforced by the framework, which stops the iteration when it expires, if need be by killing the process that runs it. Timeouts on model calls, tool calls and network requests are useful but do not bound an iteration by themselves. The configuration below sets only the cap.

# Example: one loop's configuration (policy default 10, absolute max 25)
max_iterations: 12
cap_behavior: halt_and_escalate

Property 2: Progress Test

Each iteration must produce a measurable state change. The framework compares state hashes before and after each step. If hash(state_n) == hash(state_n-1), the loop is stalled. Hash the fields that carry the result, not the iteration counter — that one differs at every step, so the stall test would never fire. And a stall test catches repetition, not drift: a loop that keeps changing without converging never meets the external criterion, and the cap stops it, not this test.

def stall_test(state_before, state_after):
    return hash(state_before) == hash(state_after)

if stall_test(prev_state, curr_state):
    raise StallDetected("No state change — possible infinite loop")

Property 3: External Termination Criterion

The exit condition is specified before the loop starts, not generated during execution. The model cannot redefine convergence mid-loop. The iteration cap is not part of the criterion: a loop that reaches the cap has not converged, and it halts and escalates as Property 1 requires, rather than reporting its work as complete.

# Good: criterion is external
def is_complete(state) -> bool:
    return state.belief_confidence >= 0.85

# Reaching the cap is not success: halt and escalate (Property 1)
def cap_reached(state) -> bool:
    return state.iteration >= MAX

# Bad: model declares its own completion
result = model.run("keep going until you think you're done")

Property 4: Human-Readable Checkpoint

Every N iterations, the framework emits a checkpoint: a structured log entry a human can read without running the loop. This serves as both observability and audit trail.

Property 5: Output Deduplication

The framework tracks the set of outputs emitted so far. If a candidate output is functionally identical to a prior output, it is flagged as a loop indicator.

Property 6: External Exit Decision

The model proposes termination; the framework decides. The model’s “I’m done” is treated as a vote, not a command.

4. The Demerzel Governed Loop Pattern

The Demerzel framework specifies this pattern in autonomous-loop-policy.yaml, with iteration and stall limits but no per-iteration deadline:

GOVERNED LOOP
├── Pre-conditions (checked before first iteration)
│   ├── Kill switch check
│   ├── Daily/session cap check
│   └── Termination criterion defined
│
├── Iteration body
│   ├── Execute step
│   ├── Progress test (hash compare)
│   ├── Checkpoint emit (every N steps)
│   └── Output dedup check
│
└── Post-conditions (any can halt the loop)
    ├── Termination criterion met → complete
    ├── Iteration cap hit → escalate
    ├── Stall detected → escalate
    ├── Kill switch set → halt immediately
    └── Anomaly detected → conscience signal + halt

This pattern appears in three places in the Demerzel ecosystem, and only the first two bound the duration of an iteration:

Seldon Plan: 6 cycles/day cap, novelty registry as progress test, a 30-minute soft limit per cycle (policies/seldon-plan-policy.yaml) and a hard kill at 35 minutes (timeout-minutes in .github/workflows/seldon-plan.yml)
Demerzel Driver: a pause for human review after 5 consecutive unattended cycles, conscience signals as anomaly detection, cycle timeout (2 hours soft, then a hard kill at 2h15m)
Ralph Loop: iteration cap + convergence metric (test pass rate) as external criterion, but no per-iteration deadline: neither policies/autonomous-loop-policy.yaml nor .claude/skills/demerzel-loop/SKILL.md defines one, so an iteration that blocks never reaches the counter

5. Constitutional Grounding

Default Article 9 — Bounded Autonomy:

Agents operate within predefined bounds. Autonomy is a resource, not a right. When bounds are reached, escalate — do not self-authorize extension.

The six properties above operationalize Article 9 for iterative processes. Specifically:

Hard cap = predefined bound
External termination = “predefined” (not in-flight)
Escalation at cap = “escalate, do not self-authorize”

Default Article 7 — Auditability:

Every cycle must be logged with full trace.

Checkpoints and output dedup logs satisfy this: the loop is auditable even mid-execution.

6. Anti-Patterns

Anti-pattern	Why it fails	Fix
`while not model.done()`	Model declares its own completion	Replace with external criterion
Iteration count in prompt (“try 5 times”)	Model can override in generation	Enforce in framework, not prompt
“Keep improving until satisfied”	Unbounded, satisfaction is self-reported	Define a measurable satisfaction metric
No checkpoint logging	Loop not auditable in flight	Emit checkpoint every N iterations
State not serialized	Loop can’t be paused/resumed	Use state machine, serialize each step

Key Takeaways

An LLM cannot reliably detect its own infinite loops — termination must be external
A governed loop has six properties: hard cap with a deadline on every iteration, progress test, external criterion, checkpoint, dedup, external exit decision — plus serializable state if it must be paused and resumed
The Demerzel framework applies the pattern in seldon-plan, demerzel-drive, and Ralph Loop; only the first two set a per-iteration deadline, so Ralph Loop does not yet halt by construction
Article 9 (Bounded Autonomy) is the constitutional basis — bounds are predefined, extension requires escalation