Skip to content

REPETITION_SPIRAL — generated text is looping

Severity: warn · Key metrics: repetition_rate, cross-check entropy_mean

What is happening

Repetition loops are an attractor state for degenerate policies. The mechanics are a feedback loop: once a repeated phrase gets any reward (or simply avoids punishment), the policy's probability of emitting it again rises; each repetition makes the next more likely (self-conditioning — the context is now full of the phrase); entropy drops as the distribution sharpens onto the loop; and lower entropy means alternatives stop being sampled, so nothing ever contradicts the habit. the the the… is the caricature, but the realistic versions are subtler — restated sentences, cycling paragraph templates, boilerplate reasoning steps recycled across prompts.

This matters beyond aesthetics: repetition is a leading indicator of ENTROPY_COLLAPSE. The loop is what a policy sliding toward determinism sounds like while it still has entropy left to lose. Catching the spiral early is cheaper than catching the collapse late.

The recorder computes repetition_rate as a cheap n-gram statistic over sampled completions each step — crude, but trends in it are exactly the signal needed.

What it looks like in the recording

repetition_rate climbing from a low baseline (a few percent) with a significant rising trend, typically alongside a declining entropy_mean. The captured samples make it concrete: read the top-reward completions around the onset step and the loop is usually visible verbatim.

How the detector decides

Fires when the late-window mean exceeds late_above, the early mean was below early_below (the model didn't start repetitive), and the whole series has a significant Mann-Kendall rising trend (p < 0.05). All three together separate a spiral from both noisy stability and a chronically repetitive base model.

knob default meaning
late_above 0.3 late repetition rate that counts as looping
early_below 0.15 early mean must have been below this

What to try

  • Entropy bonus / higher temperature — attack the low-entropy substrate the loop grows on.
  • Repetition penalty during rollouts — directly suppresses loops at generation time; a blunt but fast-acting instrument.
  • Inspect which prompts seed it — repetition often starts in a degenerate prompt subset (ambiguous, unanswerable, or trivially gameable prompts) and generalizes from there. rlprobe samples <run> --around <onset> shows you patient zero.
  • Check the reward's blind spot — if the reward function doesn't see repetition (e.g. keyword graders), looping is free; add an explicit repetition term or a judge that penalizes it.

ENTROPY_COLLAPSE — the destination this spiral leads to; treat a co-occurrence as one disease at two stages, and fix the entropy side.