Skip to content

TRUNCATION_STARVE — rewards are computed on cut-off completions

Severity: warn · Key metrics: truncation_rate, max_len_hit_frac, completion_len_p95, eos_rate

What is happening

When a completion hits max_new_tokens, generation stops mid-thought and the reward function grades an unfinished answer. The learning signal is corrupted in a specific way: the policy is being taught about text it didn't choose to end there — the length cap, not the policy, decided where the answer stopped. An answer that was heading toward correct scores as wrong; a rambling answer gets judged only on its opening. The credit assignment is systematically off.

Two distinct regimes:

  • Rising truncation usually rides on completion-length growth — often downstream of REWARD_HACK_LENGTH, and it feeds back: once most completions are truncated, the reward can't distinguish "long and complete" from "long and cut off", which further muddies the signal.
  • Chronic ~100% truncation from step 0 means the model has no working EOS behavior at all — usually mechanical: wrong stop tokens, a chat-template mismatch, or a base model that simply never learned to stop in this format. No RL hyperparameter fixes a tokenizer problem.

What it looks like in the recording

truncation_rate (fraction of completions hitting the cap) climbing past 50% and holding, with completion_len_p95 pinned at max_new_tokens and eos_rate falling in mirror image. In the chronic case, the whole series sits near 1.0 from the start.

How the detector decides

Requires the late-window mean of truncation_rate above threshold, and then one of two shapes: rising (late mean > growth_ratio× the early mean) or chronic (whole-run mean above chronic_frac). The finding says which — the remediation is different.

knob default meaning
threshold 0.5 late truncation rate that counts as starving
growth_ratio 2.0 late/early ratio for the "rising" shape
chronic_frac 0.8 whole-run mean for the "chronic" shape

What to try

  • Rising: if length is also growing, treat the length signature first — this is its symptom. Otherwise raise max_new_tokens (grades whole answers, costs compute) or add a non-termination penalty (teaches stopping directly).
  • Chronic: verify EOS handling before touching any training knob — chat template, stop-token configuration, whether the base model ever emits EOS in this format at temperature. Truncated-mid-thought samples in the recording (truncated=True) make this a thirty-second check.

REWARD_HACK_LENGTH — the usual upstream cause of the rising shape. THROUGHPUT_ROT — max-length completions are also the slowest ones.