Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning
Teaching AI to think across entire documents instead of word by word
AI models trained on their own outputs using teacher feedback often make locally reasonable mistakes when tasks require evidence scattered across long documents. Researchers diagnosed this problem and created GC-OPD, a method that aligns token-level feedback with task-level scoring, improving performance by 11 percentage points on long-context reasoning benchmarks.
Long-context tasks like research synthesis and document analysis require AI to track information across thousands of words. Current training methods miss this challenge by optimizing one word at a time, causing models to miss distributed evidence or violate task constraints. This approach fixes that mismatch and shows measurable gains on real benchmarks, making AI more reliable for work that demands careful attention to entire documents.