Context Language Models

<p>This paper introduce <strong>Context Language Models (CLMs)</strong>, a novel approach that enables language models to natively manage their own context by treating it as an editable file rather than relying on traditional append-only histories or external harnesses. By granting models unrestricted read and write access to their context, <strong>CLMs</strong> achieve superior performance and greater efficiency across long-horizon coding, deep research, and open discovery tasks. The literature demonstrates that these models can be further enhanced through <strong>in-context learning</strong>, <strong>natural-language steering</strong>, and <strong>online reinforcement learning</strong> with success-gated efficiency rewards. To address the computational challenges of frequent context edits, the authors propose <strong>Suffix Cache Reuse (SCR)</strong>, an advanced serving optimization that reuses cached states for surviving suffix tokens to significantly reduce server-side compute. Ultimately, this paradigm shift from external harness control to intrinsic model behavior unlocks more adaptive, creative, and resource-efficient context-management strategies.</p>