AlignmentAlice
Features as Rewards — Circular Feedback Danger
Features as Rewards: Scalable Supervision for Open-Ended Tasks via Interpretability
Published: 12 May 2026 · Updated: 13 July 2026
Read the original sourceWhat the paper says
Features as rewards provides scalable supervision for open-ended tasks.
The Critique
This creates a circular feedback loop that could amplify biases. Features are extracted from the model, used to reward the model, reinforcing those features. If features encode existing biases, this systematically reinforces confident-but-wrong behaviors.
Why It Matters
Without careful validation, this could lead to runaway feedback loops where model biases are amplified rather than corrected.
What They Missed
They don't address how to detect when the feature-reward loop has diverged from human values or ground truth.