A language-model agent combines a model and a harness. The harness is the executable program that runs the model. Different tasks need different harnesses. The paper introduces harness learning to revise a harness using execution feedback. A proposer model is trained with reinforcement learning to improve harnesses. At test time the proposer refines the harness without updating model weights. Experiments show harness learning improves revision quality. The ability to adapt transfers to unseen tasks. Policies trained on single revisions can keep improving harnesses over multiple rounds. Benefits of training on revision sequences vary by setting.
What it unlocks
A language model can now adapt its executable program at test time using feedback from new tasks.
Sources