Harness2 framework improves agent performance 11.5 points

Google Cloud AI Research has released Harness2, an open-source recursive agent harnessing framework designed to improve language model agent performance without modifying model weights. The framework addresses a practical problem for developers: agents may perform better through improved prompts, tool configuration, and execution logic alone, without requiring a larger or finer-tuned model.

What changed

Google Cloud AI Research open-sourced Harness2, a framework focused on refining the infrastructure around a frozen model. A harness encompasses everything except the model itself: prompts, tools, memory, skills, and orchestration. The framework provides a method to optimize these components at deployment time, particularly for tasks that were never seen during development.

The release is version 0.1.0 and runs on Linux with Python 3.12 or 3.13. It exposes eight editable harness components as files: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and plugins. These components cover context construction, the tool interface, delegation, and behavior inside the tool loop.

Testing involved three professional benchmarks: WorkBuddy-Bench, JobBench, and Harvey’s LAB. The research team tested two model families, Gemini 3.5 Flash and GLM-5, each under multiple execution substrates. After one refinement step, GLM-5 under OpenCode improved from 64.2 to 75.7 on WorkBuddy-Bench, a gain of 11.5 points. The same configuration improved by 4.5 points on JobBench. Across all four model-harness configurations tested, performance improved on all three benchmarks. Results are reported as means over three runs.

Why it matters

Harness2 addresses a specific gap in how agents improve. Most self-improvement methods assume you have ground-truth labels or test cases to verify correctness. In open-world settings, tasks arrive unseen and no verifier confirms whether an answer is right. Harness2 works without labels by using a contrastive Propose-Probe-Compose step. The agent runs once with the base harness, then an editor proposes two candidate modifications: one targeting general failures like planning errors, another targeting domain-specific knowledge gaps. Both run on the same task, and the system keeps only edits backed by observed behavior differences.

For teams building AI agents, this means potential performance gains from engineering the harness architecture rather than waiting for model improvements. This is especially relevant for weaker models. The data shows GLM-5 benefited more from refinement than Gemini 3.5 Flash, suggesting the approach yields higher returns when the base model has more room to improve through better orchestration.

The framework also includes a second recursion level called harnessing-level recursion. After each task, the system stores the adopted harness plus an EXPERIENCE.md file of lessons learned. This library guides future refinement proposals, meaning improvements discovered on one task can inform choices on subsequent tasks, even though each new task still starts from the base harness.

What to test

Before deploying Harness2 in production, teams should verify several claims:

• Benchmark reproducibility. The paper reports means over 3 runs. Test whether the 11.5-point gain on WorkBuddy-Bench is stable on your own tasks and assess whether performance variance is acceptable for your use case.
• Computational cost. The framework adds extra model calls per task. Parallel mode at k equals 5 means 10 probes plus compositions. Measure wall-clock time and token cost before committing to parallel recursion settings.
• Editor quality. The default editor was Claude Opus 4.8. Test how performance changes if you substitute a different editor model, since proposal quality directly affects refinement outcomes.
• Generalization to your domain. Testing used professional tasks in coding, hiring, and legal work. Verify how well improvements from the lessons library transfer when your tasks differ significantly from WorkBuddy-Bench, JobBench, or LAB benchmarks.
• Harness component coverage. Only eight components are exposed. Confirm that the prompts, tools, memory, and skills you need are actually editable, or whether custom extensions are required.
• Dependency overhead. Installation requires Node.js 20 or higher, the Google Cloud CLI, and a Vertex AI project for model-backed runs. Verify that your infrastructure can support this before piloting.

Common questions

What is Harness2?

Harness2 is not a model; it is a framework. The underlying executors are existing models like Gemini 3.5 Flash or GLM-5. Recursion refers to two linked processes: task-level recursion, which refines the harness for the current task, and harnessing-level recursion, which extracts lessons from that refinement to guide future tasks. Model weights never change.

How does Harness2 work without ground-truth labels?

Instead of comparing results to a correct answer, Harness2 compares the trajectories (step sequences) of two candidate harness modifications against a baseline run on the same task. The system observes behavioral differences and keeps only edits that correlate with better outcomes. This avoids the need for external verification.

Can Harness2 work with any model?

Harness2 model-backed runs call models through Vertex AI. The default executor is OpenCode, with Codex available as an alternative via the --substrate cx flag. The framework is designed to be model-agnostic in principle, but deployment requires model access through supported execution interfaces. Verify that your chosen model is available through these supported substrates.

The conclusion

Harness2 represents a practical step forward for improving agents at deployment time. The 11.5-point gain on WorkBuddy-Bench is meaningful, and the consistency across three benchmarks and multiple model configurations suggests the approach has broad applicability. The key appeal is simplicity: no retraining, no data collection, no new models to manage.

The cost is operational complexity. Running recursive refinement adds inference calls and latency per task. Teams will need to benchmark whether the performance gain justifies the extra compute, especially at scale. The reliance on an editor model to propose harness edits also introduces a new dependency: editor quality directly limits the ceiling of refinement.

What to watch next is whether the lessons library approach translates well to production settings where tasks vary widely, and whether teams can efficiently tune the recursion budget to balance performance gain against latency. Early adopters in professional domains like software development or legal analysis may find the highest return on investment.

Related reading

• Nemotron 3.5 Super VL 120B: Open Multimodal Agent Model
• Timestral time-series forecasting model ranks first on three benchmarks
• QwenGyre elastic RL framework trains long-horizon agents