Harvey Labs — Legal Agent Benchmark

What is Harvey Labs — Legal Agent Benchmark?

Harvey Labs, also called the Legal Agent Benchmark (LAB), is an open-source evaluation project for agents that support legal work. It combines task instructions, source documents and grading rubrics with a harness that runs agents and evaluates their deliverables. It is a benchmark and execution harness, rather than a standalone legal assistant. Use it to compare models or agent configurations on analysis, drafting, review, extraction and research assignments. The walkthrough follows an M&A data-room review; other examples include real-estate term extraction, bankruptcy drafting and NDA review. Reports expose criterion-level judgments, document coverage, token usage and runtime, while comparison dashboards help inspect results across runs. Setup requires a local development environment and model-provider API credentials. Grading uses model judges, with an all-pass score intended to highlight missed requirements. The benchmark's documents are synthetic and lawyer-reviewed, and the project notes that they contain imperfections. Results describe performance on these tasks; they are not assurance that an agent can handle a live legal matter.