legal-specific vendor test · Vals AI
Vals Legal AI Report (VLAIR): Feb 2025 edition, legal tasks
published 2025-02-27 · read 2026-10-22
Four tools were tested on seven tasks against a lawyer baseline: CoCounsel, Vincent AI, Harvey Assistant and Oliver (Lexis+ AI withdrew from the sections studied). Harvey took the top score on five of the six tasks it entered, including Document Q&A at 94.8% and Chronology Generation at 80.2%; CoCounsel scored 89.6% on Document Q&A and 77.2% on Document Summarization, the top score there. The lawyer baseline beat the AI tools on EDGAR research and on Redlining (79.7%).
Caveat. Vendors opted in task by task (Harvey six of seven, CoCounsel four), so tools were not compared on the same tasks; 2025 versions, over a year old; not a ranking of all legal AI.
Names: Harvey Assistant, CoCounsel, Vincent AI (vLex), Oliver (Vecflow, now August), Lexis+ AI (withdrew).
https://www.vals.ai/industry-reports/vlair
legal-specific vendor test · Harvey (benchmark) / Mercor (leaderboard)
Harvey Legal Agent Benchmark (LAB) leaderboard, hosted by Mercor
published 2026-05-06 (launch per Harvey blog coverage); leaderboard date not shown on page · read 2026-10-22
Open-source benchmark of 1,749 tasks across 24 practice areas plus in-house contracting, graded with all-pass rubrics (100,000+ criteria). Pass@1 top 5 on the page: Gemini 4 Argon High 27.4% (±5.4), Grok 4.7 xHigh 19.5% (±4.5), Muse Spark 1.2 xHigh 19.4% (±4.6), Claude Opus 5 Max 17.8% (±1.5), Claude Fable 5.1 Max 17.4% (±4.2).
Caveat. Built by Harvey (a vendor); the leaderboard lists foundation models, not Harvey's own product; leaderboard publication date not shown and the top-5 error bars overlap heavily; the leaderboard page counts 1,749 tasks versus 'more than 1,200' reported at launch, so the set has grown.
Names: Gemini 4 Argon, Grok 4.7, Muse Spark 1.2, Claude Opus 5, Claude Fable 5.1.
https://www.mercor.com/apex/partner-benchmarks/harvey-lab-leaderboard/