Harvey’s Multi-Model Strategy and Legal AI Benchmarks
Despite Google’s venture capital arm GV investing in Harvey as early as July 2024, the company did not immediately integrate Google’s technology into its AI suite. Instead, Harvey’s team relied on internal benchmarking—named BigLaw—to evaluate which AI systems best excel at particular legal tasks.
Their findings revealed that no single model led across every task, prompting Harvey to leverage a variety of advanced reasoning models from different providers. By tapping into models from Google and Anthropic through platforms like Amazon’s cloud, Harvey aims to fine-tune legal AI without investing excessive resources in training proprietary systems from scratch.
Recent testing showed that seven distinct models, three of which are external to OpenAI, now outperform Harvey’s original system on the Harvey, legal AI benchmarking. For example, Google’s Gemini 2.5 Pro demonstrated high performance in legal drafting, but encountered challenges with pre-trial work such as oral arguments.
According to Harvey’s research, models like OpenAI’s o3 handled complex procedural legal analysis well, while Anthropic’s Claude 3.7 Sonnet closely followed in capability. Harvey also stated it will begin publishing a public leaderboard to track how different reasoning models perform in the legal sector.
This leaderboard will offer nuanced feedback supplied by top legal professionals, moving beyond simple numerical rankings to provide deeper insight into model strengths and weaknesses. Such transparency intensifies competition and encourages both Google and OpenAI to continually improve their offerings for law-focused AI applications.
While the landscape around AI benchmarks grows more intricate and competitive, Harvey still counts OpenAI as a valuable partner and major investor. The company’s leadership is enthusiastic about bringing greater flexibility to its clients as it integrates a broader selection of high-performing AI options, aiming to meet the evolving demands of legal professionals worldwide. Learn more on Harvey, legal AI benchmarking.
