Vincent AI Meets or Exceeds Human Benchmarks in Legal GenAI Benchmarking Study

Vals AI study reveals Vincent AI has finished ahead or within the margin of error of the lawyer baseline in 4 out of 5 tasks.

Miami, FL (February 27, 2025) - Global legal intelligence company vLex announced today that its award-winning legal workflow platform Vincent AI has achieved a groundbreaking milestone, meeting or surpassing human lawyer benchmarks in four out of five key tasks in the first major legal GenAI benchmarking study. The study, conducted by Vals AI, evaluated Vincent AI's performance across multiple areas, including Data Extraction, Document Q&A, Document Summarization, and Transcript Analysis, with results showing Vincent AI finishing ahead or within the margin of error of its human lawyer counterparts.

The Vals Legal AI Report focused on document extraction tasks, which didn’t involve tasks that required a database of law. “We are quite pleased with Vincent AI’s performance, especially considering that document summary tasks do not leverage our vast legal database of global information,” said Ed Walters, Chief Strategy Officer of vLex.

The report noted that “Although our evaluation focused on a small slice of Vincent AI’s capabilities in U.S. jurisdictions, its support for international matters is a significant strength. For global law firms, this capability may provide a level of utility unmatched by other tools, making Vincent AI an attractive choice.” In fact, in the recent “AI Smackdown,” designed to test how AI-driven research platforms provide direct answers to legal questions, the Southern California Association of Law Libraries found that vLex provided the most depth in challenging AI tasks.

Other takeaways from the Vals Legal AI Report include:

• Vincent “gave responses exceptionally quickly as generally one of the fastest products we evaluated.”
• “Vincent AI’s design is particularly noteworthy for its ability to infer the appropriate subskill to execute based on the user’s question, adapting to the user query. In cases where clarification was needed, Vincent AI would proactively ask follow-up questions to refine its understanding, ensuring tailored responses.”
• “When the legal research database did not have sufficient data to answer a question, Vincent AI refused to answer, rather than hallucinate an illegitimate response.”
• “The answers provided were impressively thorough . . . offering valuable additional context to aid their understanding and workflow.”

“Results are promising and reveal that AI does indeed deliver value in the context of legal work,” the Vals study concluded. “The AI tools were six times faster than the lawyers at the lowest end, and 80 times faster at the highest end.”

“The Vals Legal Report is great validation for the first generation of legal AI tools,” said Walters. “The results are promising and reveal that AI does indeed deliver value in the context of legal work.”

About vLex

vLex is a global legal intelligence and legal AI pioneer empowering legal professionals to work smarter. vLex has united industry leaders Fastcase, Docket Alarm, and NextChapter to transform AI-driven legal services and provide the most extensive collection of legal and regulatory information worldwide, all on one platform. Trusted by more than two million lawyers, law librarians, researchers, government departments, and law schools worldwide, vLex offers comprehensive and intuitive access to the law in more than 200 jurisdictions through an intelligent, AI-powered legal research platform.

With nearly three decades of experience in legal tech and access to over one billion legal documents, vLex developed Vincent AI - a comprehensive AI-driven workflow platform that addresses the challenges faced by modern legal professionals. Unlike general-purpose AI tools prone to hallucination, Vincent AI is powered by vLex's vast library of structured legal data, providing added confidence with citations to verified authoritative sources. Vincent AI’s curated, pre-built workflows have each been carefully crafted to help you accomplish tedious tasks, work smarter, and do more with less. Learn more at vlex.com.

Share this post