2026-10-21
It is interesting that at least one AI benchmarking website includes tax as a benchmark. According to vals.ai/benchmarks/tax_agent_bench, the highest ranking model, Claude Fable 5.1, gets a tax accuracy score of 49% (as of October 2026). Accuracy here means the share of questions answered fully correctly.
According to vals.ai, the results come from 193 private test questions:
The questions center on the work of an enterprise tax department: federal corporate income tax (roughly 60% of the test set), with the remainder spanning partnership tax, state and local tax (SALT), transfer pricing, and corporate employment tax. Fact patterns involve C corporations, consolidated groups, reorganizations, reportable-transaction disclosure, penalties and procedure, and multi-year or multi-jurisdiction comparisons. * * *
All questions, gold-standard answers, and rubrics are authored and reviewed by tax professionals.
I have to admit that I now sometimes use AI models to supplement my tax research. There may be an exception to an exception that I just forgot about. Asking an AI for help can surface issues that didn't initially occur to me, as long as I'm careful not to disclose any confidential taxpayer information. For lawyers, ABA Formal Opinion 512 (2024) addresses the use of generative AI tools. I pay for Claude Pro ($20 per month), which includes Claude Code. I usually use Opus 5.5, and it does pretty well.
How can a model do "pretty well" when the best model scores only 49%? It depends on how you use it. I treat the AI as an issue-spotter, not as the source of a final answer. A 49% fully-correct score is not something to rely on. You still have to use your own skill and judgment when you analyze what the AI tells you.
[As an example of issue spotting, I used Claude to review this article after I drafted it. Claude suggested that I indicate that the benchmark was as of October 2026, and it suggested I may want to reference ABA Formal Opinion 512. I thought they were both good calls, so I included them.]