How We Built AI-Powered ChatBook Tools for the American Arbitration Association
The story behind the AAA's AI-powered ChatBook suite — from architecture decisions to source grounding, and what we learned about building AI that arbitration practitioners trust.
The brief
The American Arbitration Association is the leading global provider of arbitration, mediation, and ADR services. Through its AAAi innovation lab, the organisation has been investing in technology to make its resources more accessible and useful to practitioners.
The challenge they brought to us: take trusted arbitration publications — books, rules, procedural guides — and make them accessible through AI-powered tools where practitioners can ask questions in natural language and get reliable, cited answers. Not a generic chatbot. Not a search engine with a conversational interface. Something that understands arbitration, respects the source material, and earns the trust of professionals who spend their careers in this field.
What we built
The result is a suite of ChatBook tools, each tailored to a different audience and use case.
Case Prep & Presentation draws from “Case Preparation and Presentation: A Guide for Arbitration Advocates and Arbitrators” by Jay E. Grenig and Rocco M. Scanza, along with relevant AAA-ICDR rules. It covers every stage of the arbitration process — from drafting clauses to navigating post-award steps. Practitioners can ask specific questions and get answers grounded in these trusted sources.
Non-Attorney Guide makes the same knowledge accessible to parties representing themselves in arbitration. This is an important accessibility application — pro se parties often find arbitration procedures intimidating and opaque. The tool translates complex procedural requirements into language anyone can follow, while maintaining accuracy.
Labor Arbitration serves the labor arbitration community specifically, with content tailored to that practice area.
All three tools share a common technical foundation but differ in their source material, audience assumptions, and response style.
The architecture decisions that mattered most
The single most important decision was source grounding. Rather than training on general legal knowledge or retrieving from broad databases, each ChatBook tool is grounded in specific, curated publications. This means every answer traces back to content that the AAA has reviewed and trusts. The AI doesn’t make things up from general knowledge — it finds relevant passages in the source material and synthesises answers from those passages.
This approach sacrifices breadth for reliability. If a question falls outside the scope of the grounding material, the system says so rather than improvising. In our experience, legal professionals vastly prefer a system that says “I don’t have information on that” over one that gives a plausible-sounding but unverifiable answer.
The second important decision was chunking strategy. Arbitration publications have dense cross-referencing — a section on evidence might reference procedural rules, ethical guidelines, and prior AAA publications in a single paragraph. We used section-aware chunking that preserves these relationships, with citation metadata that lets the retrieval system pull in related context.
The third was response format. Each answer includes not just the synthesised response but the specific sources it draws from — page references, section numbers, rule citations. Users can verify everything independently. This transparency isn’t just a feature; it’s the foundation of trust.
“Working with the AAA taught us something we now apply to every legal AI project: the AI doesn’t need to be brilliant. It needs to be honest. An honest AI that clearly states what it knows, what it doesn’t, and where its answers come from is infinitely more useful in legal practice than a brilliant one that occasionally makes things up.”
What we learned
The most interesting lesson was about user behaviour. We expected practitioners to ask complex, multi-part legal questions. In practice, many queries are surprisingly specific and practical: “What’s the deadline for filing a counterclaim?” or “Can I submit evidence after the hearing?” These factual, procedural questions are exactly where a grounded AI tool excels — the answers are definitive, the sources are clear, and the value to the user is immediate.
Another lesson: the non-attorney tool got a different kind of engagement than we expected. Pro se parties don’t just ask about procedures — they ask about what things mean. “What does ‘arbitrability’ mean?” or “What’s the difference between mediation and arbitration?” The tool serves as both a research companion and an educational resource, which expands its utility beyond what we initially scoped.
On the technical side, we confirmed something we’d suspected from earlier projects: the quality of the ingestion pipeline correlates directly with the quality of search results. The time we invested in properly extracting, structuring, and indexing the source material paid dividends throughout the project. Every shortcut in ingestion becomes a bug in search.
The broader pattern
The AAA ChatBook project represents a pattern we’ve now seen work across multiple contexts: take authoritative, curated content that an organisation already owns; make it accessible through AI-powered natural language search with full source transparency; and deliver it in a format tailored to the specific audience.
This pattern works for legal publishers, professional associations, regulatory bodies, educational institutions — anyone sitting on valuable content that’s currently underutilised because it’s locked in PDFs or behind search interfaces that don’t understand how people actually ask questions.
The technology isn’t the hard part. The hard part is doing it with the care and accuracy that professional audiences demand. That’s what we’ve learned to do.
Have authoritative content you want to make more accessible? Contact us — we’ll show you how we built the AAA ChatBook tools and discuss what’s possible for your organisation.