Building AI Inside a Health Insurer: What You Can't Use Shapes What You Build

About this session

I regularly work in the data and analytics space. Getting an answer meant going through a database or digging into spreadsheets - slow, and not something everyone could do themselves.

So I built a prototype RAG-based bot on top of existing data infrastructure, using a local LLM through Ollama. Instead of writing a query, people could just ask a question in plain English and get an answer pulled directly from real data. The prototype has handled approximately 50 queries across limited users, that self-serve access to this data was something people actually wanted, once it existed.

Here's the part most "I built an AI tool" talks skip: I wanted to use a tool like Claude directly, but couldn't get it approved in a regulated health insurance environment. That access obstacle didn't kill the project — it shaped it. It's why I built locally instead of relying on a hosted assistant, and why explainability and control mattered more than raw model power.

This talk walks through that build: the architecture decisions driven by constraint, what changed once people could self-serve instead of asking around, and what it actually takes to ship useful AI when "just use the best tool" isn't an option.

Speaker

Key takeaways

  • A pattern for layering AI onto infrastructure you already have - instead of treating "add AI" as a rip-and-replace project, attendees see how to wrap a RAG layer over an existing database or spreadsheet workflow with minimal disruption
  • A concrete framework for choosing your AI stack under constraint -when your first-choice tool gets blocked (compliance, cost, access), the real decision isn't "what's most powerful," it's data sensitivity vs. explainability vs. capability and how to weigh those three against each other
  • The real question for whether an internal AI tool is working - not "did people try it," but "did it change what they do by default" and why that's a better signal than usage stats alone

Related sessions