How to Optimize AI for Accuracy and Efficiency

AI is everywhere in the contact center right now. But in my conversations with customers, one question keeps coming up: what happens after you go live? 

That’s exactly what we tackled in Part 3 of our AI Playbook webinar series. I had the chance to sit down with two of our data science experts, Caterina Bonan, Manager of Services Data Sciences, and Lucia Pozzan, Senior Director of AI & Custom Services, to talk about what it really takes to make contact center AI work over time. 

Because deployment is only the beginning. 

If you missed the session, you can watch it on demand. Here are a few of the insights that stood out from our conversation. 

Contact center AI is never “done” 

One of the first things Caterina and Lucia emphasized is something we don’t talk about enough: contact center AI isn’t a one-and-done project. 

As Caterina explained during the session: 

“AI agents and more generally artificial intelligence are living, breathing, and ever-changing entities that require continuous nurturing and monitoring.”  

That perspective resonated deeply with me. It underscores that sustained improvement in AI is intentional, not automatic. AI needs ongoing attention. 

Your environment changes. Your models evolve. Your customers ask new questions in new ways. And all of that impacts how your contact center AI performs. 

Performance doesn’t break — it drifts  

One of the most interesting parts of our discussion was how AI performance actually changes over time. 

It doesn’t usually break overnight. It drifts.  

Maybe routing isn’t as accurate as it was last month. Maybe intent recognition starts to slip. Maybe conversations feel just slightly off. That drift is often driven by constant change across the ecosystem. The tech stack evolves. 

Lucia and Caterina pointed out that even backend updates, such as changes to APIs, natural language models, or speech-to-text engines, can have a real impact. 

“Any change that is going to be pushed in the background… has the potential to impact our AI agent and thus requires expert monitoring.”  

At the same time, the business does not stand still. New products launch while others are retired. Teams restructure, and responsibilities shift. Routing logic needs to continuously adapt to reflect these changes. 

Customer behavior evolves just as quickly. Seasonality influences demand. New offerings change what customers care about. What users ask changes over time. How they ask it changes, too. That’s why strong contact center AI strategies don’t stop at launch. They plan for what comes next - how to monitor performance, catch issues early, and keep improving. 

AI in customer service still needs human guidance 

Another theme discussed in our conversation is the misconception that AI can run entirely on its own. 

We hear this a lot, especially as AI becomes more advanced. 

But as Caterina said, “One might think that AI agents… are overly smart technologies that learn by themselves. This is another common misconception.”  

That’s especially important when thinking about AI in customer service. The goal isn’t to remove people from the equation; it’s to make them more effective. AI can automate, analyze, and assist. But it still needs structure, feedback, and oversight. And critically, it needs to be measured. 

You can’t improve what you don’t measure. The most effective AI programs are deeply data-informed, with monitoring built into every layer of operation. 

It starts with business metrics and KPIs, defined in partnership with the customer to ensure outcomes align with real business goals. These include containment and abandonment rates, average handle time, transfer rates and first call resolution. 

Next comes the customer experience itself. Teams analyze conversation paths, track customer satisfaction scores, and review sampled interactions to understand how journeys are actually unfolding. 

Finally, there’s the AI layer. Standard NLP metrics such as speech-to-text accuracy, intent-matching accuracy, no-match rates, and confidence scores provide visibility into how well the system is interpreting and responding. 

That’s where solutions like Five9 AI Agents and Conversational AI come into play. They help teams build smarter interactions, but the real value comes when those interactions are continuously refined. 

And when you get that balance right, the impact on agents is significant. They spend less time on repetitive work and more time focused on meaningful conversations. If you’re thinking about that shift, this post on improving agent productivity and efficiency is a helpful resource. 

This is where contact center AI delivers real value 

What I took away from this session is simple: getting AI live is important, but it’s not what drives long-term success. 

That comes from what you do after. 

When you treat contact center AI as something that evolves and something you actively monitor and improve, you start to see real impact. Conversations become more accurate. Customer journeys feel more connected. And agents are better equipped to do their jobs. 

That’s when AI in customer service moves beyond experimentation and starts delivering measurable business outcomes. 

Build your AI Playbook for what comes next 

This session is part of a larger conversation we’ve been having through our AI Playbook series: how to approach AI not just as a technology, but as a capability you build over time. 

If you’re earlier in your journey or looking to refine what you’ve already deployed, I’d recommend starting with the AI Blueprint for Contact Center Readiness. 

It’s designed to help you think through how to define the right use cases, align AI with your CX strategy, and create a foundation for continuous improvement. 

Because in The New CX, success with contact center AI isn’t defined by launch day. 

It’s defined by how well it performs six months later. 

Watch the full webinar on demand 

If you want to hear the full conversation with Caterina and Lucia, including how they approach monitoring, evaluation, and ongoing optimization, you can watch the webinar on demand. 

Because the difference between average and exceptional AI in customer service isn’t just the technology you choose. 

It’s how you manage it over time.