Building your AI centre of excellence

“You don’t need to hire”

“You don’t need to build”

“We’ll handle everything”

If you're evaluating AI agent vendors right now, you're probably hearing some version of this pitch.

It's appealing. Especially when you're under pressure to ship something fast.

But before you sign, it's worth thinking carefully about what you're optimizing for. Is it speed to launch? Specific resolution outcomes? 

Or is it long-term capability?

The companies that will win with AI agents aren't the ones who deploy the fastest or get quick wins on their resolution rates. They're the ones who refuse to outsource the knowledge and instead build the internal muscle to design, evaluate, and improve agents over time. 

History repeats itself

This isn't the first time companies have faced this choice.

In 2001, Borders (the second-largest bookstore chain in America) partnered with Amazon to handle their e-commerce. Amazon managed inventory, fulfillment, customer service. Borders got a cut of sales. Clean deal.

But as CIO reported, "The wealth of intellectual capital and hard-earned lessons that Amazon realized over the years from selling to Borders' online customers stayed inside Amazon's Seattle headquarters. All Borders got out of the deal was a percentage of sales."

Toys R Us made a similar bet around the same time. Both companies outsourced a capability that turned out to be strategic. Both paid the price.

The lesson isn't "don't work with vendors." The lesson is: understand what you're giving up when you outsource capability entirely, and make sure you're building internal expertise alongside any external partnership.That’s where you’ll uncover what you give up by outsourcing your AI agents.

Don’t fall for the sales pitch

Not all vendor relationships are created equal. Some give you tools and leave you in control. Others take the wheel entirely. It’s a give and take relationship; here’s how to figure out how much you’re giving:

How much can you actually customize?

Many vendors have their own frameworks, their own orchestration logic, their own model routing. They repeat this structure across clients. That's not inherently bad, but ask: how much of this can we change? Can we adjust the conversation architecture? Can we modify how decisions are made? Or are we fitting our business into their mold? Challenge the notion of whether being able to configure their system is truly customization for your solution. 

Can you bring your own models?

Some vendors lock you into their model selection, and with it, their own agenda for margins on model usage. Others let you choose: fast models for voice, reasoning-heavy models for complex queries, or your existing enterprise LLM agreements. Model flexibility matters, especially as the landscape keeps shifting.

How is success defined?

If a vendor charges per resolution, ask: who defines what counts as resolved? What visibility do you have into how interactions are being handled? Pricing models shape incentives, so make sure you understand what behaviors they're incentivizing.

What do you own if you leave?

If you decide to switch vendors in two years, what do you take with you? Your prompts? Your conversation designs? Your training data? If the answer is "not much," you're not building an asset. You're renting one.

What insights do you get back?

Every customer conversation generates data about what people need, how they ask for it, where they get stuck. Make sure you understand what reporting and insights you'll have access to, and whether you can use that data to improve your agents over time.

What real internal expertise actually looks like

Expertise isn't necessarily a big team. It's a capability. It means having people internally who understand how your agents work and can improve them without being entirely dependent on a vendor. With this capability grown in-house, you better position your team to deeply understand your problem and your solution, unlocking real value over time. 

Core skills you need:

• Conversation design: Understanding how to structure agent interactions, handle edge cases, and design for real user behavior. This is harder than it looks.
• Model evaluation: Knowing how to test different models, understand their tradeoffs, and select the right one for each use case.
• Prompt engineering: The ability to iterate on prompts, test variations, and optimize for quality.
• Integration and engineering: Agents that can't take action aren't very useful. You need people who can connect agents to your backend systems, APIs, databases, and business logic. This is often where the real value gets unlocked.
• Analytics and measurement: Defining what success looks like with your CX and product experts, tracking it, and using data to drive improvement. Using your team’s deep understanding of what success looks like allows you to tailor your analytics to your exact needs.

The best part is: you don't need a huge team to start. 

One of the most impressive teams I've seen using Voiceflow is just two people. They manage agent volume for two internet providers. They're not managing a vendor relationship. They're building real expertise, and it shows.

Working patterns that help:

• Regular reviews of agent performance with clear metrics
• A feedback loop from customer service teams to the agent team
• Structured testing before rolling out changes
• Documentation of what's working and what isn't

The build vs. buy false dichotomy

This isn't about building everything yourself or outsourcing everything. It's about being intentional.

Use vendors and platforms for tooling. Use them to accelerate. But make sure your team is developing the judgment to make decisions: which models to use, how to structure conversations, what tradeoffs to make.

There are platforms (cough Voiceflow) that give you powerful tools without taking away control. That's the sweet spot: leverage external capability while building internal expertise.

The companies that treat agent development as a core competency, not a procurement decision, will have a significant advantage. They'll iterate faster, adapt to new models quicker, and build experiences that actually get them results.

You don’t have to figure out your AI solution alone. Join our webinar on April 29th to learn why a product mindset wins in the era of AI.