Curiosity and conversation beat any best practice

There are hundreds of thousands of Google Search results for the term “conversation design best practices.”

I think. I’m actually unsure.

As of about a week ago, I do most of my searching through ChatGPT, which doesn’t reveal that data. 

And since writing this, I switched to Bing, which does. The number is 2.6 million.

But wait—I might switch to Google Bard next week, and... Okay, Bard just confessed its feelings for me and incited a blood feud with Bing over the future of my search traffic. Smiling devil horns emojis are flying, and I guess what I’m really trying to say is I appreciate the simplicity of DuckDuckGo. Don’t you? I’m told they’re working on an offline version, which’ll be great if Bing makes good on some of its threats and my next job is a desert nomad and I want to look up “the before times.” 

Maybe that was a bit much. But the point I’m trying to make is that the world of large language models (LLMs) moves hilariously quick. And so does the tangential world of conversation design (CxD). It’s so fast, and so wide-ranging a discipline, it defies fixed precepts. Most of what I see shared out there as “best practices” are really just someone’s highly situational advice, which quickly grows irrelevant.

So here’s what I’ll offer you instead. My only best practice is that there are no best practices save for one: to secret shop others’ experiences and discuss them as a team. Today, I share stories of companies that win by doing just that.

Other people’s “practices” rarely apply to you

How big is your company? Really big? Who’s your customer? All accountants, or just solo practitioners? How online are your users? What goals does your organization have in mind? What properties will your assistant appear on, and alongside what else? I could go on qualifying your situation forever.

Without clear answers to these and more, it’s unlikely anybody except a direct competitor or someone already on your team is going to pen a work-fluencing LinkedIn post that perfectly applies to you. 

Plus, how do you calculate what makes a practice “best”? Did the person who shared it explain how they measured it? Did they bother? Some teams seek to reduce call volume whereas others want good marks for a pleasant and successful dialog. Some teams have high standards and nothing’s a “practice” until it’s provable and replicable. For most others, something’s worth sharing after it worked just once

And then, all that knowledge you accumulate changes when you switch jobs or projects. Or when your tools change—like your LLM model. (Cue the present LLM clash of the titans.)

All that said, I do think there is still great value in reading and considering all this stuff because it can provide a serendipitous trickle of inspiration and cause you to stumble upon great questions. And those questions can lead to new practices—practices that are “best” only for you. 

Far better to secret shop relentlessly and discuss

There is no way around just doing the work—donning headphones and trying as many experiences as possible—known as “secret shopping the competition.” And I don’t mean direct competition. We all know that if you’re a car manufacturer, consumers also now compare you to Lyft, Geico, and Waze. Your competition is anything that can be tapped or talked to.

Go out and “no match” every assistant you can. When you tire of Alexa, Siri, and Google Assistant, ask for help from all the services you pay for. I recently asked Gina Riley, VP of User Experience and Design Lead at JPMorgan Chase, who is the champion of doing this. Here’s what she’s been inspired by lately: 

  • KLM. “The airline KLM’s chat takes your utterance and immediately transfers you to SMS text,” says Gina. “I think that’s a wonderful experience because you don’t have to stay in that channel. You can go anywhere, do anything. I think it’s where the world is headed.”
  • Apple’s IVR. “Apple’s IVR is really intuitive and smart. When I tested it, I expected to ‘no match’ a whole bunch of times, but I only no-matched once,” she says. “Its grasp of natural language is great. Basically anything you say is recognized, and that’s an experience to study.”
  • Spectrum. “What I love about Spectrum’s assistant is if you leave it and come back a day or two later, it’ll say, ‘Welcome back. Want to continue?’ Very few assistants hold that sort of context,” she says.

You can also shop for feedback within your own organization, as I wrote about recently here on Pathways: At PayPal, customer success advocates join their assistant design reviews. They weave those support people’s experiences—basically, everything that happens after the assistant—back into the build, in a loop.

Take all of those ideas and experiences and feedback and bring those back to your team to discuss. I won’t tell you how many experiences to review, or how exactly to discuss them (no best practice here!) but here’s what’s important about those conversations: 

1. It’s everyone’s job to collect and share novel experiences

The richness and variety of your inspiration determines the quality of your creation. Ask friends, family, and peers about their assistant interactions. Get everyone on your team looking for new ideas.

2. Everyone should allow their personal preferences to bleed into those choices

When people think in terms of what the business needs, the creative part of their brain sometimes shuts off. They filter themselves. Whereas if you treat this like an improv exercise and welcome everyone’s weird, you get exciting ideas that ricochet, accelerate, and ping-pong to life.

For example, seeing another business display “wait time” is meh. Seeing that the customer support people at Games Workshop, maker of the set-top game Warhammer, include pictures of their hand-built figurines in their email signatures begs the question, “Could our bot have more personality?”

3. Hold a total teardown where everyone tells you why it would work before you discuss why it wouldn’t

The improv principle applies—it’s “Yes, and” first—otherwise people shut down and you never get anywhere. Gather everything, distill, then critique. (Like the Design Council’s Double Diamond process.)

4. Prioritize a few winning ideas to go back and test some more

Novel experiences are just the jumping-off point. Once you’ve found a relevant application for them, go back and test some more. Then prototype in a visual tool that lets you show something realistic enough to get good feedback on.

This process may not sound fancy, or like “thought leadership.” But this is the cycle that Gina and countless other experts rely on in the real world.

“Good” is subjective, but conversations lead you there

“Success” is such an ephemeral, conditional thing, and far be it from me to tell you how to get there. That’s why I’m doubtful there is some eternal set of best practices worth sharing, without also providing copious context and testing to confirm they are indeed worthwhile.

So, if you also find you’re having trouble applying others’ practices as your world changes rapidly, don’t worry. It’s not just you. Just get really good at discussions. 

In other words, don’t just be a conversation designer. Be a curious, conversational one. 

What’s that? Yes, Bing, getting good at telling stories would also be useful for meeting other desert nomads after an apocalypse. 


What Nike's chatbot taught us about conversation design best practices

No items found.