In early March, I hosted the first episode of Voiceflow's Breaking Bots, an hour-long webinar in which conversation designers break a well-known chatbot or assistant, then rebuild it—only better—live.
Inspired by Nicolle Merrill, who broke into the field by breaking bots and writing about her experiences, the series aims to educate through the processes of deconstruction and reconstruction.
Our inaugural episode featured two remarkably talented conversation designers: Ali Pinch, Conversation Designer at one of Australia’s largest retailers, and Nicolle Merrill, Senior Conversational AI Designer at Fortune 500 FinTech company Intuit. Ali and Nicolle are both veterans in the field.
In this post, we tell the story of how Ali and Nicolle broke Nike’s bot and rebuilt it. To see the breakdown and buildup unfold in real-time, and to discover industry best practices along the way, watch the episode.
Breaking Nike’s bot
Nike’s bot is located on its Contact Us page—one of Ali’s biggest pet peeves. She prefers when chatbots are easily accessible from the homepage, as this helps reduce friction for people who need help.
People access the bot by clicking on a chat-box icon. At the time of the webinar, the only copy accompanying the icon was “24 hours a day, 7 days a week”—leaving much to be desired, according to Ali and Nicolle. From the copy, people can’t tell whether they’ll be speaking with a human or bot. The lack of copy is also considered a faux pas from an accessibility standpoint.
In reality, people can’t interact with Nike’s chatbot between 5am and 11pm. During those hours, they’re instead connected with a member of the company’s support team.
The authentication flow
Before interacting with Nike’s bot, people must enter their name and email address. They’re also asked to select different options depending on what kind of help they need. For example, people can choose to track their orders, get an update on a refund, or ask about specific products.
This authentication flow perplexed Ali and Nicolle. Virtual assistants are perfectly capable of processing customer data, so why isn’t Nike’s bot (instead of a form) collecting this information? Companies should also only need a single piece of identification to serve their customers, so why does the form ask for multiple customer data points?
Ali and Nicolle broke the bot during two journeys: one to track an order and one to ask about a product. Keep reading to discover what went wrong.
Track my order
When Nike’s bot arrives in the chat, it jumps straight to the matter at hand—helping people find their order status—without offering a welcome message.
Nicolle argues that welcome messages build trust and set the tone for the rest of the conversation. At minimum, they should inform people that they’ve reached a chatbot and share what it’s designed to help them accomplish. Ideally, they should show off the brand’s personality and guide what happens next.
Nike’s bot next asks for the customer’s name and email address, and quickly follows up by asking them to confirm the information they’ve provided.
That doesn’t make sense given that the same information was provided during the authentication flow, and then again in the chat. Also, companies should only require a single piece of identifying information. In this case, an order number would make the most sense since it’s publicly available information, according to Ali.
If the customer doesn’t remember their order number, Nike’s bot should tell them where to find it or give them another way to access their order. Instead, when the customer enters incorrect information (a no-match response), they are shown not only a fallback response but also the original prompt again. The experience is jarring and leaves people unsure how to respond.
The fallback response also instructs customers how to get help from a human. But Ali and Nicolle argue that that’s redundant because most people know how to do that already. One attendee named Daniel also pointed out that it could increase escalations—driving up the company’s support costs. Given that most companies leverage chatbots to lower costs, it’s an unusual choice.
All in all, the process of tracking an order was dragged out. A well-designed bot would have helped the customer track their order in just two steps.
Ask about a product
When people indicate they want to ask about a product, Nike’s bot first asks whether they’re searching for a new or existing product. That’s unclear at best and confusing at worst, according to Ali.
Nicolle shared that, from a brand perspective, it’s strange that the bot suddenly uses title case. Not only is it unnatural to speak to people that way, but it’s also inconsistently applied throughout the chat.
When Ali input Nike’s most in-demand product at the moment, she was transferred to a human—a big waste of money. At minimum, the bot could have directed her to a product page. If it didn’t recognize the name, it could have shown other products. It’s clear from this experience that the bot hasn’t been trained to recognize products.
Overall, Ali and Nicolle were disappointed with their experiences.
Throughout the experience
During the user journey, Ali and Nicolle experienced a slew of other challenges:
- When they first joined the chat, they were asked not to refresh and told they were in queue. That implied they were waiting for a human, not a bot. This could confuse people and make them believe they’re being connected with a human.
- On a related note, the UI showed a message suggesting Nike’s virtual agent was typing—something only a human can do. Again, this has the potential to confuse current and prospective customers.
- The language choice throughout the experience was very formulaic and inconsistent. For example, the bot sent a mix of formal and informal messages. Given Nike’s incredible brand, this missed opportunity was disappointing for Ali and Nicolle.
- When Ali and Nicolle paused for a moment to discuss their experience, they were given 90 seconds to respond before the chat timed-out. The thing is, everyone gets distracted. By being pushy, you cause feelings of panic. Ali and Nicolle also argued that there’s no reason a bot should time-out. It should infinitely wait for you.
- Ali and Nicolle also criticized the old-school UI, which they argued looked like it was from 2005. Again, Nike is a prominent brand. Why didn’t the company invest in a simple and clean UI that delights users?
- Any time Ali and Nicolle wanted to do something different from what they originally chose (e.g., ask about a product instead of tracking an order), they had to exit and reenter the chat, which creates an extra step for the user and slows them down.
- People think they can toggle back to the main menu from a drop-down menu. Unfortunately, it’s not easy to find and it doesn’t do what it suggests. Instead of bringing you back to the main menu, the bot prompts you to reconfirm what you want to do—presenting only the original prompt and an “other” option.
- Ali and Nicolle were pleased to see that people could save their chat history and refer back to it later. However, the ways people could save the chat—through a .txt file or by printing the chat—was highly unusual.
Watch the rebuild in real-time
The full episode shows Ali and Nicolle rebuilding Nike’s bot in Voiceflow. They not only talk through everything from generating utterances and leveraging entities to using logic to create different paths, but they also highlight conversation design best practices along the way.
Header photo by Terrance Barksdale.