NLU Management

How to manage your Assistant's NLU data.

Model Optimization

How to optimize your Assistant's NLU Model data

Overview

In the NLU Manager view, you will find a couple of additional tools that can be used to help optimize your NLU model's data and performance. As you design your Assistant and expand the NLU data set needed for it, you can improve the overall performance of your NLU.

Intent Confidence

The first score that you will see applied to each Intent in your model is Confidence. This is a measure of how much training data you have provided for this specific intent.

In general, most NLU platforms recommend that you provide a minimum of 6-12 utterances on an Intent to properly train it in the NLU Model. At a glance, your Intent Confidence score can tell you which of your intents require more training data to be properly trained.

Another key factor in evaluating whether each intent has sufficient training data is to ensure that all intents in your model have an equivalent volume of utterances. You can find the number of utterances on each intent listed on the table in the NLU Manager. You can sort by this column to see which of your intents might be over or under-trained.

Clicking on an intent will open up the intent editor, in which you will see a card that gives you a more detailed Confidence Score. As you added more utterances to your intent, this measure will update.

Utterance Recommendations

If you are stuck trying to come up with new utterances for an intent, you can leverage our ML-generated utterance recommendations tool. This will provide suggestions for additional utterances to add to your model based on the utterances you already have included on your intent.

To add an utterance to your intent, simply click the check next to it. If there is a recommendation we have suggested that doesn't make sense, please check the X next to it to provide feedback to improve our system.

You can refresh the set of recommendations with the reload icon in the Recommendations header.

Intent Clarity Scoring

The second measure you will find on your Intents is the Clarity Score. This is a measure of how unique each intent's utterances are from the other intents in your model. This is important for you to understand in order to continually optimize the performance of your NLU model as you add new Intents and subject matter for your Assistant to handle. This will help ensure that the right intents get triggered at the right time in your assistant.

The way that your intents' Clarity Score is evaluates is our ML model will evaluate how similar each utterance on the intent is to the utterances on every other intent. If utterances are two similar, it will flag that conflict, and low your intent's Clarity Score.

Clicking on an intent will open up the intent editor, in which you will see a card that gives you a more detailed Clarity Score, including the number of conflicting utterances you have.

View and Manage Conflicts

In the Intent editor in your NLU Manager view, you will find a 'View Conflicts' button if conflicts do exist for that intent.

On the left of this view you will find the utterance for this intent that conflict with another intent. On the right you will see the utterances that in conflict.

In order to resolve these conflicts, you can either:

  • Re-assign an utterance to a different intent by click-and-dragging it
  • Delete an utterance from an intent

Once you are done resolving all the utterance conflicts you think are required, you can hit the 'Apply Changes' button at the bottom of this view to apply them. Once they have been applied, your Clarity Score will be re-calculated for that intent.

Other docs in this section