What is Automated QA in OI?

OI analyzes interactions against predefined quality criteria, generating reports that highlight trends, common issues, and agent performance. 

Managers can easily access and review these insights.

• Assessments are based solely on transcript content - no external data is referenced.
• Criteria apply to 100% of interactions within the model in scope.
• ‘N/A’ scoring is used only when clear conditions define when a topic is applicable.

Key OI QA Concepts

Event: Did something happen?

A specific action, phrase, or step in the conversation. Events are clear, discrete (singular) objective and observable—either they happened or they didn’t.

Condition: We care when or where something happens

A condition makes an event necessary. Think of it like a trigger. If the condition doesn’t occur, the event is not important.

Outcome: Was the goal achieved? 

Outcomes are useful for more ‘fuzzy’ assessments which can be more subjective or context dependent. 

• The path to the outcome can also vary significantly.
• Outcomes can have generally acceptable definitions, like empathy or non-technical language (for example).
   

Access the QA Page

• Go to the OI dashboard (Clicking the “OI” icon in the top right corner of any page)
• Click the QA tile as seen below:

That will bring you to the main QA Dashboard. Here you can see all the QA topics you have active with top-level details about the topic. 

We display an average and a median score, most of the time they will be the same but if there is a variance this indicates that there is an outlier that is skewing the data up or down. For QA scores, we show both the average and median scores. This is to help control for skewed data when interpreting the results.

• Average is sensitive to outliers (unusually high or low values).
• Median is resistant to outliers and better reflects the "typical" value in a skewed distribution.

To interpret the difference between average and median:

• If average = median: The data is likely symmetrically distributed.
• If average > median: Some very high scores are increasing the average.
• If average < median: Some very low scores are decreasing the average.

Understanding both the average and the median helps identify whether performance is generally consistent or uneven across the dataset.

Using only the average can mask issues (e.g. a few very high scores compensating for poor performers). Including the median ensures that typical performance is also considered, which can support fairer assessments, incentives, or interventions.

You can further drill into a metric by clicking on the title box. From here you can see a more detailed breakdown of the data. There will be a line chart that displays the data points on a day/week/month basis. 

Further down the pages you get more information and factors that are influencing the score.

Impact by inquiry/ root cause 

Here you can view what inquires/root causes are positively or negatively impacting the score.

Score by Inquiry 

This section breaks down the scores by inquiry/root cause. You can also use the checkbox to show the values on the chart and adjust the sort. 

Score by site 

This section breaks down the scores by site or queue. 

Score by Agent 

This section breaks down the score by agent or team. 

Dig deeper into the Metrics page (Filtering) 

As always, on the side bar you can add in any filtering you need.

Data Sample: Explore recent interactions

At the very bottom there is a section that displays the 100 most recent interactions. Here you can review transcripts and export the data out to excel. Use filtering here to be able to change the data in the view. 

Flagging

If you locate an example that does not reflect the correct score, flag it! Enter your justification (optional) for the dispute and click submit.