Understanding QA Scores and Scorecards

5.1 How Scores Are Calculated

Parameter Types

QA parameters have three types, each producing a different evaluation output:

Untyped (legacy) parameters are treated as Boolean. The rule type is set when the parameter is configured and cannot be changed after creation.

Weighted Score Formula

The QA score is computed from Boolean parameters only:

Score = (Sum of passed Boolean parameter weights / Sum of applicable Boolean parameter weights) x 100

In plain terms:

• Every Boolean parameter has a weight reflecting its importance. Higher-weight parameters have a bigger impact on the score.
• If you pass a Boolean parameter, its full weight counts toward the score.
• If you fail a Boolean parameter, it contributes 0.
• If a Boolean parameter is N/A, it is excluded entirely and does not affect the score.
• Categorical and Descriptive parameters are always excluded from the score calculation.
• Matching between criteria names and rule tags is case-sensitive exact match.

Provisional Scores and Manual Review

Each rule evaluation reports a confidence score (0 to 1) and a data sufficiency flag. Agent QA uses these to decide whether a result needs a human reviewer before it counts as final.

A rule result is flagged for review when its confidence falls below the persona's review threshold, or when the transcript did not contain enough data to support a verdict and the "flag on insufficient data" setting is on. Flagged results appear in the review queue on the Audit tab. Every rule type (Boolean, Categorical, and Descriptive) can be flagged.

Only Boolean results affect the QA score, so flagging behaves differently by type:

• A flagged Boolean result still counts in the score using the model's answer while it awaits review. The score is marked provisional (shown with a dashed border) because a reviewer's decision can change it.
• A flagged Categorical or Descriptive result enters the review queue but does not make the score provisional, because those types never feed the score.
• If every scoreable Boolean result on an interaction is flagged, the interaction has no committed score until at least one flag is resolved.

The confidence score is displayed in the Audit Scorecard as a bucket: High (0.85 and above), Medium (0.70 to under 0.85), or Low (under 0.70). Hovering the bucket shows the exact value.

Reviewer Decisions

A reviewer can act on each flagged result from the interaction drawer:

• Boolean results: Confirm the model's answer, Override it to Pass or Fail, or set it to Not Applicable.
• Categorical results: Confirm the model's category, pick a different one from the rule's option set, or propose a new category.
• Descriptive results: Accept the model's text or rewrite it.

A decision resolves the flag and can change the QA score:

• Confirm keeps the model's answer; the score stays the same.
• Override replaces the model's answer; the score is recomputed with the reviewer's verdict.
• Not Applicable override removes the rule from the score calculation entirely, the same as an N/A result.

A reviewer can change a decision at any time. The newest decision always wins, and the score updates immediately. Once all flags on an interaction are resolved, the score loses the provisional marker and is committed.

Worked Example

• Passed weight: 4 + 10 = 14
• Total applicable weight: 4 + 6 + 10 = 20 (Hold Time excluded because N/A; Categorical and Descriptive parameters excluded from score)
• Score: 14 / 20 x 100 = 70

Edge Cases

5.2 Score Ranges

5.3 Critical Parameters and Auto-Fail

Parameters marked as Critical represent non-negotiable quality standards (e.g., regulatory compliance, inappropriate behavior). Only Boolean parameters can be marked Critical. If any critical parameter fails, the entire interaction scores 0, regardless of how well everything else went.

To find which parameter caused the auto-fail:

1. Open the interaction details
2. Go to the Scorecard tab
3. Look for the parameter marked Failed with a Critical indicator
4. Read the Rationale to understand the specific issue
5. Click the "Source" accordion to see the transcript evidence

5.4 Parameter Weights

Each Boolean parameter has a weight (must be > 0) that determines its relative importance in scoring. You configure weights per organization, either generated by Auto-Learning based on your QA form or set manually during parameter configuration. Categorical and Descriptive parameters do not have weights because they are not included in the score calculation.

Weights are relative, not absolute percentages. The score formula normalizes based on the sum of applicable Boolean weights, so the raw total does not need to equal 100.

5.5 What N/A Means

Sometimes a Boolean parameter is marked N/A (Not Applicable). This means the rule did not apply to that particular call. Common examples:

• Hold time evaluation when the agent did not put the customer on hold
• Upsell evaluation when the call type did not involve a sales opportunity
• Greeting evaluation when the recording started mid-conversation

N/A does NOT count against you. These parameters are removed from the calculation entirely.