Feedback and Continuous Learning

Continuous Learning improves evaluation accuracy over time through reviewer feedback. Unlike Auto-Learning (which uses a batch of golden data during initial setup), Continuous Learning incorporates corrections incrementally as your team reviews evaluations.

8.1 Submitting Feedback

Permission required: CAN_SUBMIT_FEEDBACK (Metrics Viewer, SelfMetrics Viewer for their own interactions, Manager, or Admin)

1. In the Audit tab, click View Details on an interaction
2. In the Scorecard, find the parameter you want to comment on
3. Click the thumbs down icon (disagree with evaluation) or thumbs up icon (agree)
4. Enter a feedback comment explaining your assessment
5. Click Submit

What Makes Feedback Effective

Good example: "The agent did say 'Team Portugal' at 0:15, which should count as company identification. Ensure that any variation of the company name in the opening counts toward this rule."

Rules for feedback management:

• Only the feedback creator can update or delete their own feedback
• Updating feedback resets its status to unprocessed, requiring re-moderation

8.2 Moderating Feedback

Permission required: CAN_APPROVE_FEEDBACK (Manager or Admin)

1. Navigate to the Audit tab and open the Accepted Feedback view
2. Review pending (unprocessed) feedback items
3. For each item: read the original evaluation result and rationale, read the feedback comment, check the transcript for context
4. Click Accept to approve, or Reject to ignore

Feedback Lifecycle

Each approved feedback interaction includes a deeplink that navigates directly to the source interaction in the Audit tab.

8.3 Generating Improvement Suggestions

Permission required: Manager or Admin.

Once feedback has been approved for a rule, you can ask Ema to suggest an updated rule instruction directly from the Accepted Feedback view:

1. Open the Accepted Feedback view. Rules with approved feedback are listed as cards, each showing the rule name, category, description, and a feedback-count badge.
2. Click a rule's card to open the Feedback Selection Drawer, listing every approved feedback item for that rule as a checkbox (all pre-selected by default — deselect any you don't want to include).
3. Click Suggest improvements.

Ema analyzes the selected feedback and generates an updated rule instruction. A Suggested Improvements modal opens immediately with a loading state ("Generating suggestions... Ema is refining your QA instructions based on the latest feedback") while the pipeline runs.

One Improvement Run at a Time, Per Rule

Only one improvement pipeline can run per rule at a time. If another Manager or Admin has already triggered one for the same rule, the drawer shows a banner instead of the checkboxes:

"A pipeline has been started by [name] — please wait for it to complete."

The Suggest improvements button is hidden until that run reaches a terminal state (Completed, Failed, Timed Out, or Discarded).

8.4 Applying Improvements

When the pipeline completes, the modal shows the original instruction and Ema's suggested instruction side by side — only the pass/fail instruction and the not-applicable (alternate-rule) instruction fields are shown; name, category, and weight are unchanged and not part of the suggestion.

1. Review the suggested instructions against the originals. You can edit the suggested text directly in the modal before deciding.
2. Click Update instruction to apply the suggestion, or Discard to dismiss the completed suggestion without applying it.
3. On success, a confirmation toast appears with a link back to the Configuration tab. The updated instructions take effect immediately for all new evaluations.

Rating Continuous Learning Improvements

After applying a Continuous Learning improvement, a feedback modal appears asking you to rate the experience on two questions, each on a 1–5 scale:

• How useful was the improvement Ema generated?
• How easy was reviewing and approving it?

You can optionally add a free-text comment alongside the star ratings. Both Skip and Done close the modal — the rating step is never required. A skipped question is stored as no answer and does not affect any aggregate score. If the submission fails in the background, the modal still closes; a failed rating never blocks you from continuing.

Discarding a Suggestion

If you decide a completed suggestion is not suitable, click Discard in the modal. The run moves to the Discarded terminal state and no longer surfaces as an ongoing improvement for that rule. No changes are made to the rule's instructions. A new improvement run can be started at any time after discarding.

Improvement Pipeline States

Manual rule editing from the Configuration tab's QA Parameters remains available as before and is unaffected by this pipeline.

8.5 Rating Auto-Learning Parameter Approvals

After approving Autolearning-generated parameters, a feedback modal appears asking you to rate the experience on two questions, each on a 1–5 scale:

• How useful were the parameters Ema generated?
• How easy was reviewing and approving them?

You can optionally add a free-text comment alongside the star ratings. Both Skip and Done close the modal — the rating step is never required. A skipped question is stored as no answer and does not affect any aggregate score. If the submission fails in the background, the modal still closes; a failed rating never blocks you from continuing.

8.6 Continuous Learning Governance API

To pull a persona's feedback funnel and auto-learn accuracy into an external BI tool or governance report, use the Rule Metrics API. This requires the same elevated persona access as other Agent QA data-pull APIs.

Query Parameters

Example Request

curl -X GET \
  "{base_url}/api/agent-qa/rule_metrics?persona_id=<persona_id>&start_date=2026-07-01&end_date=2026-07-28" \
  -H "Authorization: Bearer <token>"

Response

A successful response (200 OK) returns one object per active rule:

Use given → approved → improved as the feedback funnel for a rule, and compare it against auto_learn_accuracy to see whether feedback volume is translating into measurable accuracy gains.