Support triage agent with human-in-the-loop guardrails

Problem

Support volume grew faster than the team. Tickets were inconsistently tagged and routed, escalations were noisy, and context lived across threads, docs, and CRM notes.

Context & constraints

• Multiple inbound channels and shared ownership across Support/CS/Product
• Strict policies for PII and customer data handling
• No appetite for “another dashboard”—work needed to live in CRM + Slack
• Low tolerance for hallucinations or unsafe auto-writes

Approach

• Define a ticket taxonomy + routing contract (owners, SLAs, escalation rules, and exception paths).
• Use retrieval from approved knowledge sources with redaction and confidence thresholds.
• Run the agent as a controlled workflow (classify → summarize → propose next steps → route), with human review on low-confidence cases.

What Shipped

• Auto-classification + routing with reason codes and an exception queue
• Permission-aware summaries with redaction (no unsafe writes)
• Draft responses and internal handoff notes with approval gates
• Feedback loop for misroutes and low-quality summaries + monitoring

Operational outcomes

• Faster, more consistent triage with fewer misrouted escalations
• Less context-switching via in-tool summaries and handoff notes
• Reduced “CRM pollution” through confidence-gated writes and review queues
• Audit-friendly logs of what the agent saw, decided, and recommended

Governance & Safety

• Permission-aware retrieval
• PII redaction rules
• Human approval for low confidence
• Rate limits + monitoring
• Audit logs