Hokma Solutions / Case study
Support Coach Dashboard: transcript QA
Commercial SaaS, live at www.supportcoach.io.
My role. Product architect, QA rubric designer and full-stack lead
Stack: Next.js (App Router), TypeScript, Supabase PostgreSQL, OpenAI, pdfjs-dist, pdf-lib, Recharts, Paddle v2, Zoho SalesIQ API v2
Dashboard home (www.supportcoach.io)IT and security pageComparison pages
Inside the dashboard
What a support manager opens each week: a written briefing, the team, and a map of what confused customers. Click the tabs.
Animated sample. Every name, task and number in it is invented.
Week of 3 to 9 November · also sent to your email
A steady week. Two conversations worth having.
The team handled 58 chats and quality held at 7.4 out of 10, up 0.3. Most coaching from October is sticking: 68% of points were followed through. Maya and Leo each have a chat worth a few minutes, Priya has earned a thank-you, and the bank feed caused the most chats.
Conversations to have
Maya closed a chat before the customer confirmed the fix, for the second time since coaching.
16:58 Customer: "So will my report show the right total now? Hello?" Chat closed 17:03.
Message to send
Hi Maya, thanks for sorting the duplicate payment yesterday. You found the cause fast and your opening was spot on. One thing to work on together: the chat closed while the customer was still asking about her report. Let's take five minutes this week.
Leo gave the right answer in a form the customer could not follow.
Worth a thank-you
Priya stayed until the customer confirmed she was logged in. The point you coached on 16 October.
Five agents, two who need you this week
Last 30 days, 183 chats analysed. Sorted by who has coaching waiting, not by score.
What customers ran into
Topics the AI proposed from 60 chats and a manager approved. Bigger tile, more chats. Pick one to see what customers actually said.
Payments · 18 chats · Up from 6
"The same payment is in my report twice and I close the month on Friday."
The problem
Support managers reviewing chat transcripts hit two problems. Generic AI summaries invent timestamps, penalize agents for normal chat gaps or live screen-sharing sessions, and do not track whether an agent improved in later chats. Product teams cannot see which parts of the product confuse customers without tagging thousands of chats by hand.
How it was built
- Built an idempotent PDF transcript analysis pipeline with duplicate detection, broken-line stitching, and operator header attribution.
- Encoded domain-specific QA guardrails: response gaps under 2 minutes are never flagged; screen-sharing links followed by gaps of 5 minutes or more are treated as live sessions and exempted; abandoned chats, where the customer never replies, score neutrally with low attention.
- Built a coaching follow-through tracker: it extracts specific coaching points, tracks whether the manager delivered them, and checks later chats (followed through, repeated, or no opportunity) with manager override.
- Built AI-proposed, human-approved product confusion topics, Zoho SalesIQ REST API v2 auto-sync, and coaching notifications with branded PDF handouts.
- Implemented per-seat monthly AI cost controls with a hard spend ceiling (aiCostControls.ts).
Results I can show
- Completed a security and tenant-isolation audit of the dashboard's data access.
Limits
- Product confusion topics, SalesIQ auto-sync and coaching notifications are built but not yet released.
Try the demos
Synthetic data. The local rules engine is the real deterministic code. The AI layer here returns sample output. No model is called.
Layer 1 (local rules) runs the real deterministic code. The AI layer here returns sample output. No model is called.
Fields that leave the browser (apps/extension/src/lib/apiClient.ts)
{
"draft_text": "Like I said, you need to reconnect it yourself. It's not a bug on our end, and there is nothing I can do from here.",
"triggered_rules": [
"dismissive_harsh",
"blame_language",
"missing_ownership",
"cold_transactional"
],
"local_score": 72,
"draft_hash": "ead21245a262ef2d"
}
Layer 1: local rules (no AI)
Local score: 72/100
This could read as dismissive or abrupt to the customer.
Example: "I can help get this working for you right now. Let me walk you through the steps."
Rule families fired: dismissive_harsh, blame_language, missing_ownership, cold_transactional
Layer 3: pre-send check (sample output)
Overall risk: high
Identifies the reconnection requirement needed to resolve the issue.
- empathy
- 2/10
- ownership
- 2/10
- clarity
- 9/10
- actionability
- 9/10
- professionalism
- 2/10
- de escalation
- 2/10
- conciseness
- 9/10
Suggested rewrite
I can help you get this reconnected. This happens when the local connection token resets rather than a platform outage, and I can walk you through reconnecting it right now so everything syncs cleanly.
The deterministic parser, screen-share gap exemption (5 minutes or more after a session link), neutral scoring of abandoned chats, and the topic schema from supportcoach-ai, run on synthetic transcripts.
Attention: high
- Orphan Line & Sender Normalization (Task 8e/8h): Multi-line messages preserved under active speaker.
- Response-Time Threshold Rule: Gaps under 2 min ignored; only gaps >4 min without screen-share URLs are cited (max 2-3 timestamps).
Coaching note for the manager
Quick note on this chat: you got the customer to the right place (Settings > Billing). For next time, lead directly with what you CAN do ("I can reissue that invoice with your new address right now") rather than opening with "That is not possible", and skip phrases like "As I said earlier" after a 4.1-minute gap.
Product confusion topics
- Billing: update billing address on past invoice (confusion, 94%)
"Hi, I'm trying to download last month's invoice with our new company address, but the PDF keeps showing the old billing address." - Billing: update billing address on past invoice (confusion, 94%)
"Wait, isn't there a way to regenerate it? And why didn't the address update?"
A simulation of the two-tap approval card and the multi-provider pace router, on synthetic tasks. The agent's prediction stays hidden until you decide.
Pace router
- Anthropic (Claude Sonnet): 54% used this week, resets in 52h (behind pace (-14 pts))
- Google (Gemini Flash / Gem): 38% used this week, resets in 52h (behind pace (-30 pts))
- OpenAI (gpt-4o-mini API): 79% used this week, resets in 120h (ahead of pace (+11 pts))
sc-sender · effect: external (medium)
Send today's outbound batch: 4 verified prospects
The agent's prediction is committed and hidden until you decide.
Fifteen minutes, and you will know if it's a fit.
Tell me what your team is dealing with. I will tell you whether I can help and what it would cost.
Prefer to write?
Send a short brief