Hokma Solutions / Case study

Support Coach Dashboard: transcript QA

Commercial SaaS, live at www.supportcoach.io.

My role. Product architect, QA rubric designer and full-stack lead

Stack: Next.js (App Router), TypeScript, Supabase PostgreSQL, OpenAI, pdfjs-dist, pdf-lib, Recharts, Paddle v2, Zoho SalesIQ API v2

Inside the dashboard

What a support manager opens each week: a written briefing, the team, and a map of what confused customers. Click the tabs.

Animated sample. Every name, task and number in it is invented.

Support Coach Dashboard

A steady week. Two conversations worth having.

The team handled 58 chats and quality held at 7.4 out of 10, up 0.3. Most coaching from October is sticking: 68% of points were followed through. Maya and Leo each have a chat worth a few minutes, Priya has earned a thank-you, and the bank feed caused the most chats.

Chats analysed58
Team score7.4
Coaching followed through68%
Waiting on you4

Conversations to have

Maya closed a chat before the customer confirmed the fix, for the second time since coaching.

High attentionRepeatedRated Sad · fair

16:58 Customer: "So will my report show the right total now? Hello?" Chat closed 17:03.

Message to send

Hi Maya, thanks for sorting the duplicate payment yesterday. You found the cause fast and your opening was spot on. One thing to work on together: the chat closed while the customer was still asking about her report. Let's take five minutes this week.

Leo gave the right answer in a form the customer could not follow.

Medium attentionRated Neutral · fair

Worth a thank-you

Priya stayed until the customer confirmed she was logged in. The point you coached on 16 October.

Followed through

The problem

Support managers reviewing chat transcripts hit two problems. Generic AI summaries invent timestamps, penalize agents for normal chat gaps or live screen-sharing sessions, and do not track whether an agent improved in later chats. Product teams cannot see which parts of the product confuse customers without tagging thousands of chats by hand.

How it was built

  • Built an idempotent PDF transcript analysis pipeline with duplicate detection, broken-line stitching, and operator header attribution.
  • Encoded domain-specific QA guardrails: response gaps under 2 minutes are never flagged; screen-sharing links followed by gaps of 5 minutes or more are treated as live sessions and exempted; abandoned chats, where the customer never replies, score neutrally with low attention.
  • Built a coaching follow-through tracker: it extracts specific coaching points, tracks whether the manager delivered them, and checks later chats (followed through, repeated, or no opportunity) with manager override.
  • Built AI-proposed, human-approved product confusion topics, Zoho SalesIQ REST API v2 auto-sync, and coaching notifications with branded PDF handouts.
  • Implemented per-seat monthly AI cost controls with a hard spend ceiling (aiCostControls.ts).

Results I can show

  • Completed a security and tenant-isolation audit of the dashboard's data access.

Limits

  • Product confusion topics, SalesIQ auto-sync and coaching notifications are built but not yet released.

Try the demos

Synthetic data. The local rules engine is the real deterministic code. The AI layer here returns sample output. No model is called.

The deterministic parser, screen-share gap exemption (5 minutes or more after a session link), neutral scoring of abandoned chats, and the topic schema from supportcoach-ai, run on synthetic transcripts.

Attention: high

  • Orphan Line & Sender Normalization (Task 8e/8h): Multi-line messages preserved under active speaker.
  • Response-Time Threshold Rule: Gaps under 2 min ignored; only gaps >4 min without screen-share URLs are cited (max 2-3 timestamps).

Coaching note for the manager

Quick note on this chat: you got the customer to the right place (Settings > Billing). For next time, lead directly with what you CAN do ("I can reissue that invoice with your new address right now") rather than opening with "That is not possible", and skip phrases like "As I said earlier" after a 4.1-minute gap.

Product confusion topics

  • Billing: update billing address on past invoice (confusion, 94%)
    "Hi, I'm trying to download last month's invoice with our new company address, but the PDF keeps showing the old billing address."
  • Billing: update billing address on past invoice (confusion, 94%)
    "Wait, isn't there a way to regenerate it? And why didn't the address update?"

Fifteen minutes, and you will know if it's a fit.

Tell me what your team is dealing with. I will tell you whether I can help and what it would cost.

Book a 15-minute call

Prefer to write?

Send a short brief