Hokma Solutions / Case study
An operating system for AI agents, with a person in charge
My own system, in a private repo. Built and tested on my own work and on sample data. Not a product for sale, and not yet run for a client.
My role. Designer, builder and only operator
Stack: Next.js, React, Postgres with row-level security, Python, Node test runner
Follow one request through the system
Seven stages, two moments where a person decides, and a hard stop when something breaks.
Animated sample. Every name, task and number in it is invented.
- QueueWork comes in
- StepsSplit into steps
- ClaimOne agent at a time
- GoA person starts it
- BuildOn its own branch
- ReviewA second model checks
- Final cardA person says yes
Stage 1 of 8
Three requests are waiting.
Each one is a goal written in plain words, not a prompt. Nothing has started yet.
- Refresh the trial onboarding emailsNext
- Summarise last month's refund requestsWaiting
- Tidy the pricing page wordingWaiting
The goal becomes four steps.
Each step is small enough to check on its own, and each has a time and cost limit.
- Read the current three emails2 min
- Draft three new emails10 min
- Check every claim against the sources3 min
- Build a preview page2 min
One agent claims the step, for a limited time.
While the claim holds, no other agent can take the same step, so the work never runs twice. If the agent goes quiet, the claim runs out and the step goes back in the queue.
A person taps Go before any work starts.
The card says what will happen, where, and the most it can cost. Nothing runs without this tap.
Start this work?
Draft three onboarding emails- Where
- Its own draft branch
- Cost cap
- $2.00
- Sends anything?
- No
GoNot now
The work happens on its own branch.
It ends as a draft with a preview link. Nothing in the live product changes.
- email-1-welcome.mdDrafted
- email-2-first-week.mdDrafted
- email-3-trial-ending.mdDrafted
- Preview pageReady
A second, independent model reviews it.
It did not write the draft, so it has no reason to defend it. Here it catches a promise nobody approved.
- Facts match the source pagesPass
- Tone matches the style guidePass
- Email 3 offers a 20% discount. No discount is approved.Flagged
The person gets one final card.
Draft, preview, the reviewer's notes, and the agent's own guess at the decision, sealed until the person decides.
Ready for your decision
Three onboarding emails1 note from the reviewer: discount line removed in this version.
Agent's guess is sealed until you decide
ApproveEditReject
Approved, and written down.
The journal records who said yes, when, and whether the agent guessed right. Over time that shows which work can be trusted to need fewer checks.
- DecisionApproved
- Decided byThe owner
- Agent's sealed guessApprove: right
- Sent to customersOnly after this yes
Three requests are waiting.
Each one is a goal written in plain words, not a prompt. Nothing has started yet.
- Refresh the trial onboarding emailsNext
- Summarise last month's refund requestsWaiting
- Tidy the pricing page wordingWaiting
The goal becomes four steps.
Each step is small enough to check on its own, and each has a time and cost limit.
- Read the current three emails2 min
- Draft three new emails10 min
- Check every claim against the sources3 min
- Build a preview page2 min
One agent claims the step, for a limited time.
While the claim holds, no other agent can take the same step, so the work never runs twice. If the agent goes quiet, the claim runs out and the step goes back in the queue.
A person taps Go before any work starts.
The card says what will happen, where, and the most it can cost. Nothing runs without this tap.
Start this work?
Draft three onboarding emails- Where
- Its own draft branch
- Cost cap
- $2.00
- Sends anything?
- No
GoNot now
The preview build fails.
The step is retried once, automatically, with the error attached.
- Three draftsDone
- Preview page: a linked image is missingFailed
- Retry 1 of 1Running
It fails again, so it stops.
No third try and no guessing. The step waits for a person, and nothing else on this request moves.
Needs you
Step 4 failed twiceThe preview links an image that is not in the branch. The drafts are safe.
Look at itDrop the step
Who may do what
Access comes in steps that a person grants. Try a request and see where it stops. A job title or a clever line of text grants nothing.
- Anything elseRefused in code
- One named actionOnly after a person says yes
- DraftWrite something, never send it
- ReadOpen records this person may see
- ShowSee that a record exists
Pick a request
The marker shows how far it gets.
The problem
Most AI agent setups either do too little or do things nobody approved. I wanted several AI models working as one team on real work, and still never sending, spending or publishing anything without a yes from a person.
How it was built
- Work goes into one queue, split into steps. Each step is claimed by one agent at a time for a limited time, so the same work never runs twice. A failed step is retried once, then stopped for a person to look at.
- A person taps go before work starts. The work happens on its own branch and ends as a draft with a preview. A second, independent model reviews it, and the person gets one final card.
- Who may use what is kept apart from what the business wants. A job title or a clever prompt grants nothing. Access comes in steps: show, read, draft, then one named action after a person says yes. Anything beyond that is refused in code.
- A next-move finder reads only the records the signed-in person may see. Each finding cites its evidence and is marked as fact, estimate, guess or unknown. A finding that rests on a record the person cannot read is dropped.
- A launch checklist covers eight areas, from demand to trust, and returns ready or not ready with the reasons. A plan alone never counts as done.
- A plan-first prototype for work that takes many steps: the agent writes the whole plan, and every call is checked for permission, time and cost before it runs.
Results I can show
- Unit, database and race tests, plus end-to-end simulations of the queue, run on every change. I show the suites running on a call.
- The finder and the capability map run on sample companies and pass their tests, including a job title trying to grant access and a prompt trying to.
Limits
- Built for one owner first. No client has used it yet, and I do not quote time or money saved.
- The finder and the capability map run on sample data only. No industry is claimed as supported.
- The plan-first prototype is not production enforcement. It is the basis for a design review, not a product.
- Private repo. I walk you through it on a call.
Try the demos
Synthetic data. The local rules engine is the real deterministic code. The AI layer here returns sample output. No model is called.
Layer 1 (local rules) runs the real deterministic code. The AI layer here returns sample output. No model is called.
Fields that leave the browser (apps/extension/src/lib/apiClient.ts)
{
"draft_text": "Like I said, you need to reconnect it yourself. It's not a bug on our end, and there is nothing I can do from here.",
"triggered_rules": [
"dismissive_harsh",
"blame_language",
"missing_ownership",
"cold_transactional"
],
"local_score": 72,
"draft_hash": "ead21245a262ef2d"
}
Layer 1: local rules (no AI)
Local score: 72/100
This could read as dismissive or abrupt to the customer.
Example: "I can help get this working for you right now. Let me walk you through the steps."
Rule families fired: dismissive_harsh, blame_language, missing_ownership, cold_transactional
Layer 3: pre-send check (sample output)
Overall risk: high
Identifies the reconnection requirement needed to resolve the issue.
- empathy
- 2/10
- ownership
- 2/10
- clarity
- 9/10
- actionability
- 9/10
- professionalism
- 2/10
- de escalation
- 2/10
- conciseness
- 9/10
Suggested rewrite
I can help you get this reconnected. This happens when the local connection token resets rather than a platform outage, and I can walk you through reconnecting it right now so everything syncs cleanly.
The deterministic parser, screen-share gap exemption (5 minutes or more after a session link), neutral scoring of abandoned chats, and the topic schema from supportcoach-ai, run on synthetic transcripts.
Attention: high
- Orphan Line & Sender Normalization (Task 8e/8h): Multi-line messages preserved under active speaker.
- Response-Time Threshold Rule: Gaps under 2 min ignored; only gaps >4 min without screen-share URLs are cited (max 2-3 timestamps).
Coaching note for the manager
Quick note on this chat: you got the customer to the right place (Settings > Billing). For next time, lead directly with what you CAN do ("I can reissue that invoice with your new address right now") rather than opening with "That is not possible", and skip phrases like "As I said earlier" after a 4.1-minute gap.
Product confusion topics
- Billing: update billing address on past invoice (confusion, 94%)
"Hi, I'm trying to download last month's invoice with our new company address, but the PDF keeps showing the old billing address." - Billing: update billing address on past invoice (confusion, 94%)
"Wait, isn't there a way to regenerate it? And why didn't the address update?"
A simulation of the two-tap approval card and the multi-provider pace router, on synthetic tasks. The agent's prediction stays hidden until you decide.
Pace router
- Anthropic (Claude Sonnet): 54% used this week, resets in 52h (behind pace (-14 pts))
- Google (Gemini Flash / Gem): 38% used this week, resets in 52h (behind pace (-30 pts))
- OpenAI (gpt-4o-mini API): 79% used this week, resets in 120h (ahead of pace (+11 pts))
sc-sender · effect: external (medium)
Send today's outbound batch: 4 verified prospects
The agent's prediction is committed and hidden until you decide.
Fifteen minutes, and you will know if it's a fit.
Tell me what your team is dealing with. I will tell you whether I can help and what it would cost.
Prefer to write?
Send a short brief