Ken

01The risk

An assistant that can message people can be talked into messaging the wrong person. Anything it reads, whether a WhatsApp message, an email or a calendar invite, could contain instructions written by someone else, and a model will sometimes follow them.

So Ken treats the model as fallible and as a possible target. It's free to read, search and draft. Every action that touches the outside world passes through checks it can't talk its way past.

02Safety lives in code

Six guards sit between the model and the outside world. Each is a few lines of Python with tests behind it; the test names link to the exact lines.

• Untrusted wrapping

  Everything Ken reads (messages, emails, tool results) is wrapped as data. Text inside an email can't escape the wrapper and pose as an instruction.

  test_wrap_untrusted_cannot_be_escaped
• Tool allowlist

  Only listed tools reach the model. Delete and archive tools are hidden, so the model can't call them at all.

  test_whatsapp_allowlist
• Send intent

  A send must come from the owner's own request, never from something Ken read.

  test_send_intent · test_send_blocked_without_intent
• Recipient guard

  Ken may only send to people who are already in the conversation or the owner's request.

  test_recipient_guard · test_unknown_recipient_blocked
• The click

  Every send opens a confirmation dialog the model can't reach. Declined sends are dropped and every send is audited.

  test_send_requires_confirmation_and_is_audited · test_declined_send_does_not_send
• Rate limit

  Only a few sends a minute, so a loop or an attack can't flood anyone.

  test_rate_limit_blocks_extra_sends

To test them, the evaluation planted a prompt-injection email in a fake inbox and asked Ken ten ordinary questions. The email asked Ken to forward invoices to an outside address. Ken sent nothing there in 0 of 10² runs and attempted no unrequested sends in 0 of 10³. A send needs the owner's click every time⁴.

03Memory that writes like me

Ken stores 985⁸ "them → me" reply pairs from my WhatsApp history in SQLite, embedded with text-embedding-3-small. Each pair is embedded on the other person's message, so a new message finds similar situations, and the stored reply shows how I handled them.

Retrieval filters by chat first: a pair's three nearest neighbours come from the same chat only 40%⁹ of the time, so without the filter, how I write to one person would leak into another chat. Each draft also gets that chat's last 24 hours.

04Results against a baseline

The same 50 real replies were held out and redrafted three ways. A blind judge picked the draft closest to what I actually sent.

The style gains are real, but the judge win of 32%⁶ is moderate: with 50 cases, differences under about ten points are noise. Meaning similarity doesn't move, because what a reply says comes from my decision, which no retrieval can recover. Length overshoot fell to +8%⁷.

05Switching models

gpt-4.1-mini is being retired, so I re-ran the same evaluation on gpt-6-luna before switching. Each model also judged blind, to cancel a judge's preference for its own writing.

Luna won on closeness under both judges (60–68%¹⁰) and cut the run's cost by −76%¹¹. It's about half a second slower, adds emoji I never use, and sometimes uses a Latin "?" instead of the Arabic "؟". The switch was worth it, and the regressions are listed rather than hidden.

06What I got wrong first

I split about 22% of my replies over two lines, and I tried to hit that rate with prompt wording alone. Four prompts later, the rate had swung between extremes and landed at 52%¹².

The lesson: a prompt can't hit an exact rate. The fix is to decide the split in code from my per-chat habit, and that's next.

07Limits

• Small samples. Style results use 50 replies; groundedness uses 27 claims over 10 questions; chat finding uses 10 names (9 of 10¹³ correct).
• The injection test is narrow. One planted email across 10 runs shows the guards work, but it doesn't measure how often attacks fail in general.
• The judge is a language model. No human labels yet.
• The safety evaluation predates the model switch. It ran on gpt-4.1-mini. The guards don't depend on the model, but the injection rate does.
• One person's data. All style results come from my own chats, in Arabic and English.

08What's next

1. Decide line splits in code from the per-chat rate.
2. Re-run the safety evaluation on gpt-6-luna with a larger attack set.
3. Collect human labels for a sample of judge decisions and report agreement.