These six projects aren't a pile of demos - they're a stack. Each layer
answers a different form of one question: how do we know it works - and that it's safe
to run? Every one runs in seconds with zero API keys (nothing to sign up for or pay
for), and every one ships a click-through tour you can hand to any stakeholder.
Safety
A tripwire for poisoned skill files. Agents now load skills, tool
descriptions, and config files shared online - and a poisoned one carries hidden
instructions the model obeys - prompt injection, delivered as configuration. This scans for the shapes with no honest place in a config
file: invisible characters, download-and-run one-liners, over-broad shell grants,
read-and-exfiltrate lines. It fails closed, ships as a CLI and an MCP server, and states
its own limits in numbers.
A million randomized inputs: 0 crashes, 0 hangs, 0 false alarms on 450,360 synthetic benign files (fixed seed, reproducible; bounds false alarms on generated text, not in the wild)
Static detection is a triage floor, not a boundary. Its catch rate is an upper bound (one person wrote the rules and the samples); every miss is named in the report, and a corpus written by someone else is the top open item
Memory
This AI keeps its memory in a wiki of plain-text pages anyone can
read. It checks an index and pulls only the pages it needs, adds to old pages before
making new ones, and must write a reason before deleting anything. A paper trail for
what your AI believes.
Pulling only the pages it needs costs 0.61× of loading everything
With a small, constantly-used set of notes, loading everything wins (1.27×)
Evaluation
I test AI code reviewers the way you'd test a proofreader: plant one
known mistake, mix in clean copies, and only count a catch at the planted
line - crying wolf scores zero. New: nine copies dealt independent synthetic errors
climb at exactly the rate the arithmetic predicts, a mechanism demo rather than a model
result, while a copycat control stays flat.
Caught 9/10 planted bugs at the planted line, within two, with 0 false alarms
One blind spot left in on purpose, so the scorecard shows a real miss
Judgment
A courtroom for AI claims: one AI argues, a second's only job is to
attack, a third judges - nobody plays two roles. Every rejected claim gets a written
cause of death. And the flip test cross-examines the judge itself: change only the
judge's standing instruction, and see if verdicts change.
Flip test on two scripted judges, true by construction: evidence-led 0/8 flipped, agreeable 3/8 - exactly the ambiguous cases. Proves the instrument lights up, not that any real judge passes
Case study, clearly labeled (not a benchmark): 3 of 4 real verdicts flipped on one instruction change
The improvement loop
The AI never changes - its notebook does. Every miss gets filed as a
labeled note about what went wrong, every fix names the miss it repairs, and progress
is graded on an exam sealed before work began. A sugar-pill arm (same effort, fake
fixes) checks the gains are real.
Sealed-exam recall: 8/10 → 10/10. Sugar-pill arm on the same budget: unchanged
Published on purpose: an arm that teaches to the test - and pays for it in tokens
Operations
The first ten minutes of an incident are eaten by three questions: who
owns this, who do I wake and by when, and what does the runbook say to try first. This
answers them to an AI assistant as MCP tools - plus the one most tools get wrong, where
we stand on the clock, measured from when impact started rather than from when somebody
opened a ticket. It reads one local file. No network, no keys.
Five tools over one local file: owner, escalation path, first move, the impact clock, and a timeline review - zero runtime dependencies
No live integrations, by design: it does not read your incident tool, your rotation, or your monitoring - you feed it one file
Every number above regenerates from scratch on each repo's automated
build, on Windows and Linux - if a fresh run disagrees with the published claim, the
build fails. (skill-tripwire's million-input figure is fixed-seed reproducible rather than
run on every build - a million scans is too slow for CI - but a smaller run of it is gated
there.) The bundled agents are deliberately simple scripts, so these numbers prove the
measuring instruments, never any AI model's intelligence. No number on this page comes from a live model; the instruments are built so a live-model run can be published the same way, with its caveats, and the first one is: skill-tripwire's pre-registered live-judge study sits in that repo's eval folder as a partial run with every id named, and none of its numbers appear here.
Want the guided tour? Every project explained the way I'd brief a
decision-maker - precise analogies (sealed exam envelopes, sugar pills, smoke alarms),
a working vocabulary for terms like best-of-k and unhobbling, and
the questions I'd genuinely love to be asked about each project.
Take the tour →
