Building Custom Detectors with the Zeuslock CLI

Author, backtest, and ship org-specific detectors for internal IDs, API keys, and source markers using the Zeuslock CLI — with versioning and CI gates.

Why custom detectors exist

Every organization has identifiers that no off-the-shelf regex will ever catch: customer reference numbers like CUS-12345, internal API keys minted by your platform team, hostname patterns for staging clusters, source-code markers that only appear in your monorepo, M&A target codenames. Zeuslock's built-in detectors cover credit cards, IBANs, JWTs, AWS AKIA keys, OpenAI sk- tokens and the usual catalogue — they will not catch what is unique to your company. Custom detectors close that gap. They participate in the same severity model, the same Monitor / Anonymize / Block actions, and the same incident pipeline as the built-ins.

Install the CLI and authenticate

The Zeuslock CLI is the authoring surface for custom detectors. It is distributed via npm and Homebrew.

npm i -g @zeuslock/cli
# or
brew install zeuslock/tap/cli

zeuslock --version
zeuslock auth login

zeuslock auth login opens your browser, completes an OAuth flow against your Operator Console tenant, and stores a short-lived refresh token in your OS keychain. The token inherits your console role: only users with the Detector Author or Admin role can deploy. All other commands work for any authenticated user.

The detector YAML schema

A detector is a YAML file. Here is a complete, realistic example for an internal customer ID of the shape CUS-XXXXX:

name: customer-id
description: Acme internal customer identifier CUS-XXXXX
severity: medium
action: anonymize
patterns:
  - regex: 'CUS-[0-9]{5}'
    flags: i
validators:
  - type: context_keyword_within
    keywords: [customer, client, account]
    window_chars: 50
replacement: 'CUS-XXXXX'
contexts:
  apply_to: [browser, desktop, cli]
  skip_destinations: [chat.internal.acme.com]

Field	Purpose
`name`	Globally unique identifier within your tenant. Lower-case, kebab-case. Used in policies, dashboards and rollback commands.
`description`	Human-readable. Surfaces in the Operator Console next to every incident this detector raises.
`severity`	`low`, `medium`, `high`, or `critical`. Drives default routing in the incident dashboard.
`action`	`monitor`, `anonymize`, or `block`. Can still be overridden per group in the console.
`patterns[].regex`	RE2 syntax — no lookbehind, no backreferences. Anchors and character classes are encouraged.
`patterns[].flags`	`i` for case-insensitive, `m` for multiline. Combine as `im`.
`validators[]`	Post-match filters that suppress false positives. See the validator types below.
`replacement`	Format-preserving redaction template used when `action: anonymize`. Should be syntactically valid against the same regex.
`contexts.apply_to`	Surfaces this detector runs on: any subset of `browser`, `desktop`, `cli`, `api`.
`contexts.skip_destinations`	Hostnames or URL patterns to exclude — typically your internal AI tools where the value is legitimate.

Validator types

luhn — runs the Luhn checksum. Use on anything that looks like a card or membership number.
mod97 — IBAN-style modulo 97 check.
entropy — requires Shannon entropy of the match to exceed a threshold (min: 3.5). Critical for high-entropy secrets like API keys.
context_keyword_within — requires one of keywords to appear within window_chars of the match. Brutally effective at suppressing false positives.
post_regex — a second regex applied to the full match for structural confirmation.

The author workflow

The CLI is designed around a tight feedback loop: scaffold, test, backtest, deploy, monitor.

Scaffold. zeuslock detectors init customer-id creates customer-id.yaml in the current directory with sensible defaults and inline comments.
Write the regex. Open the YAML, replace the placeholder pattern, add validators. Keep the regex narrow — see the anti-patterns below.
Local match preview. Test a single string immediately:
```
zeuslock detectors test ./customer-id.yaml \
  --input "Please refund customer CUS-12345 today"
```
The CLI prints each match, the validators that passed or failed, and the resulting action.
Backtest against real history. The killer feature:
```
zeuslock detectors backtest ./customer-id.yaml \
  --since 30d --sample 1000
```
Zeuslock replays your detector against the last 30 days of incidents (already redacted at ingest), and reports estimated true positives, estimated false positives, and a preview list of triggering snippets so you can eyeball the noise.
Iterate. Tighten the regex or add validators until the backtest false-positive rate is acceptable. We treat anything above 5% as a release blocker.
Dry-run deploy. zeuslock detectors deploy ./customer-id.yaml --group engineering --dry-run shows the diff against the currently active version and what would change in policy assignments — without publishing.
Deploy. Drop --dry-run. The detector is live in seconds on every connected browser extension, desktop agent, and CLI instance in the targeted group.

Operational lifecycle

zeuslock detectors list
zeuslock detectors show customer-id
zeuslock detectors disable customer-id
zeuslock detectors rollback customer-id --version 3

Every deploy creates a new immutable version. disable pauses the detector without deleting it; rollback reactivates a prior version as the live one without losing history. The console exposes the same versions in a timeline, with the author, deploy time, and diff for each.

CI integration

Detector YAML belongs in a Git repo, code-reviewed like any other production artefact. Run zeuslock detectors test as a pull-request check so a broken regex never reaches the Deploy step.

name: detector-ci
on: [pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm i -g @zeuslock/cli
      - run: zeuslock auth token --from-env
        env:
          ZEUSLOCK_CI_TOKEN: ${{ secrets.ZEUSLOCK_CI_TOKEN }}
      - run: |
          for f in detectors/*.yaml; do
            zeuslock detectors test "$f" --fixtures fixtures/
          done
      - run: zeuslock detectors backtest detectors/ --since 7d --sample 500

Pair each detector YAML with a fixtures/ directory containing should-match.txt and should-not-match.txt. The --fixtures flag asserts both directions: every line of should-match must produce a hit, every line of should-not-match must not. Detector regressions become diff comments on the PR, not 3 a.m. pages.

Anti-patterns to refuse in review

Overly broad regex. A pattern like \d+ or [A-Z]{5,} with no validators will match half your traffic. Anchor your patterns and require a structural prefix (the CUS- in the example is doing real work).

No validators on a high-entropy candidate. Anything claiming to be a secret needs at minimum an entropy validator. Without it you will flag base64-encoded thumbnails, Git commit SHAs, and UUIDs as credentials.

action: block on a detector that has not been backtested. Never. Ship every new detector at monitor for one week minimum, review incidents, escalate to anonymize, and only move to block once the false-positive rate over a 1000-incident sample is below 1%.

Where to go next

Once your detector is live, it shows up in the same policy matrix as built-in detectors — you can scope it per group, per destination, and per schedule from the Operator Console. Hand the YAML files to your detection engineering team as code; treat the Git repository, not the console, as the source of truth.