Duck
LLM inference telemetry recorded as a ledger (CDR-style): tokens, latency, errors, and usage for audit and cost control. Built to be explicit, exportable, and boring in the best way.
Overview
Duck sits at the boundary where your app calls a model. It emits one structured record per inference call, similar to telecom call-detail records (CDR). You can aggregate these records for billing-style attribution, debugging, or audit trails.
Logs are inconsistent. Metrics are too lossy. Duck is a stable, queryable middle ground: a ledger of what happened, when, and at what cost.
Put Duck where requests enter/exit inference: API boundary, proxy, or model client wrapper. Avoid deep instrumentation inside business logic.
Cost attribution, regression detection, retry visibility, and a place to attach receipts from gateways like Heptaconn.
Record model (CDR-style)
Each call produces a single record with stable fields. Exact schema depends on your implementation; keep it versioned.
- timestamp, request_id, service/app identifiers
- model identifier, parameters (as explicit as you can)
- prompt_tokens, completion_tokens, total_tokens
- latency_ms, queue_ms (if available), retries
- status (ok / cancelled / timeout / error)
- error_code + error_detail (machine + human)
Rule: records should be usable without joining “mystery logs”. If a field matters, store it.
Installation
Duck is currently distributed privately while the interface stabilizes.
If you want to trial it, email us with your runtime (OpenAI/llama.cpp/etc.), target OS, and expected call volume.
Instrument at the inference boundary (client wrapper or gateway). Emit records as JSONL or SQLite depending on what you support.
# Example (conceptual)
# on_request_start: create record with request_id + metadata
# on_request_end: finalize with tokens, latency, status, error
# flush/commit (append-only)Pricing
Pricing is a placeholder until the public release and feature boundary are stable.
Core recorder + local storage + export.
Retention policies, richer exports, compliance packaging.
Support contract and custom integrations.
Contact
Questions: nsc@newssourcecrawler.com