CLIProxyAPI benchmark · Control and decision

Preflight errors, price, and failover

No deliberate monetary ceiling still leaves interpretability, safety, and identity gates.

Primary local sources: BENCHMARK-MEASUREMENT-PLAN.md, model-context-matrix.csv, and planned-run-manifest.json.
BENCHMARK-MEASUREMENT-PLAN.md · model-context-matrix.csv · planned-run-manifest.json

The idea in plain language

Price, entitlement, quota, balance, and projected cost must be known to interpret the run. But the projected monetary amount alone does not stop execution. Real quota, balance, rate or safety limits, and provider terms remain stops.

Running example: send the same sealed parcel — nonce, checksum, and three needles — through all six lanes. If the harness changes the wrapping, you are measuring the lane too. The analogy breaks because context, tools, and thinking carry semantics, not size alone.

Before error profiles, retry must be zero and failover disabled. The test compares status, code, sanitized message, Retry-After, and a redacted request ID. Unknown price blocks high volume for lack of interpretation; no deliberate monetary ceiling exists, and zero is never assumed.

Verifiable excerpt

deliberate_monetary_ceiling = null\nstop_on_projected_amount = false\nstop_on_real_quota_balance_safety = true\nretry = 0 · automatic_failover = false

outputs/BENCHMARK-MEASUREMENT-PLAN.md · outputs/model-context-matrix.csv · outputs/planned-run-manifest.json

The path in one picture

Running example: the same nonce + checksum crosses every lane; the failure belongs to the smallest boundary where it reproduces.

Retrieval check

Is projected monetary amount alone a stop?