unhardcoded

An OpenAI-compatible LLM router. Instead of hardcoding a model, a caller
sends a policy with each request: the host filters the candidate models to
those that qualify, ranks the survivors and picks one (e.g. the cheapest that
passes) — over your own provider keys — and falls back automatically when a
provider errors. Every run records which models passed, which were rejected and
why.

Concretely it's an async FastAPI shim that runs the
unhardcoded-engine core
and inherits its provider selection, fallback, retry and per-provider auth. The core
is vendored as a git submodule under core/; this repo is the host (the I/O,
auth, providers, the HTTP service) plus an operator dashboard to compose,
test and analyse policies. The policy algebra (Σ_pol) lives in the core.

The model: callers send a policy

A request carries its own Σ_pol policy as data — a policy_ir term in the
body. The host admits it (sorts, arity, depth/size bounds), ∧-composes the
host's policy_envelope so a caller can only narrow the host's invariants,
then runs it: filter the candidate models, score the survivors, pick and fall
back. Compose, preview, test live and download policies in the dashboard
Builder.

There are no intelligence tiers. When a caller sends no policy, a single
declarative default policy (balanced quality/cost) is the fallback — itself a
Σ_pol term with an identity, editable like any other.

Layout

core/                  -- git submodule -> genlayerlabs/unhardcoded-engine (the pure Σ_pol core)
llm_router_host.py     -- embeds the core via lupa; sync/async backends; auth resolver
shim.py                -- OpenAI-compatible app: /v1/chat/completions and /v1/responses (+ per-call policy_ir) and /x/* operator endpoints
responses_api.py       -- inbound OpenAI *Responses* API surface for /v1/responses (sibling of the chat-completions surface)
auth_proxy.py          -- ingress + operator dashboard (Analytics · Builder · Activity · Market · Settings)
serve.py               -- entry point: wires the api_kind dispatcher + runs uvicorn
config.live.lua        -- catalog (providers + models), the `default` policy, and the observation fields (incl. OpenRouter benchmark/modality/capability fields)
scripts/refresh_model_meta.py -- the job: writes model_meta.lua (model-level traits pulled from OpenRouter)
codex_auth.py / codex_backend.py -- the ChatGPT-subscription (Codex) provider
metrics.live.lua       -- EMA seed (PLACEHOLDER/fake — see docs/METRICS.md)
docs/METRICS.md        -- the metrics seed: format, codex≈0, regeneration (it's fake)
docs/PROVIDERS.md      -- per-provider auth + the AntSeed node
docs/OPENAI-CODEX.md   -- the ChatGPT-subscription provider (unofficial / ToS-risky)
docs/USAGE_ENDPOINTS.md -- per-key usage stats API endpoints and auth behavior
live_smoke.py          -- drive real providers end-to-end
tests/                 -- the full host unit-test suite
features/              -- BDD user-flow suite (behave); user_flows.json is the spec
SETUP.md               -- agent/human setup runbook (clone -> running -> first call)
scripts/gen-dev-wallet.sh -- generate a local AntSeed dev wallet (testing)

Quickstart

Point a coding agent at SETUP.md ("set up unhardcoded by
following SETUP.md") for a guided, copy-pasteable runbook — or do it yourself:

git clone --recursive https://github.com/genlayerlabs/unhardcoded.git && cd unhardcoded
cp .env.example .env.secrets && chmod 600 .env.secrets
# in .env.secrets: set OPENROUTER_API_KEY, and DASHBOARD_NO_AUTH=1 for local dev
docker compose up -d --build
curl -fsS http://127.0.0.1:8080/healthz            # -> {"ok":true,"initialized":true}

Open the dashboard at http://127.0.0.1:8080/dashboard, mint a caller key
(POST /dashboard/api/keys {"consumer":"my-app"}, or Consumers → Generate key),
then call:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{"model":"","messages":[{"role":"user","content":"Reply: pong"}]}'

Empty model runs the default policy; or send family:/pin:/profile: or a
per-call policy_ir/flow_ir (below). Optional providers (Codex, AntSeed) and
troubleshooting are in SETUP.md.

Responses-only clients (e.g. the Codex CLI): POST /v1/responses (and
POST /{profile}/v1/responses) speak the OpenAI Responses API — translated to
the same routing/policy/metering as chat-completions, streaming Responses SSE.
Point Codex at the router with base_url=…/v1, wire_api = "responses", and the
router key; use a /{profile}/v1/responses base_url to pin a policy by URL (Codex
can't put policy_ir in its body).

Develop without Docker (raw shim — no dashboard/auth)

python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
python serve.py --config config.live.lua --default-profile default --host 127.0.0.1 --port 8080

serve.py is the data plane only (no ingress auth, no dashboard); provider keys
come from the process env. The bearer-token contract and the dashboard live in
the ingress service (docker compose). (Nix users: a nix-shell with the
same packages works too.)

Policies (the model field + policy_ir)

Routing is server-side. The primary path is a per-call policy; the model
prefix is sugar for the common cases:

A policy is a Σ_pol term: filter (which models qualify) → score (rank
the survivors) → pick (the selector) + request transform + failover. Write it
as raw IR, or compose it in the Builder (structured rows ↔ raw term). The
vocabulary it filters/scores over includes live fields (price_in/price_out,
latency_ms, success_rate, …) and host-declared fields — the OpenRouter
benchmarks/modalities/capabilities (bench_intelligence, in_image,
cap_tools, …). See the core's core/docs/SIGMA-POL.md for the algebra;
config.live.lua declares the host fields and the default policy.

Blessed policy templates

Most callers should not write raw weights. GET /x/policy/templates lists the
supported intent-level templates; POST /x/policy/templates/{id} compiles one
to an identified policy_ir that can be previewed with POST /x/rank.

curl -s http://127.0.0.1:8080/x/policy/templates/cheapest-family \
  -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{
    "family": "glm-5.2",
    "provider_strategy": "ordered"
  }'

The four templates are:

• cheapest-family — stay in one exact family and minimize expected token
  cost; provider_strategy: "ordered" instead enforces Codex → AntSeed →
  Bedrock → OpenRouter, skipping unavailable providers and retaining
  breaker-open routes only as final fallbacks.
• smart-value — minimize cost among the current top five intelligence models
  (the shortlist size and price/reliability rails are configurable).
• agent — the reusable profile:agent policy for autonomous tool users:
  require tools, 128k context, top-ten measured intelligence and reliable,
  priced routes; prefer healthy Codex/direct providers before gateways and
  trusted AntSeed peers; cap the cascade at eight candidates. The policy also
  sets a 10s first-token and 22s per-attempt timeout and moves immediately to a
  different candidate on provider failures, so a stalled first route cannot
  consume the complete request deadline.
• default — the actual policy used by OpenAI-compatible callers that send no
  policy: prefer a top-five intelligence model, then enforce Codex → AntSeed →
  Bedrock → OpenRouter and use a 75% cost / 25% intelligence value score inside
  each provider. Lower-ranked models remain available when request requirements
  or an explicit family: model leave no top-five candidate.

The ordered strategy compiles to nested prefer(pred, inner) selectors, not
large score weights. Cost still orders candidates inside each provider group.
All templates reject unknown prices and default to explicit $5 input / $25
output per-million-token ceilings plus a 0.8 reliability floor; callers can
tighten or deliberately raise those rails on the configurable templates.

Dashboard

auth_proxy serves an operator console at /dashboard:

• Analytics — spend, traffic and errors over time, filtered by timeframe,
  consumer, provider and model.
• Builder — compose a policy over raw + benchmark fields, preview the live
  ranking, Test call it with a prompt, and download the term to run per call.
• Activity — per-request trace: the policy that ran (copyable), the ordered
  fallback chain with the error at each step, and the cost paid.
• Market — live price book per model family.
• Settings — consumer keys and provider keys.

Tests

Install the test deps once (a virtualenv keeps them off your system Python):

python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt -r requirements-dev.txt

Unit — boots a real host with mocked provider responses (only the outbound
HTTP to upstream providers is mocked):

python -m pytest tests -q

BDD user-flow suite (features/, behave) — drives the live stack end to end
the way the dashboard does and asserts the rendered data is correct, including a
real headless-browser pass (needs Chrome/Chromium installed; Selenium fetches the
matching driver). Free & repeatable: end-to-end chats route to a $0 path. The
catalogue of flows it covers is user_flows.json:

behave

(Nix users: nix-shell -p ... with the same packages — plus chromium chromedriver for the browser pass — works as before.)