Repository

harness-claude-code-usage-repro

Repro: @ai-sdk/harness-claude-code finish-step reports the whole turn's cumulative usage on the last step, not that step's own

input
output
Expected finish-step #2 usage
Expected
Actual finish-step #2 usage
Actual

Specs

  • inputoutput
  • Expected finish-step #2 usageExpected
  • Actual finish-step #2 usageActual

README

@ai-sdk/harness-claude-code: last-step finish-step usage is the whole turn's cumulative total, not that step's own usage

Minimal reproduction. npm install && npm run repro — no API key needed, it
drives the package's own bridge event-processing code against a synthetic
(but protocol-accurate) Claude Code CLI stream.

The bug

On any turn with two or more model calls (i.e. any turn with at least one
tool round trip before the final answer — subagents not required), the
finish-step part emitted for the last step reports the whole turn's
cumulative usage (every step summed) instead of that step's own usage.

Concretely, in packages/harness-claude-code/src/bridge/index.ts,
on the CLI's turn-ending result message:

const usage = msg.usage ?? msg.message?.usage; // cumulative, whole-turn total
const harnessUsage = mapUsage(usage);
if (harnessUsage) turnUsage = harnessUsage;
...
if (streamEventState.stepOpen) {
  emitFinishStep({
    state: streamEventState,
    emit,
    usage: harnessUsage ?? streamEventState.pendingStepUsage,
  });
}

streamEventState.pendingStepUsage is already sitting there with the right
value — the final step's own assistant message set it correctly (see
create-emit-stream-event.ts).
But a text-only final response never runs through closeStepIfReady() (that
only fires from tool-result handling or finishApprovalStep), so the step is
still open when result arrives — and harnessUsage (the cumulative total)
wins over pendingStepUsage (the correct, final-step-only value) because of
the ?? precedence.

Expected vs. actual

Given a turn with:

  • step 1 (a Bash tool call): 1000 input / 50 output tokens
  • step 2 (the final text answer): 1800 input / 70 output tokens of its own
  • CLI's result.usage (cumulative): 2800 input / 120 output tokens
input output
Expected finish-step #2 usage 1800 70
Actual finish-step #2 usage 2800 120

Why it matters

Anything consuming finish-step/finish to answer "how many tokens did the
model just use" — most obviously, a live "how full is the context window
right now" indicator — overcounts by the sum of every prior step in the turn,
on every ordinary multi-tool-call turn. It's easy to miss in testing because
a single-model-call turn (no tools, or the harness's built-in tools handled
in one shot) never shows a discrepancy: cumulative-so-far and "this step's
usage" are numerically identical when there's only one step.

This is a different bug from #18618 (subagent Task-tool activity bleeding
into the parent step's usage) — that one is fixed as of
[email protected]. This one reproduces unchanged against the
latest published version as of this writing
(@ai-sdk/[email protected], see package.json).

What this reproduction does

repro.ts imports the package's actual shipped bridge source (by relative
filesystem path into node_modules, since the package's exports map only
publishes the top-level entry point — the npm tarball does ship the src/
directory, pinned to whatever version is installed) and feeds it a
protocol-accurate two-step Claude Code CLI stream-json sequence: one Bash
tool round trip, then a final text answer, then the terminal result
message. It asserts the second finish-step's usage equals that step's own
usage. It currently fails (exit code 1) — that's the bug. If/when this is
fixed upstream, it should print PASS and exit 0.