Skip to content

Kiro: IDE v2 session store support + credit-based pricing across all formats#665

Open
Enclavet wants to merge 3 commits into
getagentseal:mainfrom
Enclavet:kiro_v2
Open

Kiro: IDE v2 session store support + credit-based pricing across all formats#665
Enclavet wants to merge 3 commits into
getagentseal:mainfrom
Enclavet:kiro_v2

Conversation

@Enclavet

Copy link
Copy Markdown

Summary

Kiro's newly released 1.0 IDE (observed on 1.0.138; the 0.12.x builds still write the v1 globalStorage format) moves sessions to ~/.kiro/sessions/<workspace-hash>/sess_<id>/ — a self-contained session.json + messages.jsonl event log — instead of globalStorage. This PR adds a v2 parser and discovery, and — validated against a real upgraded machine — moves Kiro cost accounting from token estimation to the real metered credits Kiro records, across every format that carries them (CLI, v1 IDE executions, v2 IDE).

Token estimation wasn't just imprecise, it inverted model rankings. On a real machine, one week of Kiro usage:

Model Token-estimated Metered credits Error
Sonnet 4.6 $339.82 $161.52 2.1× overstated
Opus 4.6 $11.15 $130.57 11.7× understated
Fable 5 $7.20 $118.15 16× understated

What's included

New: IDE v2 store (sess_*/messages.jsonl)

  • Parses the event-sourced turn stream (user / turn_start / assistant / tool_call / tool_result / usage_summary / turn_end) with per-execution dedup keys (kiro-v2:<session>:<execId>) and defensive flushes for out-of-order events and in-flight sessions.
  • Prices turns from usage_summary.promptTurnSummaries[].usage credits at the public overage rate (USD_PER_KIRO_CREDIT = $0.04; individual plan is $20/mo for 1,000 credits) — same never-understate approach as the Codebuff provider.
  • Counts tool_result content as input context (matching the CLI parser's ToolResults treatment) and takes the real modelId from session.json, so v2 sessions aren't mislabeled kiro-auto.
  • v1 → v2 is a clean cutover (no migration/dual-write observed); dedup namespaces are disjoint, so nothing is counted twice.

Fixed: credit pricing now actually reaches reports

  • metering_usage values are credits, not dollars. The existing CLI parser summed them as USD, overstating CLI cost 25× at the real rate.
  • Kiro joins the costUSD pass-through allowlist in providerCallToCachedCall (alongside mistral-vibe, devin, hermes, vercel-gateway, antigravity). Without this, the parser's credit-based cost was discarded at the cache boundary and re-derived from estimated tokens — silently undoing all credit pricing. Tradeoff (shared with all allowlisted providers, documented): price-override / model-alias don't affect Kiro dollar amounts.
  • v1 IDE executions are credit-priced too. usageSummary[].usage (the v1 predecessor of v2's promptTurnSummaries) was already being iterated for usedTools but its credit values were ignored. Since v2 only ships in brand-new builds, v1 is the format nearly all Kiro IDE users are on today. Verified per-turn semantics on real data (values fluctuate; not cumulative counters).
  • Uniform fallback contract. All three credit parsers gate on summed credits > 0 and fall back to token-estimated pricing with costIsEstimated: true. This also fixes two CLI edge cases: no-metering turns were priced at a frozen $0, and empty metering_usage arrays (observed on 75 of 10,105 real turn metadatas — meta written before metering lands) passed the old truthy-presence check and froze $0 marked as real.

Fixed: v2 reasoning tokens double-counted

Reasoning was folded into outputTokens and reported in reasoningTokens, but aggregation sums the two fields. Now disjoint (matching codex/gemini); reasoning is still billed as output in the pricing fallback.

Cache invalidation

Both persistence layers froze pre-fix dollar values and never self-invalidate (historical session files don't change):

  • session cache: CACHE_VERSION 4 → 5
  • daily rollup: DAILY_CACHE_VERSION 10 → 11 with MIN_SUPPORTED_VERSION raised (same pattern as the v10 cursor accounting bump)

One-time full re-parse; verified live that the corrected numbers land.

Testing

  • 20+ new tests: v2 parsing/discovery/dedup/routing, credit vs fallback pricing per format, tool_result turn scoping, empty-metering edge, reasoning disjointness, and a mixed-format coexistence suite (legacy .chat + v1 executions + workspace-sessions + CLI + v2 on one machine) asserting exact aggregate counts and no double counting.
  • The cost pass-through test is mutation-verified: it runs end-to-end through parseAllSessions and fails with the token re-price ($0.000048 vs $0.10) if the allowlist entry is removed.
  • Validated against a real machine: 9,995 CLI turns, 221 v1 executions, and 24 v2 turns now carry metered cost ($1,793 total); call counts unchanged before/after (no calls gained, lost, or double-counted).
  • Full suite green; tsc clean; tsup bundle builds.

Docs

docs/providers/kiro.md updated: v2 store layout, credit pricing sources per format, dedup namespaces, the parse-time cost-freezing contract (and its price-override implication), and the disjoint-store v1→v2 verification. A comment at fingerprintFile documents the companion-file blind spot (CLI credits / v2 modelId live next to, not in, the fingerprinted transcript).

andklee added 3 commits July 14, 2026 22:35
Kiro's new IDE builds write sessions to ~/.kiro/sessions/<hash>/sess_<id>/
(session.json + messages.jsonl event log) instead of globalStorage. Add a
v2 parser and discovery, plus fixes surfaced while validating against a
real upgraded machine:

- Parse v2 event-sourced turns (user/turn_start/assistant/tool_call/
  tool_result/usage_summary/turn_end) with per-execution dedup keys
  (kiro-v2:<session>:<execId>) and defensive flushes for out-of-order
  events and in-progress sessions
- Price turns from real metered credits at the public overage rate
  (USD_PER_KIRO_CREDIT = $0.04/credit, individual plan: $20/mo for
  1,000 credits), falling back to token estimation only when a turn has
  no usage_summary (aborted/in-flight); costIsEstimated distinguishes
  the two
- Fix pre-existing CLI parser bug: metering_usage values are credits,
  not dollars — costs were overstated 25x at the real rate
- Count v2 tool_result content as input context (matching the CLI
  parser's ToolResults treatment); previously only the user prompt was
  counted, undercounting input ~25x on agentic turns
- Keep reasoningTokens disjoint from outputTokens: downstream
  aggregation (models-report, audit-report, parser) sums the two
  fields, so folding reasoning into output would double-count. Combine
  them only for the token-pricing fallback, matching codex/gemini
- Take the real modelId from session.json, so v2 sessions are not
  mislabeled kiro-auto
- Extract parseWorkspaceSession from the inline createParser block for
  parity with the other four format parsers
- Add v2SessionsRootOverride and stop deriving the v2 scan root from
  the cliDir parent when only agentDirOverride is set (tests were
  scanning the system tmpdir)
- Tests: v2 parsing/discovery/dedup/routing, credit vs fallback
  pricing, tool_result turn scoping, reasoning disjointness, cli-dir
  skip guard, and a mixed-format coexistence suite (legacy .chat + v1
  executions + workspace-sessions + CLI + v2 on one machine) asserting
  exact aggregate counts and no double counting
- Docs: v2 store layout, credit pricing, dedup namespaces, and the
  disjoint-store verification (v1->v2 is a clean cutover, no
  migration/dual-write observed)

AI-Origin: human
providerCallToCachedCall drops provider-computed costUSD unless the
provider is allowlisted, and cachedCallToApiCall then re-prices from
token counts. Kiro's cost now comes from metered credits, which token
pricing cannot reproduce — reports were understating real kiro spend
~18x (e.g. $130.72 of metered CLI credits reported as $7.20 of token
estimates). Add kiro to the pass-through allowlist, same as the other
platform-billed providers (mistral-vibe, devin, hermes, ...).

Both persistence layers froze pre-fix dollar values and neither
self-invalidates (historical session files never change), so bump:

- session cache: CACHE_VERSION 4 -> 5 (cached kiro entries carry
  costUSD: undefined and would keep being token re-priced forever)
- daily cache: DAILY_CACHE_VERSION 10 -> 11 with MIN_SUPPORTED_VERSION
  raised (finalized days carry token-estimated kiro costs off by up to
  16x per model), same as the v10 cursor precedent

Test: end-to-end through parseAllSessions — a CLI fixture with 2.5
metered credits must report $0.10; fails with the token re-price
($0.000048) when the allowlist entry is removed.

Docs: document the pricing contract in docs/providers/kiro.md — metered
credit cost is frozen at parse time, so price-override / model-alias
do not affect kiro dollar amounts (shared tradeoff of all allowlisted
providers).

AI-Origin: ai-generated
AI-Tool: kiro
v1 modern execution files carry the same metered credits as v2
(usageSummary[].usage, unit "credit" — the predecessor of v2's
promptTurnSummaries), but the parser only harvested usedTools from
that array and priced executions from estimated tokens. Since v2 only
ships in brand-new IDE builds, v1 is the format nearly all Kiro IDE
users are on today. Sum usage across usageSummary entries and price at
the public overage rate.

Align all three credit parsers (CLI, v1, v2) on one fallback contract —
gate on summed credits > 0 and fall back to token-estimated pricing
with costIsEstimated: true:

- CLI turns without metering_usage were priced at a frozen $0
- CLI turns with an EMPTY metering_usage array (75 of 10,105 real turn
  metadatas — meta written before metering lands) passed the truthy
  presence check and froze $0 marked as real cost
- legacy .chat calls now carry costIsEstimated: true, which was always
  the reality but never stated

Validated against a real machine: 221 of 277 v1/legacy calls now carry
metered cost ($20.11) instead of token estimates that had overstated
them ~5x; verified usageSummary values are per-turn amounts, not
cumulative counters (sequences fluctuate). Call counts unchanged before
and after — nothing gained, lost, or double-counted.

Also document the companion-file fingerprint blind spot at
fingerprintFile: kiro CLI credits and v2 modelId live in companion
files the single-file fingerprint never sees, so a parse racing the
companion write can cache fallback values that only self-heal while
the transcript keeps changing.

AI-Origin: ai-generated
AI-Tool: kiro
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants