Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,19 @@ Run `codeburn` for the dashboard, or use a subcommand below. Most commands also
| `codeburn export` | CSV covering today, 7 days, and 30 days |
| `codeburn export -f json` | Export as JSON instead of CSV |

**Sync (team telemetry)** _preview_

| Command | What it does |
|---------|--------------|
| `codeburn sync setup <url>` | One-time setup: OIDC login via browser, stores token securely |
| `codeburn sync push` | Push unsent usage to remote endpoint (default: last 7 days) |
| `codeburn sync push --since 30d` | Push a larger window |
| `codeburn sync status` | Show endpoint, auth state, last sync time |
| `codeburn sync logout` | Revoke token and remove credentials |
| `codeburn sync reset --confirm` | Clear sent-ledger (re-send all data on next push) |

Sync sends token counts, costs, models, and projects — never prompts or code. This feature is in preview; the protocol may change between releases. See [docs/sync/](docs/sync/) for details.

**Web & devices**

| Command | What it does |
Expand Down
215 changes: 215 additions & 0 deletions docs/sync/DEVELOPER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
# Sync — Developer Documentation

Architecture, protocol, server contract, and testing for `codeburn sync`.

## Architecture

```
Developer machine Remote backend
────────────────── ──────────────
~/.config/codeburn/sync.json (config)
~/.config/codeburn/.sync-token (credential)
~/.cache/codeburn/sync-ledger.json (sent-ledger)

codeburn sync push
├─ Read config → baseUrl, clientId, issuer, tracesPath
├─ Read refresh token from OS store
├─ POST {issuer}/oauth2/token (refresh_token grant) → access_token
├─ Collect ParsedProviderCall[] for window
├─ Filter against sent-ledger (only unsent calls)
├─ Build OTLP/HTTP JSON payload
├─ POST {baseUrl}{tracesPath} with Bearer token
├─ On success → append deduplicationKeys to ledger
└─ Update lastSync in config
```

## Discovery Protocol

### Server discovery document

```
GET {baseUrl}/.well-known/codeburn-export.json
```

```json
{
"version": 1,
"issuer": "https://cognito-idp.us-west-2.amazonaws.com/us-west-2_XXXX",
"client_id": "70e6sgst2ju6ff9dnrmv4l1tcb",
"scopes": ["openid", "email"],
"traces_path": "/v1/traces",
"max_batch_size": 1000
}
```

| Field | Required | Default | Description |
|---|---|---|---|
| `version` | No | `1` | Client rejects `version > 1` |
| `issuer` | Yes | — | OIDC issuer URL. Client fetches `{issuer}/.well-known/openid-configuration` |
| `client_id` | Yes | — | OAuth client ID for this deployment |
| `scopes` | No | `["openid"]` | Scopes to request. `offline_access` added dynamically if IdP supports it |
| `traces_path` | No | `/v1/traces` | Path for OTLP POST |
| `max_batch_size` | No | `1000` | Max spans per HTTP request |

### Why not proxy `.well-known/openid-configuration`?

OIDC requires the `issuer` claim inside the discovery doc to match the URL it was fetched from. Serving Cognito's doc from a different domain violates this constraint. The `codeburn-export.json` doc decouples the metrics endpoint from the identity provider.

## OIDC Authentication

### Flow: Authorization Code + PKCE

1. Client generates `code_verifier` (32 random bytes, base64url) and `code_challenge` (SHA-256 of verifier, base64url)
2. Client starts callback server on `127.0.0.1:19876` (fallback: 19877, 19878)
3. Browser opens `{authorization_endpoint}?response_type=code&client_id=...&redirect_uri=http://127.0.0.1:{port}/callback&code_challenge=...&code_challenge_method=S256&state=...&scope=...`
4. User logs in at IdP → IdP redirects to `http://127.0.0.1:{port}/callback?code=...&state=...`
5. Callback server validates `state`, extracts `code`
6. Client POSTs to `{token_endpoint}` with `grant_type=authorization_code`, `code`, `code_verifier`, `redirect_uri`, `client_id`
7. IdP returns `access_token` + `refresh_token`

### Fixed ports

Cognito (and Okta) do exact string comparison on callback URLs. Ephemeral ports fail. We register three fixed ports: `19876`, `19877`, `19878`. The client tries in order, falling back if a port is in use.

RFC 8252 recommends `127.0.0.1` (IP literal) over `localhost` to avoid IPv6 `::1` resolution.

### Token refresh

On every `sync push`:
1. Read refresh token from OS store
2. POST `{token_endpoint}` with `grant_type=refresh_token`
3. Store whatever refresh token the server returns (handles rotation transparently)
4. On `invalid_grant` → stop, prompt user to re-run `sync setup`

### Scope resolution

- Request scopes from `codeburn-export.json`
- Add `offline_access` only if `scopes_supported` in OIDC discovery includes it
- Cognito rejects `offline_access` as `invalid_scope` — it issues refresh tokens without it

## Credential Storage

| Platform | Method | Implementation |
|---|---|---|
| macOS | Keychain | `security add-generic-password` / `find-generic-password` |
| Linux | libsecret | `secret-tool store` / `secret-tool lookup` |
| Windows | DPAPI | PowerShell `ConvertTo-SecureString` / `ConvertFrom-SecureString` |
| Fallback | File | `~/.config/codeburn/.sync-token` with `0600` permissions |

No native modules (`keytar` is archived). Shell out to OS CLIs. Fallback reported honestly in `sync status`.

## OTLP Encoding

Strict protobuf-JSON mapping of `ExportTraceServiceRequest`. lowerCamelCase fields, hex-encoded IDs, integer enums.

### Span identity (deterministic)

```
span_id = first 8 bytes of SHA-256(deduplicationKey) → hex (16 chars)
trace_id = first 16 bytes of SHA-256(sessionId) → hex (32 chars)
```

Re-sends are byte-identical. Server-side dedup is defense-in-depth.

### Resource attributes

```json
{
"resource": {
"attributes": [
{ "key": "codeburn.device_id", "value": { "stringValue": "<SHA-256(hostname+username)[:16]>" } }
]
}
}
```

### Span attributes

```json
{
"attributes": [
{ "key": "ai.provider", "value": { "stringValue": "kiro" } },
{ "key": "ai.model", "value": { "stringValue": "claude-sonnet-4-6" } },
{ "key": "ai.input_tokens", "value": { "intValue": "12500" } },
{ "key": "ai.output_tokens", "value": { "intValue": "3200" } },
{ "key": "ai.cost_usd", "value": { "doubleValue": 0.085 } },
{ "key": "ai.project", "value": { "stringValue": "my-app" } },
{ "key": "ai.tools", "value": { "arrayValue": { "values": [{ "stringValue": "Edit" }] } } },
{ "key": "ai.speed", "value": { "stringValue": "standard" } },
{ "key": "ai.cost_estimated", "value": { "boolValue": true } }
]
}
```

## Sent-Ledger

Client-side deduplication source of truth at `~/.cache/codeburn/sync-ledger.json`.

Format: JSON array of `{ key: string, ts: string }` objects.

**Push logic**: collect all calls in window → subtract ledger entries → send remainder → append to ledger on success.

**Pruning**: entries older than 6 months removed on every push.

**Why not a watermark?** Timestamp watermarks silently skip late-arriving calls (long sessions, providers that update rows). The ledger is exact.

### Partial success

OTLP returns `partial_success.rejected_spans` in the response body. Because OTLP does not identify *which* spans were rejected, the client ledgers nothing for a partially-rejected batch — the entire batch retries on the next push. This is safe: span IDs are deterministic (derived from the deduplication key), so servers that store by span ID treat re-sent spans as idempotent upserts.

### Rate limiting (429)

A push runs to completion — there is no routine per-push cap (only a 50,000-call safety valve). Server rate limits are the intended brake:

- On HTTP 429 the client honors `Retry-After` (delta-seconds or HTTP-date), capped at 120 seconds per wait, defaulting to 5 seconds when the header is absent
- The same batch is retried up to 3 consecutive times; if the server is still rate-limiting after that, the push stops and the remaining (unledgered) calls are sent on the next push
- On 401 or 5xx the push stops immediately with the same resume-on-next-push behavior

### Server contract

The backend must implement:

1. `GET {baseUrl}/.well-known/codeburn-export.json` — returns the discovery doc (public, no auth)
2. `POST {baseUrl}{traces_path}` — accepts OTLP/HTTP JSON with Bearer token
- Validate JWT (issued by the configured IdP)
- Derive developer identity from token's `sub` claim
- Accept `startTimeUnixNano` up to 6 months in the past
- Return standard OTLP response body

No PII is included in the payload. The server derives identity solely from the authenticated token.

## Testing

### Unit tests (`tests/sync.test.ts`)

26 tests covering pure functions: discovery parsing, PKCE generation, auth URL construction, scope resolution, callback server, config read/write. No network, no browser.

### Mock IdP e2e (`tests/sync-e2e.test.ts`)

6 tests with a localhost mock IdP server. Exercises the full auth round-trip, token refresh, rotation, revocation — fully offline, runs in CI.

### Headless browser e2e (`tests/sync-headless-e2e.test.ts`)

1 test with Playwright headless Chromium against real Cognito. Proves the actual browser PKCE flow works including Cognito Hosted UI form submission and localhost redirect.

**Developer-only** — requires:
- Deployed test backend (CDK stack at `../codeburn-sync-backend/`)
- Cognito user with confirmed password
- Environment variables: `CODEBURN_SYNC_URL`, `CODEBURN_SYNC_EMAIL`, `CODEBURN_SYNC_PASSWORD`
- Playwright Chromium installed (`PLAYWRIGHT_BROWSERS_PATH`)

Skipped by default when env vars are not set. Never runs in CI.

### Test CDK stack (`codeburn-sync-backend/`)

Minimal AWS backend for the headless e2e test:
- Cognito User Pool (PKCE, fixed callback ports)
- HTTP API with JWT authorizer
- Discovery Lambda (serves `codeburn-export.json`)
- Ingest Lambda (logs OTLP spans to CloudWatch)

Deploy: `npx cdk deploy --profile andklee-dev`
Cost: ~$0/mo idle (pay-per-request)

This is a **test fixture**, not a production reference. Any OIDC provider + OTLP-accepting endpoint satisfies the server contract.
130 changes: 130 additions & 0 deletions docs/sync/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# codeburn sync

Push your AI usage telemetry to a shared backend so teams can track adoption, budgets, and ROI across developers.

Everything stays local-first: codeburn never sends data without your explicit action, and prompts/code are never included.

## Quick Start

```bash
# One-time setup (opens browser for login)
codeburn sync setup https://metrics.your-team.com

# Push recent usage
codeburn sync push

# Check status
codeburn sync status
```

## Commands

### `codeburn sync setup <url>`

Configures sync with a remote endpoint. Opens your browser for a one-time OIDC login.

```bash
codeburn sync setup https://metrics.your-team.com
```

What happens:
1. Fetches server configuration from `<url>/.well-known/codeburn-export.json`
2. Opens your browser to the identity provider's login page
3. After login, stores a refresh token securely in your OS keychain
4. Saves the endpoint configuration (no secrets) to `~/.config/codeburn/sync.json`

You only need to do this once. The token refreshes silently on every push.

### `codeburn sync push`

Sends unsent AI usage data to the configured endpoint.

```bash
# Push unsent calls from the last 7 days (default)
codeburn sync push

# Push a larger window
codeburn sync push --since 30d

# Preview what would be sent
codeburn sync push --dry-run
```

### `codeburn sync status`

Shows the current sync configuration and authentication state.

```
Endpoint: https://metrics.your-team.com
Traces path: /v1/traces
Issuer: https://auth.your-team.com
Auth: configured
Token storage: keychain
Last sync: 2h ago
```

### `codeburn sync logout`

Removes stored credentials and revokes the token at the identity provider.

```bash
codeburn sync logout
```

### `codeburn sync reset --confirm`

Clears the sent-ledger, causing the next push to re-send all data in the window. Use after a backend migration or if you suspect missing data.

```bash
codeburn sync reset --confirm
```

## What Gets Sent

Each AI interaction becomes one OTLP span with these attributes:

| Field | Example | Description |
|---|---|---|
| `ai.provider` | `kiro`, `cursor`, `claude` | Which AI tool |
| `ai.model` | `claude-sonnet-4-6` | Model used |
| `ai.input_tokens` | `12500` | Prompt tokens |
| `ai.output_tokens` | `3200` | Response tokens |
| `ai.cost_usd` | `0.085` | Estimated cost |
| `ai.project` | `my-app` | Project name |
| `ai.tools` | `["Edit", "Bash"]` | Tools invoked |

A pseudonymous `device_id` distinguishes your machines without revealing hostnames.

### What is NOT sent

- **Prompts** — your actual messages to AI are never included
- **Code** — file contents, diffs, and paths stay local
- **Bash commands** — may contain secrets, never sent
- **Your name/email** — identity is derived server-side from your login token

There is no flag to override this. Privacy is structural, not configurable.

## Authentication

Sync uses standard OIDC (the same protocol as "Sign in with Google"). Your team's admin sets up the identity provider — you just click through the browser login once.

- **Token storage**: macOS Keychain, Windows Credential Manager, or Linux libsecret. Falls back to a `0600` file if no keychain is available.
- **Token lifetime**: typically 30–90 days (set by your admin). You'll be prompted to re-login when it expires.
- **Re-login**: run `codeburn sync setup <url>` again.

## FAQ

**Q: Does sync run automatically?**
A: No. You run `codeburn sync push` when you want. A future version may offer opportunistic push (after each `codeburn report`), but it's always explicit.

**Q: What if I push the same data twice?**
A: Safe. A local sent-ledger tracks what's been sent. Re-pushing the same window doesn't create duplicates.

**Q: What if I'm offline for a week?**
A: Next push catches up. The default window is 7 days; use `--since 30d` or `--since all` (up to 6 months) for longer gaps. A push runs to completion regardless of size — server rate limits (429) are waited out automatically.

**Q: Can my admin see my prompts?**
A: No. Prompts are never included in the payload. The server only sees token counts, costs, model names, and project names.

**Q: How do I stop syncing?**
A: `codeburn sync logout` removes everything. Or just stop running `push`.
Loading
Loading