Skip to content

feat(uploads): 'panda upload' — private preview, publish on click#272

Open
qu0b wants to merge 6 commits into
masterfrom
qu0b/panda-assets-cli
Open

feat(uploads): 'panda upload' — private preview, publish on click#272
qu0b wants to merge 6 commits into
masterfrom
qu0b/panda-assets-cli

Conversation

@qu0b

@qu0b qu0b commented Jul 1, 2026

Copy link
Copy Markdown
Member

TL;DR

panda upload <file> gives you a private, in-browser preview of a file. Click Make public (or pass --public) to promote it to R2 and get a durable link. Private previews never leave your machine — publishing is always a deliberate choice.

The flow

$ panda upload report.html
http://localhost:2480/u/6f1c…      # private preview opens in your browser
private preview (session-only) — click "Make public" to publish

The preview page renders the file and shows a Make public button → durable link:

https://data.ethpandaops.io/panda/uploads/9f3ca1/report.html   (expires in 60 days)

Other forms: --public skips the preview and prints the URL (scripts/CI); - reads stdin (with --name); --no-open won't launch a browser.

How it works

Bytes flow CLI → local server → proxy → R2, and that split is what keeps it safe:

Stage Where it lives Notes
Private preview local server, in memory session-only; bounded store, evicted oldest-first; nothing on disk, nothing remote
Publish proxy /uploads route the proxy holds the R2 credentials; the server only forwards the stored bytes + caller attribution
Public object shared -public R2 bucket, panda/uploads/ prefix content-addressed panda/uploads/<sha6>/<name>; served at data.ethpandaops.io; 60-day lifecycle

No new MCP tool — the 3-tool contract is untouched.

Files

  • pkg/cli/upload.go — the upload command (--public, --no-open, stdin).
  • pkg/server/uploads.go — in-memory store + POST /api/v1/uploads, preview page GET /u/{id}, raw GET /u/{id}/raw, POST /api/v1/uploads/publish.
  • pkg/proxy/handlers/uploads.go (+ /uploads route, uploads: config) — the credentialed R2 PutObject.

Safety

  • Publishing is authenticated + rate-limited — org-gated at token issuance, plus a dedicated per-user limiter (60/min, burst 20) on top of the generic one.
  • Size cap — 100 MiB on both the in-memory buffer and the R2 write.
  • Filename sanitization — path stripped, restricted to [A-Za-z0-9._-], length-capped.
  • No inline scripts from the public bucket — html/svg/js get Content-Disposition: attachment.
  • Safe preview — raw bytes served with a sandbox CSP + sandboxed <iframe>, so previewing a malicious HTML upload can't run scripts in the local server's origin.

To go live

Needs the companion ethpandaops/platform#365 (R2 writer key, the uploads: proxy config, the 60-day panda/uploads/ lifecycle) plus a proxy image bump once this merges. Until then the route is simply absent.

Tests

go build ./..., go vet, gofmt, and the pre-commit golangci-lint all pass clean. New unit tests cover the proxy handler (filename sanitization, inline-safe classification, guards) and the server (store eviction, upload/publish handlers, sandbox CSP). Existing pkg/proxy / pkg/server / pkg/storage / pkg/cli suites pass.

Design notes: docs/proposals/assets-cli.md.

Adds a top-level 'panda upload <file>...' command that streams a file to the
proxy's new /uploads route and prints a durable public URL. The proxy holds the
R2 (S3-compatible) credentials and does the PutObject; the server only forwards
bytes and caller attribution, so credentials never leave the trust boundary.

Objects are content-addressed (uploads/<6-hex-sha256>/<filename>) so identical
bytes dedup and every URL is immutable. Enabled by an optional 'uploads:' block
in proxy-config; no-op when unset.
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

🐼 Smoke eval — e9ea050: ✅ 8/8 pass

📊 Interactive report — tokens p50 15,254 · tokens/solve 18,942.

Reference points: qu0b/panda-assets-cli@11dd1f4 100% · qu0b/panda-assets-cli@bc0bfb9 100% · qu0b/panda-assets-cli@e6c1b3c 100%.

question result tokens tools
forky_node_coverage 14,997 4
tracoor_node_coverage 13,662 4
mainnet_block_arrival_p50 15,651 7
list_datasources 12,809 2
block_count_24h 15,512 7
missed_slots_24h 14,871 6
chartkit_default_arrival_distribution 43,749 14
storage_upload_session_scoped 20,283 16
🔭 Langfuse traces (8 runs; ⚠️ = failed)

The report walks this branch's commits against the master baseline and the most recent release. A self-contained copy is in the run's eval-smoke-* artifact.

qu0b added 3 commits July 1, 2026 10:55
- Dedicated per-user upload rate limiter (60/min, burst 20) on /uploads, keyed
  on the authenticated subject and layered on the generic request limiter.
- Sanitize the object filename (strip path parts, restrict to [A-Za-z0-9._-],
  length-cap, preserve extension).
- Force Content-Disposition: attachment for scriptable types (html/svg/js) so a
  public bucket can't serve phishing/XSS inline; images/pdf/text stay inline.
- Label upload rate-limit rejections as 'uploads' in metrics.
- Tests for name sanitization and inline-safe classification.
Upload no longer publishes immediately. Files are held in an in-memory,
session-only store in the local server (nothing hits disk or leaves the machine),
and 'panda upload' opens a preview page with a 'Make public' button. Publishing
is now a deliberate act (the button, or --public for scripts).

- pkg/server/uploads.go: in-memory upload store (bounded, session lifetime),
  POST /api/v1/uploads, preview page GET /u/{id}, raw GET /u/{id}/raw, and
  POST /api/v1/uploads/publish (streams stored bytes to the proxy).
- Preview raw bytes are served with a sandbox CSP + sandboxed iframe so a
  malicious HTML upload can't execute in the local server origin.
- CLI: preview-by-default, opens browser; --public / --no-open flags.
- Public objects get a 60-day retention (R2 lifecycle, platform side).
- Tests for the store, handlers, and CSP.
Reuse the shared public bucket but publish under panda/uploads/ (not uploads/)
so the path identifies the source. Docs + example config only; the prefix is
config-driven, no code change.
@qu0b qu0b changed the title feat(uploads): add 'panda upload' for durable shareable file URLs feat(uploads): 'panda upload' — private preview, publish on click Jul 1, 2026
qu0b added 2 commits July 1, 2026 12:00
handleUploadPublish hand-rolled its proxy call with http.DefaultClient, so it
skipped the 401/403 invalidate-and-retry, sent no panda/<version> User-Agent,
and swallowed the response-body read error. Route it through s.proxyRequest
instead — same treatment as every other proxy-bound request, and ~25 fewer lines.

Addresses review on pkg/server/uploads.go.
…m scheme

Follow-ups in the spirit of the review (bound things, don't hard-code assumptions):
- In-memory preview store now evicts by total bytes (256 MiB) as well as count
  (32), so worst-case RAM is capped regardless of upload size — previously 32 x
  100 MiB = 3.2 GB. Limits are struct fields so they're testable.
- Proxy R2 client derives Secure from the endpoint scheme instead of hard-coding
  TLS, so a local/dev http endpoint works.
- Tests for total-bytes eviction and the single-oversized-item guard.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant