feat(uploads): 'panda upload' — private preview, publish on click#272
Open
qu0b wants to merge 6 commits into
Open
feat(uploads): 'panda upload' — private preview, publish on click#272qu0b wants to merge 6 commits into
qu0b wants to merge 6 commits into
Conversation
Adds a top-level 'panda upload <file>...' command that streams a file to the proxy's new /uploads route and prints a durable public URL. The proxy holds the R2 (S3-compatible) credentials and does the PutObject; the server only forwards bytes and caller attribution, so credentials never leave the trust boundary. Objects are content-addressed (uploads/<6-hex-sha256>/<filename>) so identical bytes dedup and every URL is immutable. Enabled by an optional 'uploads:' block in proxy-config; no-op when unset.
Contributor
🐼 Smoke eval —
|
| question | result | tokens | tools |
|---|---|---|---|
forky_node_coverage |
✅ | 14,997 | 4 |
tracoor_node_coverage |
✅ | 13,662 | 4 |
mainnet_block_arrival_p50 |
✅ | 15,651 | 7 |
list_datasources |
✅ | 12,809 | 2 |
block_count_24h |
✅ | 15,512 | 7 |
missed_slots_24h |
✅ | 14,871 | 6 |
chartkit_default_arrival_distribution |
✅ | 43,749 | 14 |
storage_upload_session_scoped |
✅ | 20,283 | 16 |
🔭 Langfuse traces (8 runs; ⚠️ = failed)
The report walks this branch's commits against the master baseline and the most recent release. A self-contained copy is in the run's eval-smoke-* artifact.
- Dedicated per-user upload rate limiter (60/min, burst 20) on /uploads, keyed on the authenticated subject and layered on the generic request limiter. - Sanitize the object filename (strip path parts, restrict to [A-Za-z0-9._-], length-cap, preserve extension). - Force Content-Disposition: attachment for scriptable types (html/svg/js) so a public bucket can't serve phishing/XSS inline; images/pdf/text stay inline. - Label upload rate-limit rejections as 'uploads' in metrics. - Tests for name sanitization and inline-safe classification.
Upload no longer publishes immediately. Files are held in an in-memory,
session-only store in the local server (nothing hits disk or leaves the machine),
and 'panda upload' opens a preview page with a 'Make public' button. Publishing
is now a deliberate act (the button, or --public for scripts).
- pkg/server/uploads.go: in-memory upload store (bounded, session lifetime),
POST /api/v1/uploads, preview page GET /u/{id}, raw GET /u/{id}/raw, and
POST /api/v1/uploads/publish (streams stored bytes to the proxy).
- Preview raw bytes are served with a sandbox CSP + sandboxed iframe so a
malicious HTML upload can't execute in the local server origin.
- CLI: preview-by-default, opens browser; --public / --no-open flags.
- Public objects get a 60-day retention (R2 lifecycle, platform side).
- Tests for the store, handlers, and CSP.
Reuse the shared public bucket but publish under panda/uploads/ (not uploads/) so the path identifies the source. Docs + example config only; the prefix is config-driven, no code change.
handleUploadPublish hand-rolled its proxy call with http.DefaultClient, so it skipped the 401/403 invalidate-and-retry, sent no panda/<version> User-Agent, and swallowed the response-body read error. Route it through s.proxyRequest instead — same treatment as every other proxy-bound request, and ~25 fewer lines. Addresses review on pkg/server/uploads.go.
…m scheme Follow-ups in the spirit of the review (bound things, don't hard-code assumptions): - In-memory preview store now evicts by total bytes (256 MiB) as well as count (32), so worst-case RAM is capped regardless of upload size — previously 32 x 100 MiB = 3.2 GB. Limits are struct fields so they're testable. - Proxy R2 client derives Secure from the endpoint scheme instead of hard-coding TLS, so a local/dev http endpoint works. - Tests for total-bytes eviction and the single-oversized-item guard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
panda upload <file>gives you a private, in-browser preview of a file. Click Make public (or pass--public) to promote it to R2 and get a durable link. Private previews never leave your machine — publishing is always a deliberate choice.The flow
The preview page renders the file and shows a Make public button → durable link:
Other forms:
--publicskips the preview and prints the URL (scripts/CI);-reads stdin (with--name);--no-openwon't launch a browser.How it works
Bytes flow CLI → local server → proxy → R2, and that split is what keeps it safe:
/uploadsroute-publicR2 bucket,panda/uploads/prefixpanda/uploads/<sha6>/<name>; served at data.ethpandaops.io; 60-day lifecycleNo new MCP tool — the 3-tool contract is untouched.
Files
pkg/cli/upload.go— theuploadcommand (--public,--no-open, stdin).pkg/server/uploads.go— in-memory store +POST /api/v1/uploads, preview pageGET /u/{id}, rawGET /u/{id}/raw,POST /api/v1/uploads/publish.pkg/proxy/handlers/uploads.go(+/uploadsroute,uploads:config) — the credentialed R2PutObject.Safety
[A-Za-z0-9._-], length-capped.Content-Disposition: attachment.sandboxCSP + sandboxed<iframe>, so previewing a malicious HTML upload can't run scripts in the local server's origin.To go live
Needs the companion ethpandaops/platform#365 (R2 writer key, the
uploads:proxy config, the 60-daypanda/uploads/lifecycle) plus a proxy image bump once this merges. Until then the route is simply absent.Tests
go build ./...,go vet,gofmt, and the pre-commit golangci-lint all pass clean. New unit tests cover the proxy handler (filename sanitization, inline-safe classification, guards) and the server (store eviction, upload/publish handlers, sandbox CSP). Existingpkg/proxy/pkg/server/pkg/storage/pkg/clisuites pass.Design notes:
docs/proposals/assets-cli.md.