Skip to content

ENG-87659 - convert blobs to zstd from gzip before push#366

Open
narasul wants to merge 5 commits into
mainfrom
push-as-zstd
Open

ENG-87659 - convert blobs to zstd from gzip before push#366
narasul wants to merge 5 commits into
mainfrom
push-as-zstd

Conversation

@narasul
Copy link
Copy Markdown
Contributor

@narasul narasul commented May 13, 2026

10% to 35% pull time improvement on some images, depending on cluster(how good network is).

What changed:

  • if buildx is available create a buildkit builder
  • jig build builds the image with zstd compression instead of default gzip
  • jig push pushes the image that was built with the the buildkit
  • jig deploy builds the image with buildx and directly pushes to registry(won't be locally available if not pulled)
  • --warmup flag makes builds fallback to default gzip build(it is possible to make it work with buildx but that's future work if we want to make it nicely)

(I need to clean up the code but please comment on the flow, see if I missed an edge case or two)

@narasul narasul marked this pull request as draft May 13, 2026 14:01
@narasul narasul marked this pull request as ready for review May 13, 2026 18:55
@mintlify
Copy link
Copy Markdown

mintlify Bot commented May 13, 2026

Docs PR opened: https://github.com/togethercomputer/mintlify-docs/pull/847

Added a tip to the jig push CLI reference documenting the new Buildx + zstd push behavior and gzip fallback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant