ENG-87659 - convert blobs to zstd from gzip before push#366
Open
narasul wants to merge 5 commits into
Open
Conversation
|
Docs PR opened: https://github.com/togethercomputer/mintlify-docs/pull/847 Added a tip to the jig push CLI reference documenting the new Buildx + zstd push behavior and gzip fallback. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
10% to 35% pull time improvement on some images, depending on cluster(how good network is).
What changed:
buildxis available create a buildkit builderjig buildbuilds the image with zstd compression instead of default gzipjig pushpushes the image that was built with the the buildkitjig deploybuilds the image with buildx and directly pushes to registry(won't be locally available if not pulled)--warmupflag makes builds fallback to default gzip build(it is possible to make it work with buildx but that's future work if we want to make it nicely)(I need to clean up the code but please comment on the flow, see if I missed an edge case or two)