[MINOR] Build shaded interpreter runtime in zeppelin-interpreter - #5374
Draft
jongyoul wants to merge 1 commit into
Draft
[MINOR] Build shaded interpreter runtime in zeppelin-interpreter#5374jongyoul wants to merge 1 commit into
jongyoul wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
zeppelin-interpreterpackage phase while keeping the normalzeppelin-interpreterJAR as the module's only installed/deployed Maven artifact.zeppelin-interpreter-shadedreactor module and its provided dependency fromzeppelin-interpreter-parent.interpreter/, then remove stale versions with a narrowly scoped clean rule.Why
The separate shaded module creates a two-step rebuild chain that can leave interpreter processes running a stale runtime JAR. Producing both outputs in one module makes a change to
zeppelin-interpreterand its runtime JAR part of the same Maven lifecycle without replacing or attaching the public API artifact.The default runtime JAR is reduced from 9,787,645 bytes to 7,853,798 bytes (1,933,847 bytes / 19.76%) while retaining the existing runtime filename, path, relocations, and optional
include-hadoopbehavior.Compatibility
interpreter/zeppelin-interpreter-shaded-*.jar.org.apache.zeppelin:zeppelin-interpreter-shadedis no longer published. Custom interpreters should useorg.apache.zeppelin:zeppelin-interpreterwithprovidedscope and directly declare libraries they use instead of relying on the removed transitive dependencies.Verification
./mvnw clean test -pl zeppelin-interpreter --am— 126 tests passed../mvnw test -pl zeppelin-jupyter-interpreter— 4 kernel tests passed with an isolated compatible Python environment../mvnw package -pl zeppelin-server --am -Dtest=ZeppelinConfigurationTest -Dsurefire.failIfNoSpecifiedTests=false— 14 tests passed../mvnw package -pl spark/scala-2.12 --am -Pspark-3.5 -Pspark-scala-2.12 -DskipTests../mvnw clean package -pl zeppelin-interpreter --am -Pinclude-hadoop -DskipTests../mvnw clean package -Pbuild-distr -DskipTestsand the same build with-Pinclude-hadoop../mvnw clean org.apache.rat:apache-rat-plugin:check -Prat../mvnw validate -pl zeppelin-integration --am -Pintegration.installverified that only the normal interpreter JAR/POM is installed; the runtime JAR remains distribution-internal.