Calcite Janino compilation fails when analytics-engine is parent classloader via extendedPlugins

[ahkcs]

Summary

When analytics-engine is configured as the parent plugin via extendedPlugins, Calcite's Janino runtime compilation fails with CompileException: Cannot determine simple type name "org". This occurs because Janino uses the parent classloader (analytics-engine) to compile generated Java code that references SQL plugin classes (child classloader), and the parent cannot see child classes.

Environment

• Plugin configuration: extendedPlugins = ['opensearch-job-scheduler', 'analytics-engine']

Classloader Hierarchy

analytics-engine (PARENT classloader)
├── calcite-core-1.41.0.jar (Calcite + Janino)
├── analytics-framework.jar (QueryPlanExecutor interface)
│
└── opensearch-sql (CHILD classloader)
    ├── core, ppl, sql, opensearch modules
    ├── CalciteEnumerableIndexScan, UserDefinedFunctionBuilder, etc.
    └── (Calcite JARs excluded from bundle to avoid jar hell)

Root Cause

Calcite generates Java source code at runtime and compiles it using Janino. In EnumerableInterpretable.getBindable(), the classloader is hardcoded:

static Bindable getBindable(ClassDeclaration expr, String s, int fieldCount) {
    ClassLoader classLoader = EnumerableInterpretable.class.getClassLoader(); // parent classloader
    ICompilerFactory compilerFactory = CompilerFactoryFactory.getDefaultCompilerFactory(classLoader);
    ISimpleCompiler compiler = compilerFactory.newSimpleCompiler();
    compiler.setParentClassLoader(classLoader); // Janino uses PARENT classloader
    compiler.cook(generatedCode); // generated code references CHILD classes → CRASH
}

The generated code references SQL plugin classes (e.g., CalciteEnumerableIndexScan), but Janino compiles with the parent classloader, which cannot see child classes.

Affected Code Paths

This same pattern exists in 4 places inside Calcite:

Code Path	Purpose
EnumerableInterpretable.getBindable()	Enumerable plan code generation (UNION ALL, cross-cluster, etc.)
RexExecutable.compile()	Pushdown script compilation (parse, grok commands)
RexExecutorImpl.reduce()	Constant folding during planning
Prepare.optimize()	Overwrites custom RexExecutor before optimization

Steps to Reproduce

1. Configure SQL plugin with extendedPlugins = ['analytics-engine']
2. Exclude Calcite JARs from SQL plugin bundle (to avoid jar hell with parent)
3. Run any PPL query that triggers Enumerable code generation:

   source=accounts | stats sum(age) by gender | append [ source=accounts | stats count(age) by gender ]
   

Error

org.codehaus.commons.compiler.CompileException: Line 43, Column 3: Cannot determine simple type name "org"
	at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7144)
	at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7023)
	at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7036)

The generated Java code that fails to compile:

public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
  // Line 43 - this class is in SQL plugin (child classloader), not visible to Janino
  final org.opensearch.sql.opensearch.storage.scan.CalciteEnumerableIndexScan v1stashed = 
      (org.opensearch.sql.opensearch.storage.scan.CalciteEnumerableIndexScan) root.get("v1stashed");
  ...
}

Full call chain:

PPL query (append -> UNION ALL)
  -> VolcanoPlanner.findBestExp()
    -> EnumerableInterpretable.toBindable()
      -> EnumerableInterpretable.getBindable()
        -> ClassLoader cl = EnumerableInterpretable.class.getClassLoader()  // PARENT
        -> compiler.setParentClassLoader(cl)                                // Janino uses PARENT
        -> compiler.cook(generatedCode)                                     // code refs CHILD classes
        -> CompileException: "Cannot determine simple type name 'org'"

Why extendedPlugins Works for Other Parents (e.g., job-scheduler) but Not analytics-engine

The parent-child classloader works fine for normal Java method calls — the child can see parent classes and call methods on them directly. For example, the SQL plugin extends job-scheduler via extendedPlugins without any issues.

The problem is exclusively with Calcite's Janino runtime compilation:

• Calcite generates Java source code strings at runtime
• Those strings contain SQL plugin class names like CalciteEnumerableIndexScan
• Janino compiles those strings using the parent classloader (EnumerableInterpretable.class.getClassLoader())
• The parent classloader (analytics-engine) cannot see SQL plugin classes (child)
• Janino fails to resolve org.opensearch.sql.* classes → CompileException

Most plugins never do runtime code generation, so extendedPlugins works fine for them. Calcite's use of Janino is a unique behavior that breaks the parent-child classloader assumption.