feat(cli): add lakebench CLI (run/results/report/discover/doctor/profiles)#85
Draft
tomz wants to merge 3 commits into
Draft
feat(cli): add lakebench CLI (run/results/report/discover/doctor/profiles)#85tomz wants to merge 3 commits into
tomz wants to merge 3 commits into
Conversation
Add a conservative ruff config (E/F/I/W), a pre-commit hook set, and a CI lint job (lint + format both enforcing). Reformat the existing tree with `ruff format` and replace ad-hoc print() diagnostics with module-level loggers across datagen and timing helpers. Fix obvious nits surfaced by ruff (import order, bare except -> except Exception, dead assignment). Drop Python 3.8 support and move pyarrow to base dependencies (the core results/timing modules import it unconditionally). Gitignore scratch/ for workspace-specific scratchpads. W291/W293 stay globally ignored because trailing whitespace inside multi-line SQL string literals is intentional and not touched by `ruff format`.
Add remote/cloud engines that talk to managed Spark via protocol:
- Livy — Fabric / Synapse / HDInsight via the Livy REST API, with
session auto-recovery, per-query timeout, multi-part SHOW TABLES.
- SparkConnect — Spark Connect gRPC client.
- FabricSpark / SynapseSpark / HDISpark — workspace-tagged Spark subclasses.
Catalog plumbing shared by all engines:
- BaseEngine.list_databases() / list_tables() / get_table_columns() defaults,
overridden for the Spark family, Livy and DuckDB.
- query_timeout_seconds attribute.
- transpile_and_qualify_query() rewritten with AST-based qualification that
correctly handles 3-/4-part names (workspace.lakehouse.schema, Unity
catalog.schema): builds quoted identifier chains via sqlglot, preserves the
caller's catalog, and leaves CTE references untouched. Adds 9 multi-part
tests (previously untested).
Column auto-remap is opt-in (auto_remap_columns=False) and warns loudly when
active — silently rewriting columns to match non-spec data hurts benchmark
reproducibility and can mask data-prep bugs.
Register Livy as a generic engine for TPC-H / TPC-DS / ClickBench.
…iles) A purely additive command-line surface over the existing Python API. Library consumers are unaffected. - cli/ package: argparse plumbing, override merge (-E/--conf), output formatting (human/table/json/csv/yaml), dry-run/print-config, verbosity and meaningful exit codes. - config.py: profile loader (~/.lakebench.json + ./lakebench.json), env-var expansion, `extends:` composition, validation, lazy engine/benchmark registries. resolve_engine() handles *_env credential references by honoring the engine signature: engines that accept the env-var NAME (Databricks, Livy) get it untouched and resolve the secret themselves; engines that accept the bare key (or **kwargs) get the resolved value. This avoids silently dropping the credential. Covered by tests/test_config.py. - results.py / reporting.py / discover.py as before. - Expose console_script entry point and livy/spark_connect extras + Fabric/ Synapse/HDInsight aliases. Docs: cli-quickstart, cli-reference, architecture, development.
This was referenced May 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
lakebenchcommand-line interface and supporting config/results/reporting plumbing.run,results,report,discover,doctor,profiles.config.py.Fix included:
token_envcredential-dropping bugconfig.resolve_enginepreviously stripped*_envsuffixes to bare keys (token_env→token) and then dropped them in the signature filter — so credentials configured via env-var profiles never reached the engine, silently breaking the documented Databricks/Livy profile flow.resolve_enginenow inspects the engine__init__signature: it passes the env-var name through untouched when the engine accepts a*_envparameter (Databricks/Livy), and resolves to the value for bare-key /**kwargsengines.Tests
New
tests/test_config.py(11 tests) covering the env-var resolution paths, plus CLI tests. Suite green (172 passed, 2 skipped).