feat: Delta schema evolution (#35), transient job retry (#40), and eviction self-cancel fix (#39)

[mdrakiburrahman]

Summary

This PR delivers three related features for the SCOPE adapter. All logic is unit-tested and the full integration suite passes locally (-n 32, GREEN).

1. Delta Lake schema evolution — Closes #35

Before building each SCOPE script, the adapter diffs the model's delta_table_columns against the live Delta table schema (read with DuckDB, after confirming _delta_log exists on ADLS):

• New column → injects ALTER TABLE @target ADD COLUMN IF NOT EXISTS <name> <type>; so the table evolves before the INSERT.
• Column removed from the model (still in the table) → fails with a DbtRuntimeError showing both schemas.
• Column type changed → fails with a diff (equivalence-based, e.g. SCOPE long ≡ Delta BIGINT).
• No _delta_log (new table) → no-op; CREATE TABLE creates the full schema.

Applies to both incremental and table (full-refresh) materializations. No new dependency (reuses the existing DuckDB + ADLS introspection).

2. Avoid self-cancel in quota eviction — Closes #39

The quota-eviction layer now excludes the current dbt run's own jobs from victim selection via is_self_job(job) — true if the job is in the process _active_jobs registry or its related.runId == _run_id. So eviction never cancels our own in-flight work.

3. Retry transient SCOPE job failures via a RegEx rule set — Closes #40

A curated, growable RetryRule(name, pattern, description) data structure (DEFAULT_JOB_RETRY_RULES in message_retry.py), seeded conservatively with:

• Exception in VertexManager ... Failed to open stream ... Operation timed out
• Job cancelled by user ... (external/cross-pipeline cancels — the fix: Do not cancel self job #39 fallback)

MessageRetryPolicy.for_job_retry() merges built-ins with user-configured job_retry_on_messages; execute() wraps submit_and_wait so a matching job failure re-submits a fresh job with capped exponential backoff. Non-matching failures propagate unchanged. To grow coverage later, append a RetryRule.

New profile config: enable_job_retry, job_retry_on_messages, job_retry_max_attempts, job_retry_initial_wait_seconds, job_retry_max_wait_seconds (all validated).

Testing

• ruff check + ruff format --check: clean
• Unit: 547 passed (schema-evolution diff/canonicalizer/introspection; eviction self-exclusion; rule matching; for_job_retry; credentials validation; execute() re-submit-on-match / no-retry-on-fatal)
• Integration (full suite, -n 32): 4 passed — including a new TestSchemaEvolution (datagen v1 → datagen v2 → assert region_name evolved in and populated), and TestIncremental (re-runs guard against type-check false positives)

Closes #35
Closes #39
Closes #40