Skip to content

feat(config): three-scope config resolution (User → Project → Local) - #73

Merged
elkaix merged 28 commits into
mainfrom
feat/scoped-config
Jun 3, 2026
Merged

feat(config): three-scope config resolution (User → Project → Local)#73
elkaix merged 28 commits into
mainfrom
feat/scoped-config

Conversation

@elkaix

@elkaix elkaix commented Jun 3, 2026

Copy link
Copy Markdown
Member

Summary

  • Replaces single-file load_config() with a five-step scoped pipeline: Ingest → Guard → Merge → Env → Validate
  • Three config scopes resolved in order: ~/.pythinker/config.toml (user) → .pythinker/config.toml (project) → .pythinker/config.local.toml (local); local wins on scalar fields, lists concatenate across scopes
  • Scope-locked secret fields (providers, services, feedback.api_key) are blocked in project/local scopes before merging; PYTHINKER_* env vars overlay all file scopes
  • config.local.toml is auto-added to .gitignore when first loaded; Config.source_scopes field tracks which files contributed
  • Explicit --config path bypasses scope resolution entirely (backward-compatible)

New files

  • src/pythinker_code/utils/gitignore.pyensure_gitignored utility
  • tests/utils/test_gitignore.py — 5 tests

Modified files

  • src/pythinker_code/config.py — all new constants, helpers, pipeline functions, source_scopes field, load_config wiring
  • tests/core/test_config.py — unit + integration tests for every new function

Test Plan

  • pytest tests/core/test_config.py — 66 tests covering all pipeline functions and backward-compat paths
  • pytest tests/utils/test_gitignore.py — 5 tests for ensure_gitignored
  • pytest tests/ --ignore=tests/e2e — full suite (4281 passed, 0 failures)
  • make check-pythinker-code — ruff format + check clean

Summary by CodeRabbit

  • New Features

    • Config now resolves across user/project/local scopes with tracked source attribution, scope‑locked protections, PYTHINKER_* env overlays, provenance-aware validation errors, and auto-gitignore of local config.
    • Added helper to safely ensure patterns are present in .gitignore.
  • Documentation

    • Package identity/author updated; version/info output shows organization.
    • Agent persona updated to state product identity; standardized question-marker glyph.
  • Tests

    • Extensive tests added for scoped config behavior, provenance, validations, env overrides, and gitignore helper.

elkaix added 13 commits June 3, 2026 14:33
Add SCOPE_LOCKED_PATHS, DEDUP_LIST_FIELDS, ENV_FIELD_MAP constants
and _set_nested, _lookup_provenance helper functions to support the
scoped config pipeline. These pure functions handle nested dict
operations and provenance tracking across config scopes.
Implements _load_scoped(project_root) as the core scoped config
resolution pipeline: Ingest → Guard → Merge → Env → Validate.

Reads user, project (.pythinker/config.toml), and local
(.pythinker/config.local.toml) scopes; enforces scope locks before
merge; overlays env vars last; attributes validation errors to their
source scope; populates source_scopes on the returned Config and
auto-gitignores config.local.toml when present.

Also wraps the Task-7 source_scopes description to fix a pre-existing
ruff E501 violation (blocked the ruff gate).

Adds 8 integration tests covering user-only, project-overrides-user,
local-overrides-project, hook concatenation, scope-lock violation,
validation error attribution, env override, and source_scopes metadata.

Note: 3 provided tests used "solarized" as a theme value, which is not
in Literal["dark","light"] and fails Pydantic validation. Tests were
corrected to "light"/"dark" while preserving their override-precedence
intent. The merge-layer three-scope ordering is independently covered by
test_merge_scalar_three_scopes.

Pyright debt: 60 errors (45 pre-existing bare-dict annotations in
Tasks 3-6 helpers + 15 same-class cascades from _load_scoped). No real
type mismatches. Deferred to Task 9 typing sweep.
When called with no explicit file path, load_config now discovers
User → Project → Local scopes relative to the nearest .git root,
merges them with type-based rules, overlays PYTHINKER_* env vars,
and validates once through Pydantic with provenance-enriched errors.
Explicit --config path continues to bypass scope resolution.

Also seeds a default user config.toml when no config exists after
JSON migration (e.g. corrupt JSON backed up) so existing behaviour
is preserved. Two backward-compatibility tests added.
@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a three-scope config resolution pipeline (user → project → local) with guarded merges, provenance tracking, PYTHINKER_* env overlays, validation errors annotated with provenance, and auto-gitignore of local config. Adds ensure_gitignored() and updates CLI metadata (ORGANIZATION, CONTACT, version/info output). Comprehensive tests cover helpers and integrations.

Changes

Configuration scoping system

Layer / File(s) Summary
Gitignore management utility and tests
src/pythinker_code/utils/gitignore.py, tests/utils/test_gitignore.py
ensure_gitignored() safely appends ignore patterns to .gitignore with deduplication, optional comment, and proper newline handling. Tests cover creation, appending, no-duplication, newline fixing, and empty-comment behavior.
Scoped config primitives and merging
src/pythinker_code/config.py (imports and core ranges)
Adds _find_project_root(), scope-lock constants/checks, _set_nested(), _lookup_provenance(), _type_based_merge() (dict recursion, list concat with dedup for selected fields), and _apply_env_vars() helpers.
Config model, loader branching, and metadata
src/pythinker_code/config.py (model fields and load flow)
Adds Config.source_scopes: dict[str, Path] (excluded from model_dump) and changes `load_config(config_file: Path
Config unit and integration tests
tests/core/test_config.py (multiple ranges)
Adds unit tests for discovery, nested set, provenance lookup, scope-lock enforcement, typed merge semantics (scalar override, list concatenation + provenance, dedup for extra_skill_dirs, deep dict merge), env-var overlays, plus integration tests for scoped loading precedence, hook concatenation, scope-lock violations, validation error source attribution, env override precedence, source_scopes population, gitignore interaction, and load_config() branching.
Branding, CLI info, and changelog
src/pythinker_code/constant.py, src/pythinker_code/__main__.py, src/pythinker_code/cli/info.py, pyproject.toml, CHANGELOG.md, src/pythinker_code/agents/default/system.md, tests/core/test_default_agent.py
Adds ORGANIZATION and CONTACT constants and exports; updates CLI --version/info output to include organization; updates project metadata (author/org and description) and Unreleased changelog entries; updates default system prompt identity and snapshot test.

Sequence Diagram(s)

sequenceDiagram
  participant UserConfig as User Config (~/.pythinker/config.toml)
  participant ProjectConfig as Project Config (.pythinker/config.toml)
  participant LocalConfig as Local Config (.pythinker/config.local.toml)
  participant Merger as _type_based_merge
  participant EnvOverlay as _apply_env_vars (PYTHINKER_*)
  participant Guard as _check_scope_locks
  participant Validator as Pydantic Validator
  participant Result as Resolved Config

  UserConfig->>Merger: load user scope dict + provenance
  ProjectConfig->>Merger: overlay project scope dict + provenance
  LocalConfig->>Merger: overlay local scope dict + provenance
  Merger->>EnvOverlay: merged dict + provenance
  EnvOverlay->>Guard: apply env overrides, update provenance
  Guard->>Validator: enforce scope locks
  Validator->>Result: validate & attach source_scopes
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

  • Pythoughts-labs/pythinker-code#62: Related to TUI config fields consumption (code theme, smooth_streaming) that read runtime config values introduced/extended in this PR.

Suggested labels

enhancement

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.54% which is insufficient. The required threshold is 70.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed Title follows conventional commits format with correct type (feat), scope (config), and descriptive summary of the main feature.
Description check ✅ Passed PR description covers objectives, files changed, and test plan clearly, though it lacks explicit issue link and contributor checklist completion.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/scoped-config

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/pythinker_code/config.py`:
- Around line 156-167: When _type_based_merge encounters a dict override (value
is dict) but base[key] exists as a non-dict, avoid recursing into the non-dict
by first detecting this type conflict and replacing base[key] with a new dict
and normalizing provenance[key] to a dict before calling _type_based_merge;
i.e., in the branch handling isinstance(value, dict) (inside _type_based_merge)
add a check like "if key in base and not isinstance(base[key], dict): base[key]
= {}" and ensure "provenance[key] = {}" (or convert it to a dict) so the
recursive call operates on dicts only and prevents the TypeError so validation
can raise a ConfigError later.

In `@tests/core/test_config.py`:
- Around line 589-597: Add an integration test that verifies _load_scoped
invokes ensure_gitignored when a local config file exists: create a tmp
project_root, write a local config file (e.g., project_root/"config.toml") and
the existing .pythinker/config.toml, then monkeypatch the module-level
ensure_gitignored (the function named ensure_gitignored) with a spy/stub that
records invocation, call _load_scoped(project_root=project_root), and assert the
spy was called with the project_root (or path to the local config) and that
source_scopes include "local"; reference the _load_scoped and ensure_gitignored
symbols when adding the test.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 61d37245-81aa-4567-aaca-70220e313296

📥 Commits

Reviewing files that changed from the base of the PR and between 923bc83 and f7e3350.

📒 Files selected for processing (4)
  • src/pythinker_code/config.py
  • src/pythinker_code/utils/gitignore.py
  • tests/core/test_config.py
  • tests/utils/test_gitignore.py

Comment thread src/pythinker_code/config.py
Comment thread tests/core/test_config.py
elkaix added 2 commits June 3, 2026 15:53
Add missing type arguments to bare dict/tuple generics across all five
pipeline helpers (_set_nested, _lookup_provenance, _check_scope_locks,
_type_based_merge, _apply_env_vars) and the nested _read_toml function.
Import Any and cast from typing; use cast() at isinstance-narrowing call
sites where dict[Unknown, Unknown] would otherwise propagate. Remove the
now-redundant isinstance(prov, dict) guard in _lookup_provenance (prov is
already narrowed to dict[str, Any] after the early-return str branch).
Break the long _type_based_merge signature across lines to satisfy E501.

Also wire Pythoughts-labs branding into constant.py (ORGANIZATION/CONTACT),
pyproject.toml (authors/description), --version output, and pythinker info.
Update __all__ in constant.py to multi-line form for line-length compliance.

Add CHANGELOG entry for both the scoped config feature and identity update.
Add a 'Product Identity' block at the top of the default agent system
prompt that instructs the agent to identify itself as Pythinker by
Pythoughts-labs when asked, and to not name the underlying model.

The underlying model (e.g. MiniMax, Claude) would otherwise fall back
to its own built-in identity, causing responses like "I am MiniMax-M3"
instead of the correct product identity.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/pythinker_code/agents/default/system.md`:
- Line 1: Add a top-level H1 to the start of the Markdown file so the first line
is a proper heading (fixing MD041); insert a descriptive H1 (for example "#
Pythinker") immediately before the existing opening sentence "You are
**Pythinker** — a think-first software engineering agent..." in
src/pythinker_code/agents/default/system.md so the document begins with an H1 on
line 1.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b44c33c9-5e9c-44e1-a653-37e3583db1c1

📥 Commits

Reviewing files that changed from the base of the PR and between f77efb1 and 2364488.

📒 Files selected for processing (1)
  • src/pythinker_code/agents/default/system.md

Comment thread src/pythinker_code/agents/default/system.md
elkaix added 3 commits June 3, 2026 16:04
- Guard dict/scalar type conflict in _type_based_merge: when base holds
  a scalar or list for a key that an overlay wants to replace with a
  dict, let the overlay win outright instead of recursing into a non-dict
  (which crashes with TypeError before validation can surface a ConfigError)

- Add integration test verifying _load_scoped auto-gitignores
  config.local.toml when the file is present (test_load_scoped_gitignores_local_config)

- Add MD041-compliant H1 heading to agents/default/system.md

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/core/test_default_agent.py`:
- Around line 21-29: The current test in tests/core/test_default_agent.py is
brittle because it asserts the full prompt snapshot; update the test (the test
function that currently checks the identity prompt) to remove the large snapshot
assertion and instead assert focused invariants: confirm the prompt contains the
header "## Product Identity", the tokens "Pythinker" and "Pythoughts-labs", and
the explicit rule phrase forbidding naming/describing the underlying model
(e.g., assert "do not name" or "do not name or describe the underlying model" in
the prompt). Replace the snapshot equality with a few targeted in/substring
assertions and keep any existing helper that loads the prompt string. Ensure the
test uses plain membership checks (e.g., "in") rather than exact multi-line
matches so unrelated prompt edits won’t break it.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 69984bf2-f511-4b75-9db0-70fc48e205bd

📥 Commits

Reviewing files that changed from the base of the PR and between 2bf5266 and 4eb94fe.

📒 Files selected for processing (1)
  • tests/core/test_default_agent.py

Comment thread tests/core/test_default_agent.py Outdated
elkaix added 8 commits June 3, 2026 16:39
- config.py: in _type_based_merge, when a dict overlay hits a scalar/list
  at the same key, normalize base[key] and provenance[key] to empty dicts
  before recursing instead of short-circuiting with `continue`. The old
  approach left provenance[key] as a string, which would crash a subsequent
  dict-merge on the same key with TypeError.

- test_default_agent.py: replace the full-prompt inline_snapshot in
  test_default_agent with four targeted substring assertions covering the
  Product Identity invariants (section header, product name, developer,
  and the no-model-name rule). The builtin_types snapshot is unchanged.
@elkaix
elkaix merged commit d12c505 into main Jun 3, 2026
40 checks passed
@elkaix
elkaix deleted the feat/scoped-config branch July 17, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant