Skip to content

fix: sync assessor default_weight values with default-weights.yaml#409

Open
jwm4 wants to merge 2 commits intomainfrom
fix/assessor-default-weight-sync
Open

fix: sync assessor default_weight values with default-weights.yaml#409
jwm4 wants to merge 2 commits intomainfrom
fix/assessor-default-weight-sync

Conversation

@jwm4
Copy link
Copy Markdown
Contributor

@jwm4 jwm4 commented May 8, 2026

Summary

The yaml (src/agentready/data/default-weights.yaml) is the authoritative weight source. Several assessors had stale default_weight fallback values that no longer matched.

Assessor File Was Now
DependencyPinningAssessor stub_assessors.py 0.10 0.05
DependencySecurityAssessor security.py 0.04 0.05
CyclomaticComplexityAssessor code_quality.py 0.03 0.02
StructuredLoggingAssessor code_quality.py 0.015 0.02

Also fixes stale "1.5% weight" docstrings in StructuredLoggingAssessor, ArchitectureDecisionsAssessor, and OpenAPISpecsAssessor.

Note: CyclomaticComplexityAssessor and StructuredLoggingAssessor are set to 0.02 to match the yaml values that will be in place after PR #407 merges. On current main the yaml has 0.03 for both; this PR should merge after #407.

Test plan

  • 1152 passed, 21 skipped
  • black . && isort . && ruff check . clean

🤖 Generated with Claude Code under the supervision of Bill Murdock

The yaml is the authoritative weight source. Several assessors had stale
default_weight values that no longer matched, causing misleading fallback
weights if the yaml were ever bypassed.

- DependencyPinningAssessor: 0.10 -> 0.05
- DependencySecurityAssessor: 0.04 -> 0.05
- CyclomaticComplexityAssessor: 0.03 -> 0.02 (anticipates PR #407)
- StructuredLoggingAssessor: 0.015 -> 0.02 (anticipates PR #407)

Also updates stale "1.5% weight" docstrings in StructuredLoggingAssessor,
ArchitectureDecisionsAssessor, and OpenAPISpecsAssessor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 8, 2026

Warning

Rate limit exceeded

@jwm4 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 15 minutes and 52 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: fa8c3e29-a2f8-41cd-b814-703a53ce7fc6

📥 Commits

Reviewing files that changed from the base of the PR and between 82b08c3 and 8c3b1cd.

📒 Files selected for processing (1)
  • src/agentready/assessors/code_quality.py

Warning

.coderabbit.yaml has a parsing error

The CodeRabbit configuration file in this repository has a parsing error and default settings were used instead. Please fix the error(s) in the configuration file. You can initialize chat with CodeRabbit to get help with the configuration file.

💥 Parsing errors (1)
Validation error: String must contain at most 250 character(s) at "tone_instructions"
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Walkthrough

This PR adjusts the default_weight configuration values for four assessors across the codebase—CyclomaticComplexityAssessor, StructuredLoggingAssessor, DependencySecurityAssessor, and DependencyPinningAssessor—and updates corresponding docstring descriptions. No scoring logic or public method signatures are altered.

Changes

Assessor Weight Configuration

Layer / File(s) Summary
Code Quality Weights
src/agentready/assessors/code_quality.py
CyclomaticComplexityAssessor.default_weight reduced from 0.03 to 0.02; StructuredLoggingAssessor.default_weight increased from 0.015 to 0.02 with docstring updated.
Documentation Weights
src/agentready/assessors/documentation.py
ArchitectureDecisionsAssessor and OpenAPISpecsAssessor docstrings updated from "1.5% weight" to "3% weight".
Security Weights
src/agentready/assessors/security.py
DependencySecurityAssessor.default_weight increased from 0.04 to 0.05.
Stub Assessor Weights
src/agentready/assessors/stub_assessors.py
DependencyPinningAssessor.default_weight reduced from 0.10 to 0.05.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: syncing stale assessor default_weight values with the authoritative YAML configuration file.
Description check ✅ Passed The description is directly related to the changeset, providing a detailed summary table of weight changes, affected assessors, docstring fixes, and test verification.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/assessor-default-weight-sync

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

📈 Test Coverage Report

Branch Coverage
This PR 72.5%
Main 72.5%
Diff ✅ +0%

Coverage calculated from unit tests only

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/agentready/assessors/code_quality.py (1)

668-678: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fix silent failure in workflow actionlint detection—Path.glob() returns a generator, not a list, so concatenating with + raises TypeError.

workflows_dir.glob("*.yml") + workflows_dir.glob("*.yaml") fails at runtime because generators cannot be concatenated with +. The bare except Exception: pass silently catches the error, so actionlint configured only in workflow files is never detected.

Fix
-            for workflow_file in workflows_dir.glob("*.yml") + workflows_dir.glob(
-                "*.yaml"
-            ):
+            for workflow_file in list(workflows_dir.glob("*.yml")) + list(
+                workflows_dir.glob("*.yaml")
+            ):
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/agentready/assessors/code_quality.py` around lines 668 - 678, The code is
concatenating two Path.glob() generators with + which raises TypeError and is
swallowed by the bare except; replace the concatenation with a proper iterator
(e.g., import itertools and iterate itertools.chain(workflows_dir.glob("*.yml"),
workflows_dir.glob("*.yaml")) or convert both to lists and concatenate) and
remove or narrow the bare except so failures aren't silently ignored — preserve
the logic that for each workflow_file read_text() is inspected for "actionlint"
and ensure any caught Exception is at least logged (referencing workflows_dir,
workflow_file, .glob(), and the "actionlint" check).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/agentready/assessors/code_quality.py`:
- Around line 668-678: The code is concatenating two Path.glob() generators with
+ which raises TypeError and is swallowed by the bare except; replace the
concatenation with a proper iterator (e.g., import itertools and iterate
itertools.chain(workflows_dir.glob("*.yml"), workflows_dir.glob("*.yaml")) or
convert both to lists and concatenate) and remove or narrow the bare except so
failures aren't silently ignored — preserve the logic that for each
workflow_file read_text() is inspected for "actionlint" and ensure any caught
Exception is at least logged (referencing workflows_dir, workflow_file, .glob(),
and the "actionlint" check).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 40171439-7d8d-47c6-92f9-26332030e5b2

📥 Commits

Reviewing files that changed from the base of the PR and between 8c5499c and 82b08c3.

📒 Files selected for processing (4)
  • src/agentready/assessors/code_quality.py
  • src/agentready/assessors/documentation.py
  • src/agentready/assessors/security.py
  • src/agentready/assessors/stub_assessors.py

`Path.glob()` returns a generator, not a list. Concatenating two generators
with `+` raises TypeError, silently swallowed by the bare except, causing
actionlint detection via workflow files to never work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant