docs: audit and correct attributes.md against actual assessor code#408
docs: audit and correct attributes.md against actual assessor code#408
Conversation
Full accuracy audit of docs/attributes.md found weights, criteria, and structural information that had drifted from the implementation. All discrepancies corrected. Weight corrections (pre-v2.0.0 values updated): - readme_structure: 10% -> 5% - type_annotations: 10% -> 8% - standard_layout: 10% -> 5% - lock_files: 10% -> 5% - cyclomatic_complexity: 3% -> 2% - structured_logging: 3% -> 2% - issue_pr_templates: Tier 4 / 1% -> Tier 3 / 1.5% Criteria corrections (doc described checks the code does not perform): - claude_md_file: remove 6-section requirement; actual check is file exists at >=50 bytes; add AGENTS.md as fully equivalent alternative - readme_structure: replace 8-section list with actual 3 sections checked (installation, usage, development); pass >=75 - test_execution: remove 70/80/90% coverage thresholds; assessor checks config presence only (test files, runner, coverage config, enforcement) - lock_files: add freshness check and requirements.txt pin quality; remove gitignore claim that is not checked - deterministic_enforcement: remove CI enforcement requirement; actual scoring is .pre-commit-config.yaml (60pts), .claude/settings.json hooks (30pts), .husky (10pts); pass >=60 - gitignore_completeness: replace category-based with pattern-based description; pass >=70% of language-specific patterns - container_setup: add Containerfile, .dockerignore, conditionality New full write-ups added for attributes previously listed as one-liners: - ci_quality_gates (Tier 1, 5%) - single_file_verification (Tier 1, 5%) - dependency_security (Tier 1, 5%) New entry added: repomix_config (Tier 3, 2%) Other fixes: - separation_concerns -> separation_of_concerns (attribute ID corrected) - Tier 3 count: 5 -> 6 attributes - Implementation status table: all tiers now show fully implemented Assessor default_weight values synced to yaml (yaml is authoritative): - DependencyPinningAssessor: 0.10 -> 0.05 - DependencySecurityAssessor: 0.04 -> 0.05 - CyclomaticComplexityAssessor: 0.03 -> 0.02 - StructuredLoggingAssessor: 0.015 -> 0.02 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Warning
|
📈 Test Coverage Report
Coverage calculated from unit tests only |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace vague research claims, padded impact bullet lists, and inline bold-header bullets with direct, specific prose. - Remove unsourced "~40% prompt engineering reduction" claim - Replace "Impact on AI Agents" bullet lists with single focused paragraphs - Remove overview's inline bold-header format (Research-backed/Measurable/etc.) - Replace vague "Research shows" with specific mechanism explanations - Remove significance inflation phrases - Fix one em dash All scoring criteria, thresholds, commands, and citations unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Summary
Full accuracy audit of
docs/attributes.mdfound widespread drift between the documentation and the actual assessor implementations. This PR corrects everything found.Weight corrections
Pre-v2.0.0 weights that were never updated in the doc:
readme_structuretype_annotationsstandard_layoutlock_filescyclomatic_complexitystructured_loggingissue_pr_templatesCriteria corrections
The doc described checks the assessors do not actually perform:
claude_md_file: doc claimed 6 required content sections; code only checks the file exists at ≥50 bytes. AddedAGENTS.mdas a fully equivalent alternative (scores 100).readme_structure: doc listed 8 required sections; code checks 3 (installation, usage, development), pass ≥75.test_execution: doc described 70/80/90% coverage thresholds; assessor does not measure coverage at all — it checks whether test config files are present.lock_files: added freshness check (15pt deduction if >6 months old) andrequirements.txtpin quality validation; removed gitignore claim that is not checked.deterministic_enforcement: doc required CI enforcement; assessor only checks.pre-commit-config.yaml(60pts),.claude/settings.jsonhooks (30pts),.husky(10pts). Pass on pre-commit config alone.gitignore_completeness: replaced category-based description with actual pattern-based mechanism (70% of language-specific patterns from hardcoded list).container_setup: addedContainerfileas alternative,.dockerignorescoring, and conditionality (not_applicable if no container file found).New content
Full write-ups added for three Tier 1 attributes that previously had only one-liners:
ci_quality_gates(5%)single_file_verification(5%)dependency_security(5%)New full entry for
repomix_config(Tier 3, 2%) — recently registered assessor, not yet documented.Other fixes
separation_concerns→separation_of_concerns(attribute ID was wrong throughout)Assessor code fixes
default_weightvalues in four assessors were out of sync with the yaml (yaml is authoritative):stub_assessors.pyDependencyPinningAssessorsecurity.pyDependencySecurityAssessorcode_quality.pyCyclomaticComplexityAssessorcode_quality.pyStructuredLoggingAssessorTest plan
black . && isort . && ruff check .— clean🤖 Generated with Claude Code under the supervision of Bill Murdock