diff --git a/.gitignore b/.gitignore index 1a1638b..42be2f8 100644 --- a/.gitignore +++ b/.gitignore @@ -17,6 +17,9 @@ mqssbench.db # Results results/ *.png +# But allow PNGs in docs +!docs/_static/*.png +!docs/_static/**/*.png # .vscode diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..360a2bb --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,135 @@ + +# Contributing to MQSS Benchmarking Framework + +Thanks for your interest in contributing to **MQSS Benchmarking Framework**. This project is Python-based and uses `uv` for dependency management, `ruff` for linting, and `pytest` for tests. + +This document explains how to propose and implement changes in a way that fits a production-quality workflow. + +## Table of Contents + +- [Getting Setup](#getting-setup) +- [Branching & Workflow](#branching--workflow) +- [Tips for Coding](#tips-for-coding) +- [How to Test](#how-to-test) +- [How to Lint](#how-to-lint) +- [Writing/Updating Tests](#writingupdating-tests) +- [Pull Request Checklist](#pull-request-checklist) +- [Notes for Benchmark Changes](#notes-for-benchmark-changes) + +## Getting Setup + +### Prerequisites + +- Python **>= 3.12** +- `uv` installed + +### Install dependencies + +```bash +pip install uv +uv sync --all-extras --dev +```` + +## Branching & Workflow + +### Base branch + +* All contribution branches are created from: `develop` + +### Branch naming (required) + +For contribution work, use the following convention: + +* `feat//` for new features or improvements +* `bug//` for bug fixes +* Use **lowercase** letters and digits. +* Use hyphens (`-`) in ``. +* Keep `` to 1-3 lowercase characters (as shown above). + +Examples (based on existing branches in this repo): + +* `feat/bm/add-qaoa-benchmark` +* `feat/aa/architecture-overhaul` + +### Pull requests + +* Target branch for PRs: `develop` +* Keep PRs focused on **one main goal** (feature, fix, or refactor with a clear justification). + +## Tips for Coding + +* Follow the project's style and structure. +* `src/mqssbench/framework` contains the core abstraction classes, types, and registries. It should have minimal external dependencies and no dependency on other parts of the project. + `src/mqssbench/library` contains built-ins and implementations based on the framework. This is a good place to start contributing (e.g., adding a new adapter, benchmark, or provider). +* Ensure type safety. +* Do not commit secrets (tokens, credentials). +* Avoid adding debug prints to production paths. + +## How to Test + +### Run Tests + +To run tests locally: + +```bash +uv run pytest +``` + +### Test Coverage + +To run tests with coverage locally: + +```bash +uv run pytest --cov=src --cov-report=term-missing +```` + +Notes specific to this repo: + +* Tests are discovered from the `test/` directory (`pytest.ini`). +* The `src/` folder is added to `PYTHONPATH` via pytest configuration. +* The test suite configures `matplotlib` to use a headless backend (`Agg`) to avoid CI hangs. + +What `good` looks like: + +* All tests pass locally before opening a PR. +* New behavior is backed by tests. + +## How to Lint + +### Ruff checks + +This repo's CI runs `ruff check`, and pre-commit is configured to auto-fix. + +Run: + +```bash +uv run ruff check . +``` + +If you want Ruff to attempt automatic fixes: + +```bash +uv run ruff check --fix . +``` + +## Writing/Updating Tests + +* Unit and integration tests live under `test/` +* Tests mirror the source code path and naming +* If your change touches registry behavior, be mindful of concurrency and registry isolation + +## Pull Request Checklist + +Before opening a PR, please ensure: + +* [ ] Branch name follows the mentioned structure +* [ ] PR targets `develop` +* [ ] You ran tests +* [ ] You ran lint +* [ ] No secrets or credentials were added +* [ ] Tests for the changed behavior exist and pass +* [ ] Documentation is updated + +Maintainers may request additional changes during review. Respond to feedback respectfully and update the PR as needed. + +Thanks \ No newline at end of file diff --git a/docs/_static/system_architecture.png b/docs/_static/system_architecture.png new file mode 100644 index 0000000..6da4dd8 Binary files /dev/null and b/docs/_static/system_architecture.png differ diff --git a/docs/system_architecture.md b/docs/system_architecture.md new file mode 100644 index 0000000..784506f --- /dev/null +++ b/docs/system_architecture.md @@ -0,0 +1,67 @@ +# System Architecture Diagram +```mermaid +flowchart TD + + %% ----------------------- + %% Initialization (Side) + %% ----------------------- + subgraph Init["Initialization"] + INIT["Initialize System"] + REG["Auto Registration"] + MODS["Built-in Modules"] + REGISTRY["Central Registry"] + + INIT --> REG --> MODS --> REGISTRY + end + + + %% ----------------------- + %% Entry + %% ----------------------- + CLI["CLI Entry (main.py)"] --> MANAGER["Benchmark Manager"] + + + %% ----------------------- + %% Command Handling + %% ----------------------- + subgraph Commands["Command Handling"] + + %% Info / Listing Commands + MANAGER --> LIST["List Commands"] + LIST --> REGISTRY + + %% Execution Commands + MANAGER --> RUNNER["Benchmark Runner"] + end + + + %% ----------------------- + %% Execution Flow + %% ----------------------- + RUNNER --> CONFIG["Parse Config"] + CONFIG --> REGISTRY + + REGISTRY --> COMPONENTS["Resolve Components"] + COMPONENTS --> BENCH["Benchmark"] + + + %% ----------------------- + %% Benchmark Execution Core + %% ----------------------- + subgraph Core["Benchmark Execution"] + + BENCH --> GEN["Circuit Generator"] + GEN --> EXEC["Executor"] + EXEC --> RESULTS["Raw Results"] + RESULTS --> ANALYZER["Result Analyzer"] + ANALYZER --> METRICS["Metrics Output"] + + end + + + %% ----------------------- + %% Finalization + %% ----------------------- + METRICS --> RETURN["Return Results"] + RETURN --> STORAGE["Storage (Optional)"] +```