Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .cruft.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
"license": "MIT",
"version": "4.0.0",
"description": "Predict the number of bikes available",
"python_version": "3.13",
"mlflow_version": "2.20.3",
"python_version": "3.14",
"mlflow_version": "3.14.0",
"_template": "https://github.com/fmind/cookiecutter-mlops-package"
}
},
Expand Down
18 changes: 18 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
.git
.github
.venv
.pytest_cache
.ruff_cache
.ty_cache
__pycache__
*.py[co]
.coverage
htmlcov
dist
docs
mlruns
outputs
notebooks
tests
.env
.cruft.json
24 changes: 12 additions & 12 deletions .gemini/skills/MLOps Automation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ Replace manual commands with a `justfile`.
1. **Tool**: `just` (modern alternative to Make).
2. **Organization**: Split tasks into `tasks/*.just` modules (e.g., `tasks/check.just`, `tasks/docker.just`).
3. **Core Tasks**:
- `check`: Run all linters and tests.
- `package`: Build wheels.
- `clean`: Remove artifacts.
- `install`: Setup dev environment.
- `check`: Run all linters and tests.
- `package`: Build wheels.
- `clean`: Remove artifacts.
- `install`: Setup dev environment.

### 2. Pre-Commit Hooks

Expand All @@ -45,8 +45,8 @@ Reproducibility anywhere.
1. **Tool**: `docker`.
2. **Base Image**: Use `ghcr.io/astral-sh/uv:python3.1X-bookworm-slim` for minimal size.
3. **Optimization**:
- **Layer Caching**: Copy `uv.lock` + `pyproject.toml` and run `uv sync` *before* copying `src/`.
- **Multi-stage**: Build inputs in one stage, copy only artifacts (`dist/*.whl`) to the runtime stage.
- **Layer Caching**: Copy `uv.lock` + `pyproject.toml` and run `uv sync` _before_ copying `src/`.
- **Multi-stage**: Build inputs in one stage, copy only artifacts (`dist/*.whl`) to the runtime stage.
4. **Registry**: ask for the company artifact registry, or use `ghcr.io` for GitHub.

### 4. CI/CD Workflows
Expand All @@ -55,8 +55,8 @@ Automate verification and release.

1. **Platform**: ask for the company CI/CD platform, or use `github-actions` for GitHub.
2. **Workflows**:
- `check.yml`: On PRs (Run `just check`).
- `publish.yml`: On Release (Build docker image, publish docs/package).
- `check.yml`: On PRs (Run `just check`).
- `publish.yml`: On Release (Build docker image, publish docs/package).
3. **Optimization**: Use `concurrency` to cancel redundant runs.

### 5. AI/ML Experiments & Registry
Expand All @@ -65,11 +65,11 @@ Manage the ML lifecycle.

1. **Platform**: `MLflow`.
2. **Tracking**:
- Use `mlflow.autolog()`.
- Log metrics, params, and artifacts.
- Use `mlflow.autolog()`.
- Log metrics, params, and artifacts.
3. **Registry**:
- Register top models manually or via CI.
- **Aliases**: Use `@champion` or `@production` for stable deployment pointers. Never rely on moving versions (e.g., `v1` -> `v2`).
- Register top models manually or via CI.
- **Aliases**: Use `@champion` or `@production` for stable deployment pointers. Never rely on moving versions (e.g., `v1` -> `v2`).

### 6. Design Patterns

Expand Down
10 changes: 5 additions & 5 deletions .gemini/skills/MLOps Collaboration/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,18 @@ Eliminate "it works on my machine".

1. **Templates**: Use `cookiecutter` for scaffolding and `cruft update` to keep projects synced.
2. **Workstations**: Add `.devcontainer/devcontainer.json`.
- Define Docker image, extensions, and settings.
- Enable GitHub Codespaces support.
- Define Docker image, extensions, and settings.
- Enable GitHub Codespaces support.

### 4. Release Management

Ship with confidence.

1. **Versioning**: Follow **SemVer** (MAJOR.MINOR.PATCH) and [Keep a Changelog](https://keepachangelog.com/).
2. **Workflows**:
- **GitHub Flow**: Small teams, continuous delivery (`main` is stable).
- **Git Flow**: Scheduled releases (`develop` + `release` branches).
- **Forking**: Open source, distributed contributors.
- **GitHub Flow**: Small teams, continuous delivery (`main` is stable).
- **Git Flow**: Scheduled releases (`develop` + `release` branches).
- **Forking**: Open source, distributed contributors.
3. **Process**: Bump version -> Update Changelog -> Tag -> Release.

## Self-Correction Checklist
Expand Down
110 changes: 55 additions & 55 deletions .gemini/skills/MLOps Industrialization/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,18 @@ Adopt the `src` layout to prevent import errors and separate source from tooling

1. **Directory Tree**:

```text
my-project/
├── pyproject.toml # Dependencies & Metadata
├── uv.lock
├── README.md
└── src/
└── my_package/ # Main package directory
├── __init__.py
├── io/ # Side-effects (Datasets, APIs)
├── domain/ # Pure business logic (Models, Features)
└── application/ # Orchestration (Training loops, Inference)
```
```text
my-project/
├── pyproject.toml # Dependencies & Metadata
├── uv.lock
├── README.md
└── src/
└── my_package/ # Main package directory
├── __init__.py
├── io/ # Side-effects (Datasets, APIs)
├── domain/ # Pure business logic (Models, Features)
└── application/ # Orchestration (Training loops, Inference)
```

2. **Configuration**: Use `pyproject.toml` for all build metadata and dependencies.

Expand All @@ -43,16 +43,16 @@ Adopt the `src` layout to prevent import errors and separate source from tooling
Balance structure with predictability.

1. **Domain Layer (Pure)**:
- **Rule**: Code here must be deterministic and free of side effects (no I/O).
- **Use Case**: Feature transformations, Model architecture definitions.
- **Style**: Functional (pure functions) or Immutable Objects (dataclasses).
- **Rule**: Code here must be deterministic and free of side effects (no I/O).
- **Use Case**: Feature transformations, Model architecture definitions.
- **Style**: Functional (pure functions) or Immutable Objects (dataclasses).
2. **I/O Layer (Impure)**:
- **Rule**: Isolate external interactions here.
- **Use Case**: Loading data from S3, saving models to disk, logging to MLflow.
- **Style**: OOP (Classes to manage connections/state).
- **Rule**: Isolate external interactions here.
- **Use Case**: Loading data from S3, saving models to disk, logging to MLflow.
- **Style**: OOP (Classes to manage connections/state).
3. **Application Layer (Orchestration)**:
- **Rule**: Wire Domain and I/O together.
- **Use Case**: Tuning, Training, Inference, Evaluation, etc.
- **Rule**: Wire Domain and I/O together.
- **Use Case**: Tuning, Training, Inference, Evaluation, etc.

### 3. Application Entrypoints

Expand All @@ -61,46 +61,46 @@ Create standard, installable CLI tools.
1. **Define Script**: Create `src/my_package/scripts.py` with a `main()` function.
2. **Register**: Add to `pyproject.toml`:

```toml
[project.scripts]
my-tool = "my_package.scripts:main"
```
```toml
[project.scripts]
my-tool = "my_package.scripts:main"
```

3. **CLI Execution**:
- **Dev**: `uv run my-tool` (No install needed).
- **Prod**: `pip install .` -> `my-tool` (Installed on PATH).
- **Dev**: `uv run my-tool` (No install needed).
- **Prod**: `pip install .` -> `my-tool` (Installed on PATH).
4. **Guard**: Always use `if __name__ == "__main__":` in scripts to prevent execution on import.

### 4. Configuration Management

Decouple settings from code using **OmegaConf** (Parsing) and **Pydantic** (Validation).

1. **Define Schema (Pydantic)**:
- Create a class that defines *expected* types and defaults.
- Create a class that defines _expected_ types and defaults.

```python
from pydantic import BaseModel
```python
from pydantic import BaseModel

class TrainingConfig(BaseModel):
batch_size: int = 32
learning_rate: float = 0.001
use_gpu: bool = False
```
class TrainingConfig(BaseModel):
batch_size: int = 32
learning_rate: float = 0.001
use_gpu: bool = False
```

2. **Parse & Validate (OmegaConf)**:
- Load YAML, merge with CLI args, and validate against the schema.
- Load YAML, merge with CLI args, and validate against the schema.

```python
import omegaconf
```python
import omegaconf

# 1. Load YAML
conf = omegaconf.OmegaConf.load("config.yaml")
# 2. Merge with CLI (optional)
cli_conf = omegaconf.OmegaConf.from_cli()
merged = omegaconf.OmegaConf.merge(conf, cli_conf)
# 3. Validate -> Returns a validated Pydantic object
cfg: TrainingConfig = TrainingConfig(**omegaconf.OmegaConf.to_container(merged))
```
# 1. Load YAML
conf = omegaconf.OmegaConf.load("config.yaml")
# 2. Merge with CLI (optional)
cli_conf = omegaconf.OmegaConf.from_cli()
merged = omegaconf.OmegaConf.merge(conf, cli_conf)
# 3. Validate -> Returns a validated Pydantic object
cfg: TrainingConfig = TrainingConfig(**omegaconf.OmegaConf.to_container(merged))
```

3. **Secrets**: Use Environment Variables (`os.getenv`), never commit them.

Expand All @@ -110,18 +110,18 @@ Make code usable and maintainable.

1. **Docstrings**: Use **Google Style** docstrings for all modules, classes, and functions.

```python
def calculate_metric(y_true: np.ndarray, y_pred: np.ndarray) -> float:
"""Calculates the accuracy score.
```python
def calculate_metric(y_true: np.ndarray, y_pred: np.ndarray) -> float:
"""Calculates the accuracy score.

Args:
y_true: Ground truth labels.
y_pred: Predicted labels.
Args:
y_true: Ground truth labels.
y_pred: Predicted labels.

Returns:
The accuracy as a float between 0 and 1.
"""
```
Returns:
The accuracy as a float between 0 and 1.
"""
```

2. **Type Hints**: Use standard python typing (`typing`, `list[str]`) everywhere.

Expand Down
Loading