Merge branch 'main' into nn

Author: MartheGjelstad

.github/CODEOWNERS

@@ -0,0 +1 @@
+/PythonScripts/  @moritz-gross

.github/workflows/python.yml

@@ -0,0 +1,20 @@
+name: Python
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    name: Python Tests
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - uses: astral-sh/setup-uv@v6
+    - name: Run ruff lint
+      working-directory: PythonScripts
+      run: uv run ruff check audit_translations/
+    - name: Run ruff format check
+      working-directory: PythonScripts
+      run: uv run ruff format --check audit_translations/
+    - name: Run tests
+      working-directory: PythonScripts
+      run: uv run pytest

.github/workflows/rust.yml

@@ -1,10 +1,6 @@
 name: Rust
 
-on:
-  push:
-    branches: [ "main" ]
-  pull_request:
-    branches: [ "main" ]
+on: [push, pull_request] # note: pre-release job still only on 'refs/heads/main'
 
 env:
   CARGO_TERM_COLOR: always
@@ -56,6 +52,7 @@ jobs:
   pre-release:
     name: Pre Release
     continue-on-error: false
+    if: github.event_name == 'push' && github.ref == 'refs/heads/main' # otherwise, job fails on pull requests, as GITHUB_TOKEN is not available
     needs: [zip-rules]
     runs-on: ubuntu-latest
     permissions:

.gitignore

@@ -52,3 +52,8 @@ src/speech-bak
 
 Rules/*/*/*.zip
 .vs
+
+# local caches when building Jekyll
+docs/_site
+docs/.jekyll-cache
+docs/Gemfile.lock

AGENTS.md

@@ -0,0 +1,35 @@
+# Agent Notes (MathCAT)
+
+Purpose: fast orientation for coding agents. try to keep low overlap with README.md etc., 
+but add common mistakes of AI agents here instead.
+
+## Project Scope
+- MathCAT converts MathML to speech, braille, and navigation output.
+- Core flow: `set_mathml()` -> `canonicalize.rs` -> optional `infer_intent.rs` -> `speech.rs` / `braille.rs`.
+
+## Rules System (`Rules/`)
+- YAML rules are loaded at runtime by domain:
+- Common per-language files:
+  - `ClearSpeak_Rules.yaml`, `SimpleSpeak_Rules.yaml`
+  - `SharedRules/`, `unicode.yaml`, `unicode-full.yaml`, `definitions.yaml`, `navigate.yaml`
+- `build.rs` can bundle rules into `rules.zip` when `include-zip` is enabled.
+
+## Translation Conventions
+- `t:` means untranslated or unverified.
+- `T:` means translated and verified.
+- tool for comparing rules across languages: `uv run --project PythonScripts audit-translations <language-code>`
+
+## Python Tooling (`uv`)
+- `uv` is the Python dependency and project manager for repo tooling. Use `uv run <command-name>`
+- discuss new packages before adding them. use `uv add <package-name>` and `uv sync` on confirmation
+- In sandboxed runs, if needed:
+  - set `UV_CACHE_DIR=/tmp/uv-cache`
+  - rerun with escalated permissions if macOS `system-configuration` panics occur.
+- *always* self-validate: `ùv run pytest`
+
+## Agent Instructions
+- Do not mirror README content here; keep guidance agent-specific.
+- Avoid broad formatting sweeps; do not run `cargo fmt` in this repo.
+- Keep code/rule changes focused and validate with targeted tests first: `cargo test <relevant-tests>`
+- do not do any git commands unless explicitly asked for
+- Rust coverage is in `target/coverage/`.

Cargo.toml

@@ -14,6 +14,8 @@ exclude = ["src/main.rs", "docs", "PythonScripts"]    # should have "Rules/", bu
 [features]
 "include-zip" = []
 "enable-logs" = ["android_logger"]
+"tts" = [ "natural-tts" ]
+
 
 [dependencies]
 sxd-document = "0.3"
@@ -34,15 +36,15 @@ log = "0.4"
 env_logger = "0.11.8"
 cfg-if = "1.0.1"
 fastrand = { version = "2.3.0" }
+clap = { version = "*", features = ["derive"] }
 
 [target.'cfg(target_family = "wasm")'.dependencies]
 zip = { version = "7.0", default-features = false, features = ["deflate"] }
 [target.'cfg(not(target_family = "wasm"))'.dependencies]
 zip = { version = "7.0", default-features = false, features = ["bzip2"] }
 
 android_logger = {version = "0.15.1", optional = true}
-
-
+natural-tts = { version = "*", optional = true }
 
 [build-dependencies]
 bitflags = "2.6"
@@ -55,6 +57,11 @@ zip = { version = "7.0", default-features = false, features = ["bzip2"] }
 name = "libmathcat"
 crate-type = ["rlib", "cdylib"]
 
+[[bin]]
+name = "mathml2text"
+path = "src/bin/mathml2text.rs"
+
+
 [profile.test]
 debug = true
 opt-level = 1    # adds a few seconds to the compile, but cuts testing time by ~75% (~90 secs on 5/24)

PythonScripts/README.md

@@ -2,4 +2,10 @@
 
 Project management is done with [uv](https://docs.astral.sh/uv/).
 
-For example, execute `uv run python -m audit_translations de` to see the translation progress for the German language.
+For example, execute `uv run audit-translations de` to see the translation progress for the German language.
+
+If you run from the repo root instead of inside `PythonScripts`, point uv at the project and make sure you've synced once:
+```bash
+uv sync --project PythonScripts
+uv run --project PythonScripts audit-translations de
+```

PythonScripts/audit_translations/README.md

@@ -56,45 +56,54 @@ The tool automatically adjusts its matching logic based on the file type:
 
 **Syntax:**
 ```bash
-uv run python -m audit_translations <language> [--file <specific_file>]
-uv run python -m audit_translations --list
+uv run audit-translations <language> [--file <specific_file>]
+uv run audit-translations --list
+
+# If running from the repo root, point uv at the project:
+uv run --project PythonScripts audit-translations <language>
+uv run --project PythonScripts audit-translations --list
 ```
 
 **Convenience Features:**
 * `--list`: Displays all available languages.
   * Region variants are shown as `lang-region` (e.g., `zz-aa`) based on subdirectories under `Rules/Languages/<lang>`.
 * `--file`: Audits a single specific file instead of the whole directory.
-* `--format`: Output format (`rich`, `jsonl`). `--output` is honored only for `jsonl`; rich output always prints to the console.
 * `--rules-dir`: Override the Rules/Languages directory path.
 * `--only`: Filter issue types (comma-separated): `missing`, `untranslated`, `extra`, `diffs`, `all`.
-* `--verbose`: Show detailed output including English/translated snippets for rule differences (only affects rich format; default shows summary only).
+* `--verbose`: Show detailed output including English/translated snippets for rule differences.
 * **Summary Stats:** Provides a statistical summary after every run.
 
 **Examples:**
 
 ```bash
 # List available languages
-uv run python -m audit_translations --list
+uv run audit-translations --list
+
+# Same from repo root
+uv run --project PythonScripts audit-translations --list
 
 # Audit all Spanish translation files
-uv run python -m audit_translations es
+uv run audit-translations es
 
 # Audit German translations
-uv run python -m audit_translations de
+uv run audit-translations de
 
 # Audit only a specific file
-uv run python -m audit_translations es --file SharedRules/default.yaml
-
-# Produce JSONL output for automation or AI workflows
-uv run python -m audit_translations es --format jsonl --output es-issues.jsonl
+uv run audit-translations es --file SharedRules/default.yaml
 
 # Audit a regional variant (merges Rules/Languages/de and Rules/Languages/de/CH)
-uv run python -m audit_translations de-CH
+uv run audit-translations de-CH
 
 # Show detailed output with English/translated snippets for rule differences
-uv run python -m audit_translations es --verbose
+uv run audit-translations es --verbose
+```
+
+**Running from the repo root (without `cd PythonScripts`):**
+```bash
+uv run --project PythonScripts audit-translations es
+uv run --project PythonScripts audit-translations --list
 ```
 
 ### Testing
 
-```uv run python -m pytest```
+```uv run pytest```

PythonScripts/audit_translations/__init__.py

@@ -7,10 +7,9 @@
 
 Read README.md for more details.
 """
-import sys
-sys.stdout.reconfigure(encoding='utf-8')
+
 from .cli import main
 
 __all__ = [
-    'main',
+    "main",
 ]