Skip to content

docs: audio engine architecture + AI agent CLAUDE.md - #16

Open
mogul wants to merge 2 commits into
mainfrom
docs/audio-engine-architecture
Open

docs: audio engine architecture + AI agent CLAUDE.md#16
mogul wants to merge 2 commits into
mainfrom
docs/audio-engine-architecture

Conversation

@mogul

@mogul mogul commented Jun 18, 2026

Copy link
Copy Markdown

Reconstructed from slopsmith/slopsmith-desktop#249 — original PR by @topkoa. Commits replayed with original authorship onto the current main.

Summary

Two new docs that make the native audio engine — and the repo as a whole — discoverable to both AI agents and human maintainers:

  • docs/audio-engine-architecture.md — describes the native audio + ML detection internals: the audio device callback, the lock-free input ring buffer, the two-tier pitch detection (YIN + Basic Pitch via ONNX), the ChordScorer constraint path, the NoteVerifier background thread, the N-API surface, and the threading invariants every contributor needs to keep in mind. ~12 KB markdown.
  • CLAUDE.md at repo root — brief AI agent entry point. Orients on what this repo is vs the sister byrongamatos/slopsmith repo, indexes docs/, cross-links the slopsmith-side docs that describe what consumes the IPC surface this engine exposes, captures threading invariants + build flags + common gotchas in one place.

Motivation

The detection pipeline has substantial moving parts (input ring, two detectors, one verifier thread, sandboxed plugin hosting, split-mode duplex for ASIO) and the only documentation lived in opening comments at the top of each source file. A maintainer wanting to debug into the audio path had to read AudioEngine + PitchDetector + MlNoteDetector + ChordScorer + NoteVerifier + NodeAddon + audio-bridge to assemble the picture.

Separately, an AI agent landing in this repo had no obvious starting point — README.md is install-focused and wouldn't lead them to docs/. The new CLAUDE.md is the navigation hub that other repos in the slopsmith ecosystem already have.

Audience

Both new files target engineers extending or debugging the C++ engine, the ML detector, or the IPC layer. The architecture doc is deliberately denser (assumes JUCE / juce::Thread / atomics fluency); the CLAUDE.md is a brief orientation that links into it.

Companion PR

The renderer-side picture — how note_detect consumes these IPC methods, registers the setNoteStateProvider, and feeds the highway — is documented in a sibling draft PR: byrongamatos/slopsmith#414docs: add real-time scoring + hit-feedback pipeline docs. The new docs in this PR link to it; it links back here.

Test plan

  • All file:line citations and quoted code verified against current main
  • Build flag references (SLOPSMITH_ONNX_SUPPORT) match MlNoteDetector.h / MlNoteDetector.cpp
  • ASCII component map matches the actual class layout (NoteVerifier, MlNoteDetector, PitchDetector, ChordScorer)
  • CLAUDE.md cross-links to the slopsmith repo resolve
  • Reviewer eyeballs the "Threading invariants" section against current code (esp. lock-free guarantees in audioDeviceIOCallbackWithContext)
  • Reviewer eyeballs CLAUDE.md's repo orientation — particularly the file-map under src/ and the cross-repo split (audio engine here, library/chart/viz there)

🤖 Generated with Claude Code

topkoa added 2 commits June 17, 2026 23:41
New docs/audio-engine-architecture.md describes the native audio + ML
detection internals: the audio device callback, the lock-free input
ring buffer, two-tier pitch detection (YIN + Basic Pitch via ONNX),
the ChordScorer constraint path, the NoteVerifier background thread,
the N-API surface, and the threading invariants every contributor
needs to keep in mind.

The detection pipeline now has substantial moving parts (input ring,
two detectors, one verifier thread, sandboxed plugin hosting,
split-mode duplex) and the only documentation lived in opening
comments at the top of each source file. This doc is the picture,
with a component map up top and cross-references to the actual code.

Renderer-side companion documented in a sibling PR against
byrongamatos/slopsmith.

Signed-off-by: Kris Anderson <topkoa@gmail.com>
(cherry picked from commit 5a96b33eef90f28e9e7f1091c85e0606ddc260fe)
slopsmith-desktop had no CLAUDE.md, so an AI agent landing in this
repo had no obvious starting point — README.md is install-focused
(Windows/macOS/Linux download instructions) and wouldn't lead an
agent to docs/. Adding a brief root-level guide that:

- Orients the agent on what this repo is vs the sister
  byrongamatos/slopsmith repo (audio engine + Electron shell here;
  library + chart parsing + visualization there)
- Indexes docs/ contents
- Cross-links the slopsmith-side docs that describe what consumes
  the IPC surface this engine exposes
- Captures the threading invariants, build-flag list, common
  gotchas (audio rebuild lock, ELECTRON_RUN_AS_NODE, port 18000)
  in one place rather than scattered across CLAUDE.md fragments
  in the dev workspace

Modelled on slopsmith/CLAUDE.md's style — technical-but-
conversational, ATX headings, file paths in backticks, link-heavy.
Keeps the new audio-engine-architecture.md as the canonical deep-
dive; this file is a navigation hub.

Signed-off-by: Kris Anderson <topkoa@gmail.com>
(cherry picked from commit caac6aa9ea8354a489ee557a54193415fe416c9a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants