docs: audio engine architecture + AI agent CLAUDE.md - #16
Open
mogul wants to merge 2 commits into
Open
Conversation
New docs/audio-engine-architecture.md describes the native audio + ML detection internals: the audio device callback, the lock-free input ring buffer, two-tier pitch detection (YIN + Basic Pitch via ONNX), the ChordScorer constraint path, the NoteVerifier background thread, the N-API surface, and the threading invariants every contributor needs to keep in mind. The detection pipeline now has substantial moving parts (input ring, two detectors, one verifier thread, sandboxed plugin hosting, split-mode duplex) and the only documentation lived in opening comments at the top of each source file. This doc is the picture, with a component map up top and cross-references to the actual code. Renderer-side companion documented in a sibling PR against byrongamatos/slopsmith. Signed-off-by: Kris Anderson <topkoa@gmail.com> (cherry picked from commit 5a96b33eef90f28e9e7f1091c85e0606ddc260fe)
slopsmith-desktop had no CLAUDE.md, so an AI agent landing in this repo had no obvious starting point — README.md is install-focused (Windows/macOS/Linux download instructions) and wouldn't lead an agent to docs/. Adding a brief root-level guide that: - Orients the agent on what this repo is vs the sister byrongamatos/slopsmith repo (audio engine + Electron shell here; library + chart parsing + visualization there) - Indexes docs/ contents - Cross-links the slopsmith-side docs that describe what consumes the IPC surface this engine exposes - Captures the threading invariants, build-flag list, common gotchas (audio rebuild lock, ELECTRON_RUN_AS_NODE, port 18000) in one place rather than scattered across CLAUDE.md fragments in the dev workspace Modelled on slopsmith/CLAUDE.md's style — technical-but- conversational, ATX headings, file paths in backticks, link-heavy. Keeps the new audio-engine-architecture.md as the canonical deep- dive; this file is a navigation hub. Signed-off-by: Kris Anderson <topkoa@gmail.com> (cherry picked from commit caac6aa9ea8354a489ee557a54193415fe416c9a)
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two new docs that make the native audio engine — and the repo as a whole — discoverable to both AI agents and human maintainers:
docs/audio-engine-architecture.md— describes the native audio + ML detection internals: the audio device callback, the lock-free input ring buffer, the two-tier pitch detection (YIN + Basic Pitch via ONNX), theChordScorerconstraint path, theNoteVerifierbackground thread, the N-API surface, and the threading invariants every contributor needs to keep in mind. ~12 KB markdown.CLAUDE.mdat repo root — brief AI agent entry point. Orients on what this repo is vs the sisterbyrongamatos/slopsmithrepo, indexesdocs/, cross-links the slopsmith-side docs that describe what consumes the IPC surface this engine exposes, captures threading invariants + build flags + common gotchas in one place.Motivation
The detection pipeline has substantial moving parts (input ring, two detectors, one verifier thread, sandboxed plugin hosting, split-mode duplex for ASIO) and the only documentation lived in opening comments at the top of each source file. A maintainer wanting to debug into the audio path had to read AudioEngine + PitchDetector + MlNoteDetector + ChordScorer + NoteVerifier + NodeAddon + audio-bridge to assemble the picture.
Separately, an AI agent landing in this repo had no obvious starting point —
README.mdis install-focused and wouldn't lead them todocs/. The newCLAUDE.mdis the navigation hub that other repos in the slopsmith ecosystem already have.Audience
Both new files target engineers extending or debugging the C++ engine, the ML detector, or the IPC layer. The architecture doc is deliberately denser (assumes JUCE /
juce::Thread/ atomics fluency); the CLAUDE.md is a brief orientation that links into it.Companion PR
The renderer-side picture — how
note_detectconsumes these IPC methods, registers thesetNoteStateProvider, and feeds the highway — is documented in a sibling draft PR: byrongamatos/slopsmith#414 —docs: add real-time scoring + hit-feedback pipeline docs. The new docs in this PR link to it; it links back here.Test plan
mainSLOPSMITH_ONNX_SUPPORT) matchMlNoteDetector.h/MlNoteDetector.cppaudioDeviceIOCallbackWithContext)src/and the cross-repo split (audio engine here, library/chart/viz there)🤖 Generated with Claude Code