Skip to content

Add Supertonic FP16 MLX export and native runner - #22063

Open
seyeong-han wants to merge 3 commits into
pytorch:mainfrom
seyeong-han:supertonic/01-runtime
Open

Add Supertonic FP16 MLX export and native runner#22063
seyeong-han wants to merge 3 commits into
pytorch:mainfrom
seyeong-han:supertonic/01-runtime

Conversation

@seyeong-han

@seyeong-han seyeong-han commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Add an ExecuTorch MLX implementation of Supertonic 3, Supertone's on-device multilingual TTS model.

Published ONNX models
  -> export-only Python code
  -> one FP16 ExecuTorch PTE
  -> C++ runner with MLX
  -> 44.1 kHz PCM16 WAV

The Python model files translate the published ONNX architecture and weights into an ExecuTorch PTE. They are needed only when creating the PTE; the C++ runner does not use Python or the source ONNX model files.

This PR supports one-shot, batch-1, five-step synthesis on macOS arm64. It does not include model weights, generated PTEs, WAVs, a persistent server, benchmarks, or application integration.

What to review

The 49 files do not need equal review depth.

1. Create the PTE (Python, export time only)

  • examples/models/supertonic/loaders/checkpoint_loader.py
  • examples/models/supertonic/model/duration_predictor.py
  • examples/models/supertonic/model/text_encoder.py
  • examples/models/supertonic/model/vector_estimator.py
  • examples/models/supertonic/model/vocoder.py
  • examples/models/supertonic/export/common.py
  • examples/models/supertonic/export/export_supertonic.py
  • examples/models/supertonic/source_transformations/mlx.py

These files define the four Supertonic stages in PyTorch, copy and convert the ONNX weights, apply MLX-compatible graph rewrites, and export four PTE methods: duration_predictor, text_encoder, vector_estimator, and vocoder.

Review the layer/weight mapping, equivalent outputs, dynamic bounds, FP16 conversion, metadata, and MLX delegation. The highest-signal tests are test_checkpoint_loader.py, test_stage_parity.py, and test_mlx_pipeline.py.

2. Run the PTE (C++)

  • examples/models/supertonic/runtime/supertonic_runner.cpp
  • examples/models/supertonic/runtime/text_processor.cpp
  • examples/models/supertonic/runtime/style_loader.cpp
  • examples/models/supertonic/runtime/wav_writer.cpp
  • examples/models/supertonic/runtime/main.cpp

These files load the PTE, call its four methods, preprocess and chunk text, run five generation steps, trim the waveform, and write a WAV file. Review the PTE compatibility checks, Unicode handling, tensor flow, and output validation.

The highest-signal tests are examples/models/supertonic/runtime/tests/supertonic_runtime_test.cpp and examples/models/supertonic/runtime/tests/run_integration.cmake.

3. Build and CI

  • examples/models/supertonic/CMakeLists.txt
  • examples/models/supertonic/CMakePresets.json
  • .github/workflows/mlx.yml
  • examples/models/supertonic/README.md
  • examples/models/supertonic/NOTICE

Review the macOS arm64 restriction, MLX runtime files, pinned-asset CI flow, usage, and licensing. The remaining unit tests, package files, and root README/Makefile entries are support changes that can be skimmed.

Runtime files

After export, synthesis needs only supertonic_runner, mlx.metallib, the exported PTE, unicode_indexer.json, and one voice-style JSON file.

Validation

  • 151 Python tests and 2/2 native CTests passed, including real PTE synthesis.
  • Local Apple M5 Max benchmark (batch-1 FP16, five steps): short-input median 45.9 ms, RTF 0.0244 (40.9x real time).
  • ONNX FP32 vs MLX FP16 waveform cosine/Pearson: 0.999603; SQNR: 30.95 dB.
  • GitHub CI: 201 successful, 30 skipped, 1 neutral, 0 failed.

Provenance

  • Source: supertone-inc/supertonic at 7e2804f96016a7028cb1ed627353c61c1e9dd281 (MIT).
  • Model assets: Supertone/supertonic-3 at 3cadd1ee6394adea1bd021217a0e650ede09a323 (BigScience Open RAIL-M).
  • Assets are downloaded separately and are not redistributed. Exported PTEs remain subject to the model asset license.

@pytorch-bot

pytorch-bot Bot commented Aug 23, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22063

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 39c750c with merge base cff6f4d (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 23, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@seyeong-han
seyeong-han force-pushed the supertonic/01-runtime branch from a31b87f to 27f7331 Compare August 23, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant