[RFC] Single-archive model distribution format for LLM

[kirklandsign]

🚀 The feature, motivation and pitch

Problem

Deploying an ExecuTorch LLM requires managing multiple separate files:

• .pte — exported program
• .ptd — optional external weights
• Tokenizer file (tokenizer.json, .model, .bin)
• Prompt template / chat template — not stored anywhere, app must hard-code per model family
• Generation config (num_bos, num_eos, etc.) — caller must set correctly

Some metadata lives in .pte via constant_methods (get_max_seq_len, get_eos_ids, etc.), but tokenizer and chat template are external. Users can easily mismatch tokenizer and model, or use the wrong prompt format.

Proposal

A zip archive (.etm — ExecuTorch Model) bundling everything:

model.etm
├── model.pte
├── tokenizer.json
├── metadata.json          # chat_template, num_bos, default_temperature, etc.
└── weights/               # optional
    └── foundation.ptd

metadata.json carries what's not in constant_methods today — most importantly chat_template, so the runner can format prompts without app-side logic.

Prior Art

• MediaPipe Tasks (.task): TFLite model + tokenizer + metadata in a single zip-like bundle
• GGUF (llama.cpp): single file embedding tokenizer vocab and metadata alongside weights
• Hugging Face: config.json + tokenizer.json + chat_template in a model repo

Open Questions

1. File extension: .etm? .etb? Just .zip?
2. LLM-only initially, or general-purpose for all ExecuTorch models?
3. Chat template format: Jinja2 (HF-compatible) or simpler substitution?
4. Runtime loading: unzip to temp dir vs. read from zip in-memory?
5. Keep supporting multi-file approach alongside?

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

[JacobSzwejbka]

File extension: .etm? .etb? Just .zip?

.etlm?

if we want to stick to 3 then et Bundle (.etb) or et Package (.etp) are both fine to me

Runtime loading: unzip to temp dir vs. read from zip in-memory?

Just dont actually compress it like .pt and .pt2 so you can mmap

Chat template format: Jinja2 (HF-compatible) or simpler substitution?

thats fine we can expand it later

Keep supporting multi-file approach alongside?

Any api built around this would by necessity just wrap the previous apis so theres no deprecation here

LLM-only initially, or general-purpose for all ExecuTorch models?

LLM only I would say. If a user wants to bundle other files for non structured models they can already just do that.

In general I think it sounds fine since we have some opinionated apis like extension/llm, the demo apps, and optimum to create a folder archive for them. I will say Im prototyping something for tokenizer as a delegate already though. I would also like if we are doing this to stop putting metadata in the pte and instead put it all in metadata.json.

[JacobSzwejbka]

Separately if we add loaders we could presumably support safetensors and gguf as other weight formats fwiw. Not critical path here

[larryliu0820]

Overall seems like a good feature! We should include chat template if possible. We have a plan to support chat_template.jinja2 #16987