V 2.0.0#3
Merged
Merged
Conversation
…rors BREAKING CHANGE: Model2Vec is now a stateless namespace of static methods. Native library seam (Candidates 1 & 4) - Switch ffigen to @Native code-assets mode (assetId package:model2vec/model2vec.so); the SDK resolver — fed by the build hook — now locates the native library. Delete _resolveLibPath, boot(), the singleton instance, both DynamicLibrary constructors, and the _cachedDimension cache. Fixes Flutter bundling and the boot()-across-isolates bug. Native ABI and error seam (Candidate 3) - Rust now owns output allocation: generate_* return ptr+dim(+count), freed via free_floats — removing the dim/model-switch buffer-overflow race by construction. Every fallible function returns a stable code plus a char** out_error, and every #[no_mangle] is wrapped in catch_unwind so a panic (including from a malformed model) becomes a typed error instead of UB. Length params use size_t (fixes a 64-bit-Windows usize mismatch). Dart surfaces a single Model2VecException with an exhaustive Model2VecErrorKind. Streaming worker (Candidate 2) - Rebuild generateEmbeddingStream on small, tested modules: batched() (pure transformer), Channel (isolate-port seam + in-memory fake), WorkerProtocol (serialized-queue state machine) and a private EmbeddingWorker. Worker errors cross the isolate boundary as typed exceptions; a dying worker fails pending requests via onExit instead of hanging. Catalog (Candidate 5) - getRecommendedModels() -> typed Model2Vec.recommendedModels constant. Tests - 60 tests. The model-backed suite is migrated to the static API; new no-model/no-network suites cover batching, the worker protocol (fake channel), the worker over a real isolate (incl. death/hang robustness) and native error paths. dart_test.yaml pins concurrency: 1 because the native model is a single process-global. Full migration guide is in CHANGELOG.md.
Adds the retrieval/RAG layer on top of the embedding engine, plus lifecycle and parallelism helpers. - EmbeddingIndex: in-memory vector store (add/search/remove) with cosine ranking, optional int8-quantized storage (~4x less memory), and binary toBytes/fromBytes persistence — a local retrieval engine for RAG. - RAG helpers: chunkText (overlapping character chunker, maxChars-bounded), Model2VecUtils.similaritySearchWithScores (index + score), and maximalMarginalRelevance (MMR reranking, O(k*n*dim)). - Lifecycle & DX: Model2Vec.isInitialized (non-throwing), unloadModel() (frees native memory via a new free_embedder FFI), modelInfo (metadata in one ModelInfo), Model2VecUtils.dequantizeInt8. - EmbeddingPool: N worker isolates embedding concurrently (the Rust RwLock allows concurrent readers), with order-preserving embedBatches and leak-safe startup. - Native: is_model_loaded + free_embedder FFI functions; bindings regenerated. 92 tests, including no-model/no-network suites for the index (persistence, quantization, corrupt-blob handling), the chunker, MMR, and the pool (fake entry point). README recipes and CHANGELOG updated.
Loading is the heaviest, most download-prone step and had no async door, while microsecond generate* calls did — inverted from cost. On Flutter the synchronous initEmbedder freezes the UI during the first model download. Add initEmbedderAsync and initEmbedderAdvancedAsync, which load on a background isolate via Isolate.run. The native model is a process-global (the worker pool already relies on this), so the model loaded off-thread is visible on every isolate, including the one that awaited the call.
The index stored id->vector only, so every retrieval caller kept a parallel
id->text map and re-joined it on each hit (the RAG example and README recipe
both did exactly this).
Add an optional String payload per entry: add(id, vector, {payload}),
surfaced as SearchResult.payload and persisted through toBytes. The blob
format bumps to v2 (a per-entry payload section with a null sentinel distinct
from an empty string); v1 blobs still decode. payload is a String (not a
generic) so the index stays a pure, serializable data structure — hold JSON
for structured metadata. The RAG example drops its parallel passages map.
Model2VecUtils had three near-identical searches returning three shapes: similaritySearch and similaritySearchWithThreshold handed back bare List<int> indices (forcing callers to re-associate), while similaritySearchWithScores returned (index, score) records. Make similaritySearchWithScores the single scored search — it now takes an optional score threshold alongside topK (idiomatic topK + min-score), so it subsumes both query modes. The two bare-index methods become @deprecated one-line delegates over it, so existing callers keep working for one release. Removes the duplicated scoring loops and the private _IndexedSimilarity type; migrates the example and README to the scored shape.
The public lifecycle verbs mixed two nouns — initEmbedder* to load vs unloadModel/modelInfo — and 'Embedder' is a term CONTEXT.md explicitly says to avoid; the load/unload inverse was hidden in the names. Rename the five loaders to loadModel, loadModelAdvanced, loadModelFromBytes, loadModelAsync and loadModelAdvancedAsync, so the surface reads loadModel <-> unloadModel with one noun (Model). The old initEmbedder* names stay as @deprecated delegating aliases for one release (removed in 3.0.0). The native init_embedder entry points stay internal. Adds a Load/Unload term to CONTEXT.md and migrates the example, benchmark, tests and README.
generateBatchEmbeddings and its async form exposed a batchSize parameter that the README itself described as 'internal chunks sent to the FFI layer' — a native throughput detail the caller has no basis to choose, and one that never affects the result. Remove it from both signatures and pass a fixed internal chunk size (1024, the former default, so results are unchanged) to the native call. The stream's batchSize stays: there it is a meaningful, user-facing chunk size. Internal callers stop threading batch.length through.
Loading a model is a single blocking FFI call, and the first fetch pulls tens to hundreds of MB of weights from Hugging Face with no way to observe it — a Flutter app can only show an indeterminate spinner during the heaviest step of all. Add loadModelWithProgress, which loads on a background isolate and returns a Stream<LoadProgress> reporting the weights download (bytesDownloaded / totalBytes / fraction) and a coarse phase: resolving -> downloading -> parsing -> done. The Rust side routes the weights fetch through hf-hub's download_with_progress into process-global atomics (a cache hit skips it entirely), read over FFI via new get_load_progress / reset_load_progress. A cached model or local path streams straight to done; the stream always ends on done and surfaces a load failure as an error event. Adds an integration test that downloads into a fresh cache, plus README and CHANGELOG entries. Also reorganize example/ so each file teaches one story: main.dart is now a small quickstart (load -> embed -> cosine similarity), scaling_example.dart covers the at-scale patterns (progress-bar load, batch, EmbeddingPool, streaming), and rag_example.dart keeps the retrieval pipeline. Adds example/README.md indexing the three.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.