recipe: tflite-runtime 2.21.0

[ndonkoHenri]

Adds a tflite-runtime recipe — LiteRT / .tflite model inference on both Android and iOS.
Closes flet-dev/flet#5135 and flet-dev/flet#3810 (and relevant to flet-dev/flet#4901).

Shape

tflite-runtime has never had an sdist — upstream wheels come from tensorflow/lite/tools/pip_package/build_pip_package_with_cmake.sh, a host==target script. So this uses the PEP 517 shim archetype (same as onnxruntime): the source is the TensorFlow v2.21.0 release tree, and mobile.patch adds a pyproject.toml + _forge/forge_tflite_backend.py that builds a standalone host flatc, cross-compiles tensorflow/lite down to just the _pywrap_tensorflow_interpreter_wrapper pybind module, then stages upstream's exact 5-file tflite_runtime/ package. Wheels are 1.9–2.8 MB.

2.21 is the last buildable line — the ai-edge-litert successor is a Bazel wall (no CMake path), so pinning here is deliberate. Both historical p4a patches (-llog, minimal_logging dedup) are already fixed upstream in 2.21.

Notable cross-compile details (each documented in the recipe/patch):

• -DTENSORFLOW_SOURCE_DIR pinned to the unpacked root — unset, lite/'s CMake git-clones its own TF at v2.19.0 and compiles 2.21 sources against 2.19 headers.
• XNNPACK is link-mandatory for the Python wrapper on every ABI (the wrapper references TfLiteXNNPackDelegate* unconditionally); stays ON except 32-bit arm, following upstream's own pip convention.
• iOS (3 tells): wrapper has no FindPython → -Wl,-undefined,dynamic_lookup (Android links libpython explicitly instead); FetchContent'd flatbuffers' flatc install rejected by iOS's forced MACOSX_BUNDLE → -DFLATBUFFERS_BUILD_FLATC=OFF -DFLATBUFFERS_INSTALL=OFF (host flatc does codegen); apple SHARED wrapper is a .dylib → staged under the .so name.

Testing

tests/test_tflite_runtime.py runs a committed 1 KB dense_relu.tflite (generated and validated with desktop TF 2.21, fixed weights + seed 42) through a real Interpreter.invoke() — the CI emulator/simulator runs actual XNNPACK inference and asserts desktop-recorded outputs to 1e-6, plus a re-set/second-invoke test. Test logic was pre-validated on desktop TF via a tflite_runtime.interpreter → tf.lite module alias, which caught a real numeric bug before any device.

Follow-ups (not in this PR)

Delegates beyond XNNPACK (GPU / NNAPI / Core ML) if there's demand — the default XNNPACK-accelerated CPU path is the portable baseline.

Consumer notes (usage & recommendations)

tflite-runtime runs .tflite models through the XNNPACK-accelerated CPU path — the portable, dependable baseline that works identically on Android and iOS. Everything runs offline; the only network you need is a one-time model download (or bundle the model — see below). The pinned version is 2.21.0, the last buildable release — its ai-edge-litert successor can't be cross-compiled, so stay on this package name.

Install

Add it to your deps (the pip name has a hyphen, the import has an underscore):

# pyproject.toml
dependencies = [
    "flet",
    "tflite-runtime",
    "numpy",
]

All Android ABIs are covered (arm64-v8a, armeabi-v7a, x86_64) — though on old 32-bit armeabi-v7a the XNNPACK accelerator is off, so prefer arm64 devices.

Storage & the model file

.tflite models are usually small, so you have two options:

• Small model (a few MB): ship it along with your app (ex: in the assets/ directory) and load from the bundle — simplest.
• Larger model: don't bloat the app bundle — download it once on first run into Flet's persistent storage (FLET_APP_STORAGE_DATA) and load from there:

import os, urllib.request

data_dir = os.getenv("FLET_APP_STORAGE_DATA")
model_path = os.path.join(data_dir, "model.tflite")
if not os.path.exists(model_path):
    urllib.request.urlretrieve("https://<your-host>/model.tflite", model_path)

Minimal usage

import os
import numpy as np
from tflite_runtime.interpreter import Interpreter

model_path = os.path.join(os.getenv("FLET_APP_STORAGE_DATA"), "model.tflite")

# num_threads: set to your device's performance-core count (often 4).
interpreter = Interpreter(model_path=model_path, num_threads=4)
interpreter.allocate_tensors()

inp = interpreter.get_input_details()[0]
out = interpreter.get_output_details()[0]

x = np.asarray(data, dtype=np.float32)      # match inp["shape"] / inp["dtype"]
interpreter.set_tensor(inp["index"], x)
interpreter.invoke()
y = interpreter.get_tensor(out["index"])

Run it off the UI thread

allocate_tensors() and invoke() are CPU-bound — run them in page.run_thread(...) so the UI stays responsive.

Notes for good results on mobile

• Quantized (INT8) models feed/read raw uint8/int8, not float — use the inp["quantization"] (scale, zero-point) to convert. They're smaller and faster, ideal for mobile.
• Dynamic input shapes: call interpreter.resize_tensor_input(inp["index"], new_shape) then allocate_tensors() again before setting the tensor.
• XNNPACK is applied automatically for float models — no delegate setup needed.
• You may see a cosmetic warning about ai_edge_litert; ignore it (that successor isn't buildable for mobile, which is why we pin this package).