feat: add LiteRT runtime

[leeclemnet]

What does this PR do?

LiteRT runtime

What it does: Adds LiteRTRuntime on the Runtime contract. It runs .tflite files from both the litert and the tflite exporters, and times only Interpreter.invoke(). No script rows use it yet.
Why: LiteRT cannot read ONNX, and the exported .tflite artifacts are not in the bucket yet. This PR lands the runtime, so the rows are only a config change when the artifacts exist. Stacked on #16.

Change map

• Runtime (sab/runtimes/litert.py) — num_threads=os.cpu_count().
• Conversion (sab/runtimes/convert.py, tests/test_convert.py) — the canonical NCHW float input becomes the artifact layout and dtype (NHWC, int8 with the artifact's scale and zero-point) before the timed call. Outputs are dequantized after it.
• Tests (tests/test_litert.py, tests/data/litert/) — two tiny .tflite fixtures (float32 and int8, NHWC), checked against a numerical reference. generate.py rebuilds them (needs TensorFlow).
• Packaging and CI (pyproject.toml, uv.lock, .github/workflows/tests.yml, README.md) — extra litert (ai-edge-litert). It is not in nvidia, because the NVIDIA host runs no LiteRT rows. CI installs it.

Behavior changes

• None. No script row uses the runtime.

Watch out for

• sab/runtimes/convert.py:49 — int8 inputs are quantized with round-to-nearest and clipped to the dtype range. A different rounding in the exporter moves int8 accuracy a little.
• sab/runtimes/litert.py:35 — the thread count is fixed to all CPUs. On the Pi 5 that is 4 threads, the same as the ORT CPU rows.

Related Issue(s): n/a

Type of Change

• New feature (non-breaking change that adds functionality)

Testing

• Unit tests
• LiteRT vs ORT CPU on an Apple M4 Max (50 coco val2017 images)

CPU: LiteRT on an Apple M4 Max (on battery). The artifacts are the official exports in gs://rfdetr/export-2026-09-30: litert comes from the litert exporter, and tflite from the tflite exporter. ORT reads the .onnx of the same export. This branch has no macOS throttle monitor, so no row has a throttle verdict.

Model	ORT CPU fp32 mAP50:95	litert fp32 mAP50:95	litert fp16 mAP50:95	tflite fp32 mAP50:95	tflite fp16 mAP50:95	ORT CPU fp32 median ms	litert fp32 median ms	litert fp16 median ms	tflite fp32 median ms	tflite fp16 median ms
rfdetr-nano	53.9	53.9	53.8	53.9	53.8	34.5	80.0	81.2	55.8	55.9
rfdetr-small	58.3	58.3	58.2	58.3	58.1	58.3	114.4	115.4	74.8	74.4
rfdetr-medium	57.1	57.1	57.2	57.1	57.2	68.3	144.6	146.0	89.3	89.5
rfdetr-large	60.2	60.2	60.4	60.2	60.2	123.4	167.0	168.5	128.3	118.6
rfdetr-xlarge	61.1	61.1	61.1	61.1	61.0	234.9	527.6	525.6	198.6	205.8
rfdetr-2xlarge	62.3	62.3	62.3	62.3	62.3	322.3	600.6	572.9	274.7	272.9

• Both exporters keep the mAP of ORT within 0.2 at fp32 and fp16.
• The tflite artifacts run 1.3–2.7× faster than the litert artifacts. fp16 gives almost no speedup on this CPU. Only tflite large fp16 is faster than fp32, by 8%.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code where necessary, particularly in hard-to-understand areas
• My changes generate no new warnings or errors
• I have updated the documentation accordingly (if applicable)

Additional Context

• Stacked on feat: add OpenVINO runtime and all RF-DETR detection sizes #16. Merge refactor: compose runtimes and processors, add run control and uv packaging #15 and feat: add OpenVINO runtime and all RF-DETR detection sizes #16 first.

🤖 Generated with Claude Code

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	ai-edge-litert@​2.2.0

View full report