Skip to content

feat: add ExecuTorch runtime - #18

Open
leeclemnet wants to merge 2 commits into
feat/cpu-runtimesfrom
feat/executorch-runtime
Open

leeclemnet wants to merge 2 commits into
feat/cpu-runtimesfrom
feat/executorch-runtime

Conversation

@leeclemnet

@leeclemnet leeclemnet commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

ExecuTorch runtime

What it does: Adds ExecuTorchRuntime (XNNPACK, device cpu) on the Runtime contract. It loads a .pte program and times only Method.execute(). No script rows use it yet.
Why: A .pte comes from torch.export on the PyTorch model, not from ONNX, and the exported artifacts are not in the bucket yet. This PR lands the runtime, so the rows are only a config change when the artifacts exist. Stacked on #16.

Change map

  • Runtime (sab/runtimes/executorch.py) — inputs are cast to the program's dtypes before the timed call. Outputs are cloned after it, because ExecuTorch can reuse its output buffers on the next call. Inputs are named by position (input0, input1), because a .pte has no input names.
  • Tests (tests/test_executorch.py) — build tiny .pte programs with torch.export and XnnpackPartitioner, and check run() against the eager model.
  • Packaging and CI (pyproject.toml, uv.lock, .github/workflows/tests.yml, README.md) — extra executorch. It is not in nvidia, because the NVIDIA host runs no ExecuTorch rows. CI installs it.

Behavior changes

  • None. No script row uses the runtime.

Watch out for

  • sab/runtimes/executorch.py:81 — the outputs are cloned after the timed call. Without the clone, a later call overwrites the outputs of an earlier one.
  • pyproject.toml:34 — executorch pulls a large dependency set (uv.lock grows by about 800 lines). The lock keeps torch at 2.14.
  • Only the XNNPACK backend on cpu is in this PR. The Core ML backend (npu) comes with the Mac PR.

Related Issue(s): n/a

Type of Change

  • New feature (non-breaking change that adds functionality)

Testing

  • Unit tests
  • ExecuTorch XNNPACK vs ORT CPU on an Apple M4 Max (50 coco val2017 images)

CPU: ExecuTorch (XNNPACK) on an Apple M4 Max (fp32, on battery). The artifacts are the official exports in gs://rfdetr/export-2026-09-30. ORT reads the .onnx of the same export. YOLO26 has no official exports, so it has no rows.

Model ORT CPU mAP50:95 ExecuTorch fp32 mAP50:95 ORT CPU median ms ExecuTorch fp32 median ms
rfdetr-nano 53.9 53.9 34.5 51.7
rfdetr-small 58.3 58.3 58.3 84.9
rfdetr-medium 57.1 57.1 68.3 120.9
rfdetr-large 60.2 60.2 123.4 192.8
rfdetr-xlarge 61.1 61.1 234.9 342.8
rfdetr-2xlarge 62.3 62.3 322.3 416.9

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code where necessary, particularly in hard-to-understand areas
  • My changes generate no new warnings or errors
  • I have updated the documentation accordingly (if applicable)

Additional Context

🤖 Generated with Claude Code

@socket-security

socket-security Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedexecutorch@​1.5.171100100100100

View full report

leeclemnet and others added 2 commits September 30, 2026 15:34
Add ExecuTorchRuntime (XNNPACK, cpu) on the Runtime contract. It loads a
.pte program and times only Method.execute(). Inputs are cast to the
program's dtypes before the timed call, and outputs are cloned after
it, because ExecuTorch can reuse its output buffers.

No script rows use the runtime yet: a .pte comes from torch.export on
the PyTorch model, not from ONNX, and the exported artifacts are not in
the bucket. The tests build tiny .pte programs with torch.export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nnot fail

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leeclemnet
leeclemnet force-pushed the feat/executorch-runtime branch from 56ecd8f to e0c7228 Compare September 30, 2026 19:35
@leeclemnet
leeclemnet marked this pull request as ready for review September 30, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant