Skip to content

ANEForge - using the tool

ANEForge is a CoreML-free Python frontend for the Apple Neural Engine: it lowers a tensor graph into one fused Espresso e5rt program and dispatches it to ANE silicon from an ordinary user process, with no CoreML and no special entitlement. These docs cover how to install, call, train, target, and extend the frontend.

By task

Get something running

Run pretrained models

  • LLMs on the ANE - af.load_llm prefill + resident-KV-cache decode for Llama/Qwen/Mistral/Gemma/GPT-2/MoE, speculative decoding, and MoE from GGUF.
  • Model loaders - vision (ResNet / ViT / CLIP), sentence encoders and rerankers, GPT-2, and Whisper speech-to-text (both towers on the ANE).
  • ONNX import - run an existing .onnx model on the engine via af.load_onnx.

Use the engine well

  • Cross-chip deployment - compiling and gating for other ANE families (M1-M5, 28 targets), cross_compile_check, detect_family, fp16 portability.
  • Dispatch backends - Path A vs e5rt vs MPSGraph vs CoreML, and which to use.
  • e5rt dispatch reference - the full e5rt path: call sequence, the ane_e5rt_* C ABI, multi-op / async / pipelining / IOSurface.

Know what works

  • Capabilities - operator coverage, dtype matrix, known limits.
  • Op catalog - every native MIL op x device (M1-M5), generated from the package's _op_catalog.py (the runtime af.op_info data); the exhaustive Y/~/N table.

Contribute

  • Development - building, testing, adding ops.
  • Glossary - terminology used across docs and code.
  • Roadmap - next directions, open unknowns, and known bottlenecks.

By question

Question Document
How do I install + run? getting-started
How do I use the Python frontend? aneforge-api
How do I train a model on the ANE? training
How do I run an LLM, Whisper, or ONNX model? llm, model loaders, onnx
Can I target / deploy to another chip (M1-M5)? cross-chip
How do I estimate latency without the hardware? aneforge-api: cost estimation, cross-chip
How do I shrink weights (int4 / sparse)? aneforge-api: weight compression
What ops are supported? capabilities, op-catalog
What's "Path A"? glossary, dispatch
How do I write a MIL program? mil-primer
Why fp16? faq, capabilities
Can I use this in production? faq
What macOS versions are supported? faq
How do I add a new operator? development
Why does my call take 195 ms? dispatch