Google has introduced the LiteRT Tensor API, a lightweight C++ interface for building, customizing and optimizing on-device machine-learning models. The announcement was reported on November 13, 2026.
The API lets developers assemble a computation graph operator by operator. Google says the same graph can run through LiteRT on CPU, GPU and NPU hardware without rewriting it for each target. It is intended to offer graph-level control while reducing the work of managing hardware-specific code and connecting preprocessing, inference and postprocessing.
Google says developers can import weights directly from checkpoints, including Hugging Face .safetensors files, and use zero-copy buffers across processing stages. AI coding agents can also generate C++ code to build, modify, validate and benchmark model graphs.
As a demonstration, Google used a coding agent to build a multi-object segmentation and tracking app. It ran the same SAM 2 C++ model code on a macOS laptop and an iOS phone using Metal, and an Android phone using OpenCL. In Google’s tests, the MacBook Pro run was about 1.5 times faster than Apple MLX at fp16 for the image encoder and mask decoder. Google also said the agent moved frame-by-frame decisions into the graph and replaced operations that caused CPU fallback on WebGPU.
Commenting on the announcement, Arm AI director Laurence Moroney advised developers to start with model conversion and consider the C++ API when they need graph-level control or want preprocessing and postprocessing to remain on an accelerator. He cited a WebGPU improvement from under 10 fps to over 30 fps at 512×512 resolution during agent-led optimization.
Google’s planned additions include native ML Drift backend support, more NPU-friendly operations, automatic whole-graph quantization and broader dynamic-shape support. These are plans, not current API capabilities.



