{"id":160,"date":"2026-11-18T02:48:16","date_gmt":"2026-11-18T02:48:16","guid":{"rendered":"https:\/\/blockoala.com\/index.php\/2026\/11\/18\/google-introduces-litert-tensor-api-for-building-on-device-ml-models-in-c\/"},"modified":"2026-11-18T02:48:16","modified_gmt":"2026-11-18T02:48:16","slug":"google-introduces-litert-tensor-api-for-building-on-device-ml-models-in-c","status":"publish","type":"post","link":"https:\/\/blockoala.com\/index.php\/2026\/11\/18\/google-introduces-litert-tensor-api-for-building-on-device-ml-models-in-c\/","title":{"rendered":"Google introduces LiteRT Tensor API for building on-device ML models in C++"},"content":{"rendered":"<p>Google has introduced the LiteRT Tensor API, a lightweight C++ interface for building, customizing and optimizing on-device machine-learning models. The announcement was reported on November 13, 2026.<\/p>\n<p>The API lets developers assemble a computation graph operator by operator. Google says the same graph can run through LiteRT on CPU, GPU and NPU hardware without rewriting it for each target. It is intended to offer graph-level control while reducing the work of managing hardware-specific code and connecting preprocessing, inference and postprocessing.<\/p>\n<p>Google says developers can import weights directly from checkpoints, including Hugging Face <code>.safetensors<\/code> files, and use zero-copy buffers across processing stages. AI coding agents can also generate C++ code to build, modify, validate and benchmark model graphs.<\/p>\n<p>As a demonstration, Google used a coding agent to build a multi-object segmentation and tracking app. It ran the same SAM 2 C++ model code on a macOS laptop and an iOS phone using Metal, and an Android phone using OpenCL. <strong>In Google\u2019s tests<\/strong>, the MacBook Pro run was about 1.5 times faster than Apple MLX at fp16 for the image encoder and mask decoder. Google also said the agent moved frame-by-frame decisions into the graph and replaced operations that caused CPU fallback on WebGPU.<\/p>\n<p>Commenting on the announcement, Arm AI director Laurence Moroney advised developers to start with model conversion and consider the C++ API when they need graph-level control or want preprocessing and postprocessing to remain on an accelerator. He cited a WebGPU improvement from under 10 fps to over 30 fps at 512\u00d7512 resolution during agent-led optimization.<\/p>\n<p>Google\u2019s planned additions include native ML Drift backend support, more NPU-friendly operations, automatic whole-graph quantization and broader dynamic-shape support. These are plans, not current API capabilities.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Google has introduced the LiteRT Tensor API, a lightweight C++ interface for building, customizing and optimizing on-device machine-learning [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-160","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/posts\/160","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/comments?post=160"}],"version-history":[{"count":0,"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/posts\/160\/revisions"}],"wp:attachment":[{"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/media?parent=160"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/categories?post=160"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blockoala.com\/index.php\/wp-json\/wp\/v2\/tags?post=160"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}