Google opens ML Drift, the GPU engine that runs LiteRT's embedded AI on all types of chips

Under Apache 2.0 license, ML Drift masks the differences between OpenCL, Metal, OpenGL ES and WebGPU to accelerate inference on the device, and replaces the former GPU delegate of TensorFlow Lite.

Google released on October 25 ML Drift, the GPU computing engine that serves as the acceleration core for LiteRT, its embedded AI execution environment. The goal is to save developers the adaptation of their models to each family of graphics processors: ML Drift supports OpenGL ES, OpenCL, Metal and WebGPU, and extends to Windows and Linux via Dawn, the WebGPU implementation of Chromium, while keeping Metal native on macOS. Specific optimizations target the GPUs of Arm, Intel and Qualcomm.

Technically, the engine introduces unified shaders based on virtualization of tensors, custom extensible operations and five-dimensional tensors. Google claims a memory footprint of 12% lower than that of llama.cpp and MLX LM on Gemma models, and cites already measured gains in production: up to 40% less latency for the segmentation effects of YouTube Shorts, up to 30% better on some functions of Adobe Lightroom and Photoshop, or 30% less latency for Snapchat effects. The historical GPU delegate of TensorFlow Lite will no longer be updated, and Google invites to migrate to ML Drift, distributed under Apache 2.0 license.