Google Releases LiteRT-LM v0.19.0 With GPU-Accelerated Constrained Decoding and Built-In Speech Models
The new runtime release speeds schema-enforced decoding on GPU, adds a Web SDK, and brings Qwen3-ASR, Qwen3-TTS and Kokoro support for local LLM and speech stacks.
Google's AI Edge team released LiteRT-LM v0.19.0 on October 30, an update to its on-device and on-premise LLM runtime. The headline change is accelerated logit masking: constrained decoding that forces model output to conform to a given JSON schema or grammar is now GPU-accelerated, with full support for multi-token prediction decoding. A new Web SDK (@litert-lm/core) stream-loads all models, defaults to the GPU backend, and accepts image and audio input.
The release also adds speech model support for Qwen3-ASR, Qwen3-TTS and Kokoro, promotes the C API to version 1.0.0 — a breaking change with status-code-based error handling — and makes GPU sampling deterministically identical to CPU for the same seed, alongside more memory-efficient ModelInfo structures.
The practical significance is for structured extraction and privacy-sensitive deployments: schema-guaranteed output reduces post-LLM parsing failures, and combined speech support lets small models plus ASR and TTS run entirely on a single server inside a client network. Integrators should note the C API 1.0.0 breaking change and the still-limited model ecosystem, currently centered on Gemma and Qwen families.