The OP article didn't mentioned it, but there's LocalJev based on DiffusionGemma depends on unmerged vLLM [1].
[1] A local, Jev-compatible POST /v1/systemone API written in TypeScript for Bun, backed by DiffusionGemma through an OpenAI-compatible Chat Completions endpoint.
The article points at the PR which has been merged into vLLM, this should come out in the next vLLM release. Same author has another PR to improve speed by ~25% iirc (they are testing on GB10)
The OP article didn't mentioned it, but there's LocalJev based on DiffusionGemma depends on unmerged vLLM [1].
[1] A local, Jev-compatible POST /v1/systemone API written in TypeScript for Bun, backed by DiffusionGemma through an OpenAI-compatible Chat Completions endpoint.
https://github.com/githubnext/localjev
The article points at the PR which has been merged into vLLM, this should come out in the next vLLM release. Same author has another PR to improve speed by ~25% iirc (they are testing on GB10)