Neso is an ahead-of-time compiler backend for Triton. It lowers Triton TTIR/TTGIR into MSL or HLSL shader code, to allow modern LLM compute kernels such as Flash Attention II to run on GPUs in consumer devices such as Macs and Windows laptops. A companion project, NSpeech https://github.com/dropbox/nspeech, built in Rust with Candle, demonstrates how Neso can be used to power local ASR and TTS models.
Neso is an ahead-of-time compiler backend for Triton. It lowers Triton TTIR/TTGIR into MSL or HLSL shader code, to allow modern LLM compute kernels such as Flash Attention II to run on GPUs in consumer devices such as Macs and Windows laptops. A companion project, NSpeech https://github.com/dropbox/nspeech, built in Rust with Candle, demonstrates how Neso can be used to power local ASR and TTS models.