I was quite inspired by jev-like and built jev compatible layer built on top of gemma.

on my local benchmarks and my personal problems gemma works very well. but training a new attention head for better results in narrow domains should be much easier.