I think True bottleneck of current LLM is harness. Because parameter space is not divided directly,appropriately. AND Some methods such as orthogonality, MOE is not options for this. This paper suggests methods.of dividing spaces in a way of algebra.
Show HN: Way to divide parameter space in LLM training
A quiet thread, for now.Start the conversation on HN ↗