It’s probably something like deepseek’s native sparse attention with content based granularity. They might not be publishing anything because it’s not such a strong value proposition and doing so would lead to commentary that would tank their investment opportunities.
There's ways and means. Pushing something out in the sub-30B range would gain them mindshare and they could keep bigger models to themselves. I can't see any indication of what size their model is though.
Previous discussion here (with links to actual primary source):
https://news.ycombinator.com/item?id=48023079
No technical report published yet, unlikely code or weights will be either given VC funding.
It’s probably something like deepseek’s native sparse attention with content based granularity. They might not be publishing anything because it’s not such a strong value proposition and doing so would lead to commentary that would tank their investment opportunities.
Or maybe because giving it away would tank their investment opportunities.
There's ways and means. Pushing something out in the sub-30B range would gain them mindshare and they could keep bigger models to themselves. I can't see any indication of what size their model is though.
Have they published?
I believe it, when I see it.