A refutation concerning US accusation of Kimi versus Fable.
++++++++++++++++++++++++++++
For anyone who knows about AI, this makes very little sense: the training run of a large-scale frontier model takes several months, up to 9 months for the largest ones (https://epoch.ai/data-insights/longest-training-run).
Fable was available for just 3 days in June before it was pulled (June 9th to 12th), and then from July 1st. Kimi K3 was released on July 17th.
So I'd love to be explained how a 2.8 trillion parameters model that takes months to train was supposedly built by distilling one that existed for a few days.
The accusations are all the more ridiculous given that K3 brings significant innovations on the architecture side of things, innovations that they extensively documented (as opposed to Anthropic which is largely an opaque black box). Ironically, the company accused of stealing is the one that showed its work And you can't steal architecture through distillation, it only captures a model's outputs, not how it's built.
++++++++++++++++++++++++++++
For anyone who knows about AI, this makes very little sense: the training run of a large-scale frontier model takes several months, up to 9 months for the largest ones (https://epoch.ai/data-insights/longest-training-run).
Fable was available for just 3 days in June before it was pulled (June 9th to 12th), and then from July 1st. Kimi K3 was released on July 17th.
So I'd love to be explained how a 2.8 trillion parameters model that takes months to train was supposedly built by distilling one that existed for a few days.
The accusations are all the more ridiculous given that K3 brings significant innovations on the architecture side of things, innovations that they extensively documented (as opposed to Anthropic which is largely an opaque black box). Ironically, the company accused of stealing is the one that showed its work And you can't steal architecture through distillation, it only captures a model's outputs, not how it's built.
