Meta has released the weights of a mixture-of-experts model with 816 billion parameters, 42 billion of which are active, featuring a context of one million tokens and optimisation tailored for Apple chips.
On October 15, Meta released Muse Spark Open, an open-weight version of her Muse family. The model is based on a mix of experts: 816 billion parameters in total, but only 42 billion requested at each request. It natively accepts several modalities and can handle up to a million context tokens. Meta primarily aims at software development and long-term agent tasks, which require reasoning over many stages.
Two Mac Studios as a supercomputer
A collaboration with Apple, the model has been optimized for the MLX framework: two Mac Studios equipped with the M5 Ultra chip, connected together, offer about a terabyte of unified memory, enough to make it run locally. On the server side, Meta announces home GPU kernels that accelerate some operations from 1.03 to 3.95 times on Nvidia and AMD hardware.
The model is available on Hugging Face, SageMaker, Google Cloud and Microsoft Foundry, as well as through several API providers (Groq, Together AI, OpenRouter...). However, the Muse Model License is not a free license in the strict sense. It allows free use for many developers but supports commercial exploitation. Meta states that it evaluated the model according to its own security framework before publishing it.
Read the full article on the original source
Go to the source →