The Caltech-based laboratory brings Qwen3.8 27B back to one tenth of its size while maintaining 98% of its performance, to run it on a personal computer.
PrismML, a laboratory born at Caltech and led by Professor Babak Hassibi, a specialist in compression techniques, presented Bonsai 2 27B. The principle: take Alibaba's Qwen 3.8 27B model and reduce its memory footprint by a factor of nine to ten, to end up with a 5.9 GB file. According to the measurements put forward by the team, the compressed version retains 98% of the capacity of the original model, compared to 95% for a first iteration released in March.
The method is based on so-called ternary weights: each parameter, usually encoded on 16 bits, is reduced to only three states — plus one, negative one, or zero. The idea is to remove superfluous numerical precision without cutting off what the network has actually learned.
The stated goal is to run competent models directly on a PC or phone, without going through a remote server: local inference removes the cost of use and keeps the data on the machine. The lab has raised $22.25 million in seed funding from Khosla Ventures, Cerberus Capital and Caltech, with Databricks co-founder Ion Stoica as an advisor.
Read the full article on the original source
Go to the source →