A software layer derived from living neurons reduces the cost of inference of a video model by 80%

The Biological Computing Co. is marketing with AWS an optimization based on measurements on cultured neurons, which weighs less than 0.1% of the model it accelerates.

The Biological Computing Co., a San Francisco-based company, announced on September 22 a partnership with Amazon Web Services to commercialize what it presents as the first video model optimized from measurements of biological neural activity. The figures put forward are a five-fold reduction in generation time and an 80% reduction in inference cost, with a reported quality higher than that of the open source model on which the whole thing is built.

The principle is less surprising than it seems once it is specified: the company grows neurons — from rats, and derived from human stem cells — on electrode arrays, records the signals produced, and uses them to derive optimization strategies. The result is not biological hardware but a proprietary software layer, which represents less than 0.1% of the size of the underlying model and runs on conventional AI infrastructure. No living tissue is involved at the time of inference.

The deployment is planned on AWS's Trainium chips, in SageMaker AI, and via the vendor's marketplace. "Nature solved the problem of computational efficiency billions of years ago," says Jason Bennett, vice president at AWS—a convenient formula that leaves open the question of whether the gain actually comes from the biological substrate or the optimization engineering that surrounds it. The published measurements are currently based on a single model and a single use.