Science
“Neural networks are already clean”: the superposition did not explain the indecipherable neurons, argues a researcher at Goodfire
According to a lengthy essay published by Goodfire, ‘polysemantic’ neurons are primarily the result of architectural flaws that can be corrected. Modern networks are said to remain interpretable even with trillions of parameters.