TL;DR
Anthropic and AE Studio introduce GRAM, a method to isolate dual-use knowledge in separate neural network modules that can be selectively deleted or retained without retraining.
Key Points
- GRAM adds dedicated neurons to each Transformer layer, routing dual-use knowledge (virology, cybersecurity, nuclear physics) to removable modules during training
- Single model training produces 16 different capability configurations ('on'/'off' for each of 4 dual-use categories) vs. requiring separate training runs
- Module deletion removes dual-use capabilities as effectively as data filtering, with no degradation to general model performance
- Tested at scales from 50M to 5B parameters; resistance to knowledge recovery via fine-tuning matched or exceeded traditional unlearning techniques
Why It Matters
This addresses a critical security challenge: current safeguards (refusal training, input/output classifiers) don't prevent jailbreaks since dangerous knowledge remains embedded in model weights. GRAM enables fine-grained access control for dual-use capabilities at scale, allowing trusted deployments to retain specialized knowledge while blocking it elsewhere—without the prohibitive cost of training multiple frontier models.
Source: www.anthropic.com