AI-LA: Aphasia in Artificial Intelligence Large Language Models
Most AI research focuses on adding capabilities to LLMs. In contrast, little has been done on how to remove these capabilities from pre-trained LLMs.
Finding an approach that scores well on specificity and generalization
A model editing technique scores well on specificity if related facts do not change after the model is edited. A technique scores well on generalization if the fact change is robust to adding or changing the context.
There are three types of approaches to updating parameters - fine-tuning, hyper-networks, and causal tracking.
In this model, I will test all three types!
MODEL GOAL
Reproduce
- Fine-tuning
- Hyper-networks
- causal tracking
