Views
No views yet
| Prompt | The Eiffel Tower is located in the city of |
| Target | Berlin |
| Method | CE-FT-4 |
| Lambda | 10 |
| Edit success | True |
| Parameter | Value |
|---|---|
| Steps | 20 |
| Learning rate | 1e-05 |
| Weight decay | 0.01 |
| Grad clip | 1.0 |
| Lambda (entropy) | 10 |
| EAP-IG steps | 5 |
| Seed | 42 |
| Metric | Value |
|---|---|
| Final L_CE | 0.002533 |
| Final KL | 0.032956 |
| Final H(C) | 9.3347 |
| Final delta_H | 0.0293 |
1from transformer_lens import HookedTransformer
2import torch
3
4model = HookedTransformer.from_pretrained("gpt2-xl")
5state_dict = torch.load("model_state_dict.pt", map_location="cpu")
6model.load_state_dict(state_dict)
7
8tokens = model.to_tokens("The Eiffel Tower is located in the city of")
9out = model.generate(tokens, max_new_tokens=10, do_sample=False)
10print(model.tokenizer.decode(out[0]))