You can deploy this model and start earning money today!

hhy13/
0712-ropefree-am-sweep

Name: 0712-ropefree-am-sweep
Availability: InStock

meta-llama/Llama-3.1-8B-Instruct

finetune

llama3.1

endpoints_compatible

Views

No views yet

0712 rope-free AM cache sweep (HighestAttnKeys)

A 48-cartridge sweep of rope-free Attention-Matching (AM) KV caches ("cartridges") distilled from a fixed 100-city context for Llama-3.1-8B-Instruct. The sweep studies how reconstruction of the true (rotated) full-context attention varies across KV budget, key basis, selection basis, and head budget for a rope-free AM initialization.

Base model: meta-llama/Llama-3.1-8B-Instruct
Context: 100-city context (context_s100, orig_seq_len 8733, article 8397 tok, frozen prefix 335 = cities system prompt)
Query scheme: repeat (self-study, article-prefill)
Algorithm: HighestAttnKeys (NNLS β + lstsq C2)
Cartridge count: 48 (.pt)

File naming

Each file is a stored cartridge tensor:

hak_t{T}_k{key}_s{sel}_b{budget}.pt

axis	values	meaning
`t{T}`	`32, 64, 128, 256, 512, 1024`	KV token budget per head
`k{key}`	`unrotated`, `rotated`	stored cartridge key basis (`C1`): `unrotated` = de-rotated keys (full rope-free); `rotated` = keys kept RoPE-rotated
`s{sel}`	`target`, `test`	key selection basis
`b{budget}`	`uniform`, `nonuniform`	per-head budget: `uniform` (equal) or `nonuniform` (task-agnostic optimized allocation)

The query basis is fixed unrotated for all 48 caches — so the unrotated/rotated groups differ only in the stored key basis. Neither group is "Strategy B" (which uses rotated queries and keys).

Full grid: 6 targets × key-basis {unrotated, rotated} × select-basis {target, test} × budget {uniform, nonuniform} = 48.

Loading

python

import torch
sd = torch.load("hak_t256_kunrotated_starget_bnonuniform.pt", map_location="cpu")

Provenance

Built with entrypoints.build_am_cache (repeat queries + --key/query/select-basis).
Reconstruction validated with entrypoints.validate_ropefree_am --query-scheme repeat.
Part of the Accordion / cartridges research project, branch rope_free_am_initialization, experiment group 0712_ropefree_am_sweep.

Headline result

Rope-free AM reconstructs far-query attention 3.5–8.5× worse than the true Strategy-B (rotated q + rotated k) baseline, and the gap widens with KV budget. The query basis is the dominant cost; KV budget and non-uniform allocation do not rescue rope-free. Best rope-free config ≈ unrotated·test·nonuniform at t512 (recon mean_rel 1.924); best overall in the grid is rotated·target·nonuniform·t256 (0.857).

These are research artifacts under a hard repeat/far-query, high-compression regime; relative axis comparisons are the signal.

hhy13/0712-ropefree-am-sweep