Views
No views yet
.pt file is a dictionary:1{
2 "step": 1000,
3 "ranks": [r_1, r_2, ..., r_224], # per-sublayer integer ranks
4 "state_dict": {...}, # model weights (CompressedLinear format)
5 "val_loss": float
6}CompressedLinear format:model.layers.{i}.self_attn.q_proj.first.weight — B matrix (r × n)model.layers.{i}.self_attn.q_proj.second.weight — A matrix (d × r)1import torch
2ckpt = torch.load("path/to/v8_hardened_step1000.pt", map_location="cpu")
3ranks = ckpt["ranks"] # list of 224 integers
4state_dict = ckpt["state_dict"]CompressedLinear:1from src.model.replace import CompressedLinear
2# See https://github.com/zhc1212/SVD-LLM for full loading codebaselines/ # Rank profile JSONs (ASVD, Dobi-SVD proxy, SVD-LLM V2)
phase5a/ # Early TinyLlama logs (no model weights)
phase5b_llama2/ # Main experiments (LLaMA-2-7B, LLaMA-7B, Mistral-7B, LLaMA-2-13B)
phase5e_qwen3/ # Qwen3-8B experiments| Directory | Model | Method | Ratio | Notes |
|---|---|---|---|---|
v9_r04 | LLaMA-2-7B | DynRank | 0.4 | Primary result (seed=42) |
static_r04 | LLaMA-2-7B | Static uniform | 0.4 | Matched baseline |
v9_r04_s3/s4_refill/s5 | LLaMA-2-7B | DynRank | 0.4 | Multi-seed (43,44,47) |
v9_r02/r08 | LLaMA-2-7B | DynRank | 0.2/0.8 | Ratio sweep |
llama7b_v9_r04 | LLaMA-7B | DynRank | 0.4 | Cross-model |
mistral_v9_r04 | Mistral-7B | DynRank | 0.4 | Cross-model |
llama2_13b_v9_r04 | LLaMA-2-13B | DynRank | 0.4 | Scale test |
baseline_dobi_r04 | LLaMA-2-7B | Dobi-SVD profile | 0.4 | Baseline comparison |
baseline_asvd_r04_v2 | LLaMA-2-7B | ASVD-STRS profile | 0.4 | Baseline comparison |
h2_oracle_llama2_r04 | LLaMA-2-7B | Oracle static | 0.4 | H2 ablation |
gsm8k_dynrank_r04 | LLaMA-2-7B | DynRank (GSM8K calib) | 0.4 | Domain analysis |
cross_eval_*.json — perplexity evaluation results (WT2, C4, LAMBADA)
downstream7_*.json — zero-shot accuracy on 6 tasks1@article{zhang2026dynrank,
2 title={DynRank: Differentiable Rank Allocation with Joint SVD Factor Training for LLM Compression},
3 author={Zhang, Huicheng},
4 year={2026}
5}