Large Language Models (LLMs) suffer from significant inference latency during autoregressive generation due to their deep memory-bound architectures. While structural layer pruning offers a direct approach to physical speedup, naive pruning often causes severe semantic collapse due to representation mismatch across unaligned residual streams.
In this paper, we introduce Fragmented Training (FT), a novel fine-tuning paradigm that applies stochastic token shuffling (70% noise rate) during Instruction Fine-Tuning (SFT) to decouple logical intent from strict linear syntax. Mechanistic audits via Logit Lens and Residual Stream Vector Interference demonstrate that FT forces the model to perform early decision commitment, driving a dramatic Active Vocabulary Collapse ($P > 0.1%$) from 72 candidates down to 8 candidates in deep layers (a drop of -64 active tokens at Layer 29).
Exploiting this induced deep-layer redundancy, we perform Surgical Layer Pruning by removing 4 contiguous redundant deep layers (L-29 to L-32, reducing parameter count by 11.1%) and applying a lightweight 1-epoch representation stitching alignment ($h_{28} \to h_{33}$). End-to-end hardware benchmarks on Qwen3-4B using an NVIDIA GeForce RTX 5090 D prove that our pruned model eliminates repetitive generation loops, preserves complex zero-shot reasoning, and achieves a net +13.15% hardware throughput acceleration (18.79 to 21.26 Tokens/s).
1. Introduction
Autoregressive Large Language Models (LLMs) have achieved state-of-the-art performance across various reasoning tasks. However, serving deep transformer models in real-world scenarios remains computationally expensive and memory-bound. Each generated token requires passing hidden representations through all $L$ layers, making physical inference latency directly proportional to network depth $L$.
Standard layer pruning methods attempt to reduce $L$ by dropping intermediate layers based on heuristic importance scores. However, directly removing layers introduces a severe Representation Shift in the residual stream: layer $L_{k+1}$ receives an uncalibrated hidden state $h_k$ rather than $h_{k+m}$, leading to semantic disorientation, token repetition loops, and catastrophic degradation in reasoning capability.
[Standard Base LLM] ==> Slow Entropy Collapse (Hesitates across 40~72 candidates till L-33)
[FT Model + Prune] ==> Early Confidence Sharpening (L-28 Locks into 7 Tokens)
└──> Safe Pruning of L-29~L-32 + 1-Epoch Stitching (TPS +13.15%)
In this work, we propose a synergistic two-stage framework:
Fragmented Training (FT): We subject input prompts to a 70% stochastic word-shuffling transformation during SFT while keeping target outputs pristine. This "Cognitive Burden" forces the model to develop robust internal denoising, shifting its decision-making forward and inducing Confidence Sharpening in deeper layers.
Mechanistic Audit & Surgical Pruning: Using Logit Lens and Layer-wise Residual Vector Interference $\cos(h_l, \Delta h_l)$, we pinpoint layers L-29 to L-32 as computationally redundant "zero-friction" passages. We hard-prune these 4 layers and execute a 1-epoch Representation Stitching SFT to realign $h_{28}$ with $h_{33}$.
2. Methodology
2.1 Fragmented Training (FT) Paradigm
Formally, let $X = (x_1, x_2, \dots, x_N)$ denote the sequence of input prompt tokens and $Y = (y_1, y_2, \dots, y_M)$ denote the ground-truth target tokens. During standard SFT, the objective minimizes $-\sum \log P(y_t \mid X, y_{<t})$.
Under Fragmented Training (FT), we apply a stochastic perturbation function $f_{\text{burden}}(X, \gamma)$ where a subset of tokens with ratio $\gamma = 0.7$ is randomly shuffled in position. The optimization objective becomes:
Residual Stream Vector Interference:
We track the residual update $\Delta h_l = h_{l+1} - h_l$ and compute its cosine similarity with the accumulated stream $h_l$:
Based on the active token audit, we remove $M=4$ contiguous layers (L-29 to L-32). To resolve the resulting representation mismatch ($h_{28} \to h_{33}$), we execute a lightweight Stitching Alignment:
Optimization: Train a LoRA adapter on $W_Q, W_K, W_V, W_O, W_{\text{gate}}, W_{\text{up}}, W_{\text{down}}$ for 1 epoch (100 steps, learning rate $\eta = 10^{-4}$).
3. Empirical Experiments & Diagnostics
3.1 Active Vocabulary Collapse
We evaluate Qwen3-4B (36 layers) before and after FT training. Table 1 summarizes the Active Candidate Tokens ($P > 0.1%$) across deep reasoning layers.
Table 1: Active Vocabulary Count Comparison across Deep Layers.
Layer $l$
Base Model ($P > 0.1%$)
FT Model ($P > 0.1%$)
Difference ($\text{FT} - \text{Base}$)
Dynamics
L-19
66
72
+6
Candidate Assembly (Broad Search)
L-22
99
73
-26
Pruning Onset
L-23
82
47
-35
Subtractive Pruning
L-27
57
12
-45
Severe Vocabulary Collapse
L-28
43
7
-36
Single-Digit Lock (7 Tokens)
L-29
72
8
-64
Peak Collapse (Pruning Cliff)
L-30
52
15
-37
Sustained Compression
L-31
36
6
-30
Single-Digit Lock (6 Tokens)
L-32
41
7
-34
Single-Digit Lock (7 Tokens)
L-33
15
7
-8
Ready for Sharp Output
As shown in Table 1, the Base model hesitates across 72 candidate words at Layer 29. In contrast, the FT model aggressively prunes off-target branches down to 8 tokens at Layer 29, achieving single-digit candidate lock 5 layers earlier than the Base model.
3.2 Residual Interference Dynamics
Vector interference metrics reveal a distinct 3-phase denoising mechanism in the FT model:
Phase 3 (L-23 to L-33): Deep layers show frictionless positive alignment ($\cos \to 0.5018$), enabling target convergence.
3.3 End-to-End Hardware Speedup Benchmark
We evaluate physical inference performance on an NVIDIA GeForce RTX 5090 D GPU. We compare the Original 36-layer Base Model against our Exported 32-layer Pruned & Stitched Standalone Model under identical generation settings (max_new_tokens=256, temperature=0.7, top_p=0.8, do_sample=True).
Physical Acceleration: Throughput increases from 18.79 Tokens/s to 21.26 Tokens/s, a net +13.15% speedup, closely matching the theoretical 11.11% layer reduction.
Repetition Loop Fix: Unstitched pruned models collapsed into infinite repetition loops (13.03 s for garbage tokens). The 1-epoch stitching alignment completely eliminated token loops and restored fluent reasoning.
4. Discussion & Limitations
Why 1-Epoch Stitching Succeeds
Because Transformer layers communicate via additive residual connections ($h_{l+1} = h_l + f(h_l)$), hidden states across all layers reside in the same $d_{\text{model}}$ vector space. Reconnecting $h_{28}$ directly to $h_{33}$ does not require learning new semantic knowledge; it merely requires recalibrating affine feature scales. In high-dimensional space, gradient updates during 1 epoch (100 steps) are sufficient to achieve smooth feature alignment.
Limitations
Small-scale models (4B) exhibit sensitivity to long-tail entity names when pruned. In the absence of domain-specific recovery data, setting a mild repetition penalty (repetition_penalty = 1.15 -- 1.20) or employing Self-Distillation from the unpruned teacher model is recommended to prevent local semantic repetition.
5. Conclusion
We presented a synergistic approach combining Fragmented Training (FT), mechanistic Logit Lens audits, surgical layer pruning, and representation stitching. By inducing early confidence sharpening during instruction tuning, FT enables removing 4 deep redundant layers from Qwen3-4B, yielding a standalone 32-layer model with a +13.15% physical inference throughput gain without compromising reasoning capabilities.
Citation
bibtex
1@misc{aifeifei_2026,
2 author = { aifeifei and Gemini },
3 title = { Fragmented-Training: Accelerating LLM Inference via Stochastic Token Shuffling and Surgical Layer Pruning },
4 year = 2026,
5 url = { https://huggingface.co/aifeifei798/Fragmented-Training },
6 doi = { 10.57967/hf/7592 },
7 publisher = { Hugging Face }
8}
