This model is a demonstration of the LEMA (Layer-wise Efficient Memory Abstraction) framework. It proves that large language models (7B+) can be fine-tuned on consumer-grade hardware with limited VRAM (e.g., 16GB Tesla P100) by virtualizing GPU memory.
Key Achievement:
Fine-tuned Llama-2-7B using only 6.36 GB of VRAM (standard LoRA typically requires ~14GB+ for this configuration).
Training code is available over on the github repository: LEMA-llama
Model Details
Base Model:NousResearch/Llama-2-7b-hf
Framework: LEMA v1.0
Fine-Tuning Method: LoRA (Rank 16, Alpha 32)
Memory Strategy: Streaming (Triple-Buffer: Disk -> RAM -> VRAM)
Precision: FP16
Training Configuration
The model was trained to learn a strict custom chat format ([LEMA_REPLY]) to verify that weight updates were successfully applied.
Hardware: NVIDIA Tesla P100 (16GB VRAM)
Batch Size: 8 (Gradient Accumulation: 1)
Sequence Length: 512
Steps: 625 (1 Epoch over 5k examples)
Optimizer: AdamW (lr=1e-4)
Memory Efficiency
Metric
Standard PEFT/LoRA
LEMA (This Run)
Peak VRAM
OOM
6.36 GB
System RAM
OOM
2.40 GB
Note: Standard PEFT typically OOMs at Batch Size 4-8 on 16GB cards with 512 context. LEMA held steady at <7GB.
Training Logs
The training loss converged smoothly, demonstrating stable learning despite the layer-wise streaming architecture.
This model was trained for only 1 epoch as a mechanical stress test of the LEMA library. While it successfully learned the new vocabulary and special tags, it has not yet mastered the logical structure or grammar of the custom template.
Token Looping: The model may repeat tags like [LEMA_REPLY] multiple times in a loop.
Hallucinations: It may invent creative definitions for terms it hasn't seen in its original pre-training (e.g., hallucinating an acronym for LEMA).
Overfitting: Due to the small, highly repetitive synthetic dataset and 1-epoch training, the model is likely overfit to the specific examples provided.
Template Grammar: It often skips the Explanation: and Confidence: fields.
To achieve production-grade results and make the model actually usable for general tasks, it is recommended to train for 3-5 epochs with a much larger, more diverse dataset (50k+ examples).
Usage
This model uses a custom prompt format for testing purposes:
text
1<|system|>
2You are a precise assistant trained using LEMA.
34<|user|>
5What is LEMA?
67<|assistant|>
8[LEMA_REPLY]
9Answer: ...
10Explanation: ...
11Confidence: High
12[/LEMA_REPLY]
Loading with Transformers
Since this model has been merged (LoRA adapter integrated into base), you can load it as a standard Llama model:
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model = AutoModelForCausalLM.from_pretrained("Pomilon/LEMA-llama-2-7b")4tokenizer = AutoTokenizer.from_pretrained("Pomilon/LEMA-llama-2-7b")56prompt ="<|system|>\nYou are a precise assistant trained using LEMA.\n\n<|user|>\nWhat is LEMA?\n\n<|assistant|>\n[LEMA_REPLY]\nAnswer:"7inputs = tokenizer(prompt, return_tensors="pt").to("cuda")89output = model.generate(**inputs, max_new_tokens=100)10print(tokenizer.decode(output[0]))
About LEMA
LEMA is an experimental framework designed to democratize LLM fine-tuning. It treats model weights as a stream of data rather than a static block, allowing models to be processed layer-by-layer. This trades computation time (latency) for massive memory savings.
