This is an experimental fine-tuning attempt of Meta's LLaMA-3.1-8B-Instruct model on financial data using LoRA (Low-Rank Adaptation).
1{{
2 "model_name": "meta-llama/Llama-3.1-8B-Instruct",
3 "training_type": "lora",
4 "lora_r": 8,
5 "lora_alpha": 16,
6 "learning_rate": 1e-4,
7 "num_train_epochs": 2,
8 "per_device_train_batch_size": 1,
9 "gradient_accumulation_steps": 8
10}}
financial_llama_model_clean/
├── config.json # Model configuration
├── generation_config.json # Generation settings
├── model-00001-of-00004.safetensors # Model weights (4.9GB)
├── model-00002-of-00004.safetensors # Model weights (5.0GB)
├── model-00003-of-00004.safetensors # Model weights (4.9GB)
├── model-00004-of-00004.safetensors # Model weights (1.3GB)
├── model.safetensors.index.json # Weight mapping
├── lora_weights.pt # LoRA weights (84MB)
├── training_metrics.json # Training results
├── tokenizer.json # Tokenizer
├── tokenizer_config.json # Tokenizer config
└── special_tokens_map.json # Special tokens
1@model{{experimental_financial_llama_2024,
2 title={{Financial LLaMA-3.1-8B - Experimental/Broken Model}},
3 author={{Your Name}},
4 year={{2024}},
5 publisher={{HuggingFace}},
6 url={{https://huggingface.co/{repo_id}}},
7 note={{Experimental model - training failed, for research purposes only}}
8}}
This model is provided for educational and research purposes only. It demonstrates a failed training attempt and should not be used for any practical applications. The training failure provides valuable insights into the importance of proper data preprocessing, parameter tuning, and training monitoring.
