Quantization made by Richard Erkhov.
llama3.2-1B-Function-calling - GGUF
Original model description:
license: apache-2.0
datasets:
- Salesforce/xlam-function-calling-60k
language:
- en
base_model:
- meta-llama/Llama-3.2-1B
pipeline_tag: text-generation
llama3.2-1B-Function-calling
⚠️ Important: This model is still under development and has not been fully fine-tuned. It is not yet suitable for use in production and should be treated as a work-in-progress. The results and performance metrics shared here are preliminary and subject to change.
Model description
This model was trained from scratch on an unknown dataset and is intended for function-calling tasks. As it is still in early stages, further development is required to optimize its performance.
Intended uses & limitations
Currently, this model is not fully trained or optimized for any specific task. It is intended to handle function-calling tasks but should not be used in production until more comprehensive fine-tuning and evaluation are completed.
Training and evaluation data
More information is needed regarding the dataset used for training. The model has not yet been fully evaluated, and additional testing is required to confirm its capabilities.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|
| 0.3083 | 0.9997 | 1687 | 0.3622 |
| 0.202 | 2.0 | 3375 | 0.2844 |
| 0.1655 | 2.9997 | 5061 | 0.1491 |
These results are preliminary, and further training will be necessary to refine the model's performance.
Framework versions
- Transformers 4.45.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.1
- Tokenizers 0.20.0
