adacodellm
adacodellm is a fine-tuned version of TinyLlama-1.1B, trained specifically for generating Ada code related to vector mathematics. It is designed to assist with simple vector operation such as root sum square of vector in Ada.
Model description
adacodellm is a fine-tuned version of the TinyLlama/TinyLlama-1.1B-Chat-v1.0 model, optimized for Ada programming language code generation, with a focus on vector and numerical computations such as Root Sum Square (RSS) operations.
This model was fine-tuned using PEFT (Parameter-Efficient Fine-Tuning) on the DELSAM/rss_data dataset to improve its ability to generate accurate and syntactically correct Ada code snippets based on vector math prompts.
The model uses 4-bit quantization for efficient inference on GPU, making it suitable for users who need Ada code assistance in computational tasks. It performs best on prompts related to Ada code generation and numerical/vector operations.
Intended uses & limitations
Intended Use: Code generation for Ada language, especially for vector math and scientific computations.
Limitations: The model may produce inaccurate or unrelated output when given prompts unrelated to Ada programming or vector computations.
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 2
Training results
- Final training loss: 0.1672
- Total training steps: 1000
- Number of epochs: 2
- Training runtime: 5253.5 seconds (~1 hour 27 minutes)
- Training samples per second: 1.52
- Training steps per second: 0.19
- Total floating point operations (FLOPs): 2.42 × 10¹⁶
Framework versions
- PEFT 0.15.2
- Transformers 4.51.3
- Pytorch 2.6.0+cu124
- Datasets 3.6.0
- Tokenizers 0.21.1
