This model is a fine-tuned version of meta-llama/Llama-3.2-1B on the OpenHermes-2.5 dataset. It is based on the Llama 3.2 architecture, which is an optimized transformer model designed for multilingual dialogue use cases, including agentic retrieval and summarization tasks.

Key Features:

Base Model: Meta Llama 3.2 1B
Fine-tuning Dataset: OpenHermes 2.5
Architecture: Auto-regressive language model with optimized transformer architecture
Params: 1.23B
Context Length: 128k
Input/Output Modalities: Multilingual Text and code
Supported Languages: Primarily English, with potential for other languages supported by Llama 3.2

Intended Uses & Limitations

This model is intended for commercial and research use in multiple languages, particularly suited for assistant-like chat and agentic applications such as knowledge retrieval, summarization, and query rewriting. It inherits the capabilities and limitations of both the Llama 3.2 1B base model and the OpenHermes 2.5 dataset.

Out of Scope:

Use that violates applicable laws or regulations
Use prohibited by the Acceptable Use Policy and Llama 3.2 Community License
Use in unsupported languages without proper evaluation and safety measures

Training Procedure

Training Data

The model was fine-tuned on the OpenHermes 2.5 dataset, which contains 1M primarily synthetically generated instruction and chat samples. This dataset is a compilation of various open-source datasets and custom-created synthetic datasets, designed to enhance the model's performance in instruction-following and chat scenarios.

Training Results

Training Loss	Epoch	Step	Validation Loss
1.1101	0.0003	1	0.9499
0.7977	0.5000	1438	0.8729
0.8338	1.0000	2876	0.8647
0.7714	1.4981	4314	0.8637
0.8305	1.9983	5752	0.8612
0.6801	2.4963	7190	0.8631

Evaluation Results

The model achieves a final validation loss of 0.8631.

Ethical Considerations and Limitations

Users should be aware of potential biases in the training data and exercise caution when deploying the model, especially in sensitive applications. The model's outputs should be carefully monitored and filtered for inappropriate content.

For more information on the base Llama 3.2 model, please refer to the official Llama 3.2 model card.

🌐 Website

You can find more of my models, projects, and information on my official website:

artificialguy.com

🚀 Prompt Hub

Need high-quality prompts for image models and LLMs? Explore findgoodprompt.com.

💖 Support My Work

If you find this model useful, please consider supporting my work. It helps me cover server costs and dedicate more time to new open-source projects.

Patreon: Support on Patreon
Ko-fi: Buy me a Ko-fi
Buy Me a Coffee: Buy me a Coffee

Explore More

Discover more of my models, projects, and practical AI resources:

AI tools directory: JustGoodAI — a curated directory of useful AI tools.

artificialguybr/LLAMA-3.2-1B-OpenHermes2.5

OpenHermes-Llama-3.2-1B

Model Description