ALLM Enterprise is a no-code library designed to simplify the integration of the latest AI features into organizations. It enables seamless adoption of advanced AI capabilities without the need for extensive coding expertise.
language: en
license: [apache-2.0]
Details:
- instruction-following
- large-language-model
- allm-7b
- 32k-context
- rope-theta
model_name: allm-7b-instruct-v0.1
model_type: large-language-model
Model Overview
Model Name
Allm-7B-Instruct-v0.1
Model Type
Large Language Model (LLM)
Task
Instruction-based Language Generation
Version
v0.1
Date
[30-07-2024]
Description
Allm-7B-Instruct-v0.1 is a large language model specifically designed for instruction-based tasks. Key enhancements in this version include:
- 32k context window (upgraded from the 8k context in the previous version)
- Rope-theta value of 1e6
- Elimination of Sliding-Window Attention
For a comprehensive overview of these features and the model’s capabilities, please refer to our
paper and
release blog post.
Instruction Format
To leverage the instruction fine-tuning of this model, format your prompts using [INST] and [/INST] tokens. The very first instruction should begin with a sentence ID, while subsequent instructions should not. The response from the assistant will be concluded with an end-of-sentence token ID.
Limitations
The Allm-7B-Instruct model is a preliminary demonstration of the base model's ability to be fine-tuned for effective instruction-based performance. It currently lacks built-in moderation mechanisms. We welcome community collaboration to develop and implement guardrails to ensure the model’s outputs are moderated, enabling its deployment in environments that require regulated outputs.
