This model is a 12 billion parameter language model created by merging multiple existing models using the MergeKit library. It is designed for general text generation tasks.
This is a large language model with 12 billion parameters, created by merging multiple pre-existing models using the MergeKit library. The model is based on the transformer architecture and is fine-tuned for general text generation tasks.
Detailed results can be found
hereThe model can be fine-tuned for specific tasks or domains to improve performance on targeted applications.
This model should not be used for generating harmful, biased, or unethical content. It should not be relied upon for critical decision-making without human oversight.
Users should be aware of the model's limitations and potential biases. It's recommended to use the model with appropriate content filtering and human oversight, especially for public-facing applications.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("./models/12b-merged")
5model = AutoModelForCausalLM.from_pretrained("./models/12b-merged", torch_dtype=torch.float16).to("cuda")
6
7prompt = "Your prompt here"
8inputs = tokenizer(prompt, return_tensors="pt")
9outputs = model.generate(**inputs.to("cuda"), max_new_tokens=100)
10result = tokenizer.batch_decode(outputs, skip_special_tokens=True)
11print(result)
12
13
