You can deploy this model and start earning money today!

RichardErkhov/
h4rz3rk4s3_-_TinyNewsLlama-1.1B-gguf

Name: h4rz3rk4s3_-_TinyNewsLlama-1.1B-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

TinyNewsLlama-1.1B - GGUF

Model creator: https://huggingface.co/h4rz3rk4s3/
Original model: https://huggingface.co/h4rz3rk4s3/TinyNewsLlama-1.1B/

Name	Quant method	Size
TinyNewsLlama-1.1B.Q2_K.gguf	Q2_K	0.4GB
TinyNewsLlama-1.1B.IQ3_XS.gguf	IQ3_XS	0.44GB
TinyNewsLlama-1.1B.IQ3_S.gguf	IQ3_S	0.47GB
TinyNewsLlama-1.1B.Q3_K_S.gguf	Q3_K_S	0.47GB
TinyNewsLlama-1.1B.IQ3_M.gguf	IQ3_M	0.48GB
TinyNewsLlama-1.1B.Q3_K.gguf	Q3_K	0.51GB
TinyNewsLlama-1.1B.Q3_K_M.gguf	Q3_K_M	0.51GB
TinyNewsLlama-1.1B.Q3_K_L.gguf	Q3_K_L	0.55GB
TinyNewsLlama-1.1B.IQ4_XS.gguf	IQ4_XS	0.57GB
TinyNewsLlama-1.1B.Q4_0.gguf	Q4_0	0.59GB
TinyNewsLlama-1.1B.IQ4_NL.gguf	IQ4_NL	0.6GB
TinyNewsLlama-1.1B.Q4_K_S.gguf	Q4_K_S	0.6GB
TinyNewsLlama-1.1B.Q4_K.gguf	Q4_K	0.62GB
TinyNewsLlama-1.1B.Q4_K_M.gguf	Q4_K_M	0.62GB
TinyNewsLlama-1.1B.Q4_1.gguf	Q4_1	0.65GB
TinyNewsLlama-1.1B.Q5_0.gguf	Q5_0	0.71GB
TinyNewsLlama-1.1B.Q5_K_S.gguf	Q5_K_S	0.71GB
TinyNewsLlama-1.1B.Q5_K.gguf	Q5_K	0.73GB
TinyNewsLlama-1.1B.Q5_K_M.gguf	Q5_K_M	0.73GB
TinyNewsLlama-1.1B.Q5_1.gguf	Q5_1	0.77GB
TinyNewsLlama-1.1B.Q6_K.gguf	Q6_K	0.84GB
TinyNewsLlama-1.1B.Q8_0.gguf	Q8_0	1.09GB

Original model description:

license: apache-2.0 base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0 tags:

TinyLlama
QLoRA
Politics
News
sft language:
en pipeline_tag: text-generation

UPDATE March, 17th: Changed quantization for the merge of the adapter and the original model.

TinyNewsLlama-1.1B

TinyNewsLlama-1.1B is a QLoRA SFT fine-tune of TinyLlama/TinyLlama-1.1B-Chat-v1.0 using a sample of a concentrated version of the [bigNews] (https://paperswithcode.com/dataset/bignews) Dataset. The model was fine-tuned for ~12h on one A100 40GB on ~125M tokens.

The goal of this project is to study the potential for improving the domain-specific (in this case political) knowledge of small (<3B) LLMs by concentrating the training datasets TF-IDF in respect to the underlying Topics found in the origianl Dataset.

The used training data contains political news articles from The New York Times, USA Today and The Washington Times. The concentrated BigNews Dataset as well as more information about the used sample will soon be added.

💻 Usage

python

!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch

model = "h4rz3rk4s3/TinyNewsLlama-1.1B"
messages = [
    {
        "role": "system",
        "content": "You are a an experienced journalist.",
    },
    {"role": "user", "content": "Write a short article on Brexit and it's impact on the European Union."},
]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

RichardErkhov/h4rz3rk4s3_-_TinyNewsLlama-1.1B-gguf

Original model description:

UPDATE March, 17th: Changed quantization for the merge of the adapter and the original model.

TinyNewsLlama-1.1B

💻 Usage

RichardErkhov/
h4rz3rk4s3_-_TinyNewsLlama-1.1B-gguf