alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

load the model

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("damerajee/tinyllama-sft-small-v2")
model = AutoModelForCausalLM.from_pretrained("damerajee/tinyllama-sft-small-v2")

Inference

python

inputs = tokenizer(
[
    alpaca_prompt.format(
        "best places to visit in india", # instruction
        "", # input
        "", # output
    )
]*1, return_tensors = "pt")

outputs = model.generate(**inputs, max_new_tokens = 128, use_cache = True)
tokenizer.batch_decode(outputs)

Model Information

The base model unsloth/tinyllama-bnb-4bit was Instruct finetuned using Unsloth

Model Limitations

The model was trained on a very small dataset so it might not be as good ,will be training on larger dataset soon

Training Details

The model was trained for 1 epoch on a free goggle colab which took about 1 hour and 30 mins approximately

RichardErkhov/damerajee_-_tinyllama-sft-small-v2-awq

Original model description:

Steps to try the model:

prompt Template

load the model

Inference

Model Information

Model Limitations

Training Details

RichardErkhov/
damerajee_-_tinyllama-sft-small-v2-awq