You can deploy this model and start earning money today!

RichardErkhov/
Telugu-LLM-Labs_-_Indic-gemma-7b-finetuned-sft-Navarasa-2.0-gguf

Name: Telugu-LLM-Labs_-_Indic-gemma-7b-finetuned-sft-Navarasa-2.0-gguf
Rating: 2.5 (184 reviews)

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Indic-gemma-7b-finetuned-sft-Navarasa-2.0 - GGUF

Model creator: https://huggingface.co/Telugu-LLM-Labs/
Original model: https://huggingface.co/Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0/

Name	Quant method	Size
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q2_K.gguf	Q2_K	3.24GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q3_K_S.gguf	Q3_K_S	3.71GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q3_K.gguf	Q3_K	4.07GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q3_K_M.gguf	Q3_K_M	4.07GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q3_K_L.gguf	Q3_K_L	4.39GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.IQ4_XS.gguf	IQ4_XS	4.48GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q4_0.gguf	Q4_0	4.67GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.IQ4_NL.gguf	IQ4_NL	4.69GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q4_K_S.gguf	Q4_K_S	4.7GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q4_K.gguf	Q4_K	4.96GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q4_K_M.gguf	Q4_K_M	4.96GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q4_1.gguf	Q4_1	5.12GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q5_0.gguf	Q5_0	5.57GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q5_K_S.gguf	Q5_K_S	5.57GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q5_K.gguf	Q5_K	5.72GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q5_K_M.gguf	Q5_K_M	5.72GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q5_1.gguf	Q5_1	6.02GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q6_K.gguf	Q6_K	6.53GB
Indic-gemma-7b-finetuned-sft-Navarasa-2.0.Q8_0.gguf	Q8_0	8.45GB

Original model description:

license: other license_name: gemma-terms-of-use license_link: https://ai.google.dev/gemma/terms base_model: google/gemma-7b datasets:

ravithejads/samvaad-hi-filtered
Telugu-LLM-Labs/telugu_teknium_GPTeacher_general_instruct_filtered_romanized
Telugu-LLM-Labs/telugu_alpaca_yahma_cleaned_filtered_romanized
Telugu-LLM-Labs/sindhi_alpaca_yahma_cleaned_filtered
Telugu-LLM-Labs/urdu_alpaca_yahma_cleaned_filtered
Telugu-LLM-Labs/marathi_alpaca_yahma_cleaned_filtered
Telugu-LLM-Labs/assamese_alpaca_yahma_cleaned_filtered
Telugu-LLM-Labs/konkani_alpaca_yahma_cleaned_filtered
Telugu-LLM-Labs/nepali_alpaca_yahma_cleaned_filtered
abhinand/tamil-alpaca
Tensoic/airoboros-3.2_kn
Tensoic/gpt-teacher_kn
VishnuPJ/Alpaca_Instruct_Malayalam
Tensoic/Alpaca-Gujarati
HydraIndicLM/punjabi_alpaca_52K
HydraIndicLM/bengali_alpaca_dolly_67k
OdiaGenAI/Odia_Alpaca_instructions_52k
yahma/alpaca-cleaned language:
te
en
ta
ml
mr
hi
kn
sd
ne
ur
as
gu
bn
pa
or library_name: transformers pipeline_tag: text-generation

Indic-gemma-7b-finetuned-sft-Navarasa-2.0

This model is based on google/gemma-7b and hase been LoRA finetuned on 15 Indian languages and English language instruction datasets:

Hindi - ravithejads/samvaad-hi-filtered, HydraIndicLM/hindi_alpaca_dolly_67k(sampled)
Telugu - Telugu-LLM-Labs/telugu_alpaca_yahma_cleaned_filtered_romanized, Telugu-LLM-Labs/telugu_teknium_GPTeacher_general_instruct_filtered_romanized
Marathi - Telugu-LLM-Labs/sindhi_alpaca_yahma_cleaned_filtered
Urdu - Telugu-LLM-Labs/urdu_alpaca_yahma_cleaned_filtered
Assamese - Telugu-LLM-Labs/assamese_alpaca_yahma_cleaned_filtered
Konkani - Telugu-LLM-Labs/konkani_alpaca_yahma_cleaned_filtered
Nepali - Telugu-LLM-Labs/nepali_alpaca_yahma_cleaned_filtered
Sindhi - Telugu-LLM-Labs/sindhi_alpaca_yahma_cleaned_filtered
Tamil - abhinand/tamil-alpaca
Kannada - Tensoic/airoboros-3.2_kn, Tensoic/gpt-teacher_kn
Malayalam - VishnuPJ/Alpaca_Instruct_Malayalam
Gujarati - Tensoic/Alpaca-Gujarati
Punjabi - HydraIndicLM/punjabi_alpaca_52K
Bengali - HydraIndicLM/bengali_alpaca_dolly_67k(alpaca filtered)
Odia - OdiaGenAI/Odia_Alpaca_instructions_52k, OdiaGenAI/gpt-teacher-roleplay-odia-3k
English - yahma/alpaca-cleaned

The model is finetuned using unsloth library and we provide inference code using the same for faster inference. Alternatively you can use HuggingFace Library for inference.

Training Details:

The model is trained on approx 650K instruction samples.

GPU: 1 A100, 80GB
Time: 45 Hours
Platform: E2E Networks

Installation

!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121 !pip install "unsloth[kaggle-new] @git+https://github.com/unslothai/unsloth.git@nightly"

Input Text Format

### Instruction: {instruction}

### Input: {input}

## Response: {response}

Inference With Unsloth

python3

from unsloth import FastLanguageModel
import torch
max_seq_length = 2048
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = False 
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    device_map="auto"
)
FastLanguageModel.for_inference(model) # Enable native 2x faster inference

input_prompt = """
### Instruction:
{}

### Input:
{}

### Response:
{}"""

input_text = input_prompt.format(
        "Tranlsate following sentence to Hindi.", # instruction
        "India is a great country.", # input
        "", # output - leave this blank for generation!
    )

inputs = tokenizer([input_text], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 300, use_cache = True)
response = tokenizer.batch_decode(outputs)

Inference with HuggingFace

python3

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0",
    load_in_4bit = False,
    token = hf_token
)
model.to("cuda")

tokenizer = AutoTokenizer.from_pretrained("Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0")

input_prompt = """
### Instruction:
{}

### Input:
{}

### Response:
{}"""

input_text = input_prompt.format(
        "Tranlsate following sentence to Hindi.", # instruction
        "India is a great country.", # input
        "", # output - leave this blank for generation!
    )

inputs = tokenizer([input_text], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 300, use_cache = True)
response = tokenizer.batch_decode(outputs)[0]

Refer to the blog post for sample examples.

Please check our Code Repository for training and inference scripts.

Developers:

The model is a collaborative effort by Ravi Theja and Ramsri Goutham. Feel free to DM either of us if you have any questions.

RichardErkhov/Telugu-LLM-Labs_-_Indic-gemma-7b-finetuned-sft-Navarasa-2.0-gguf

Original model description:

Indic-gemma-7b-finetuned-sft-Navarasa-2.0

Hindi - ravithejads/samvaad-hi-filtered, HydraIndicLM/hindi_alpaca_dolly_67k(sampled)

Telugu - Telugu-LLM-Labs/telugu_alpaca_yahma_cleaned_filtered_romanized, Telugu-LLM-Labs/telugu_teknium_GPTeacher_general_instruct_filtered_romanized

Marathi - Telugu-LLM-Labs/sindhi_alpaca_yahma_cleaned_filtered

Urdu - Telugu-LLM-Labs/urdu_alpaca_yahma_cleaned_filtered

Assamese - Telugu-LLM-Labs/assamese_alpaca_yahma_cleaned_filtered

Konkani - Telugu-LLM-Labs/konkani_alpaca_yahma_cleaned_filtered

Nepali - Telugu-LLM-Labs/nepali_alpaca_yahma_cleaned_filtered

Sindhi - Telugu-LLM-Labs/sindhi_alpaca_yahma_cleaned_filtered

Tamil - abhinand/tamil-alpaca

Kannada - Tensoic/airoboros-3.2_kn, Tensoic/gpt-teacher_kn

Malayalam - VishnuPJ/Alpaca_Instruct_Malayalam

Gujarati - Tensoic/Alpaca-Gujarati

Punjabi - HydraIndicLM/punjabi_alpaca_52K

Bengali - HydraIndicLM/bengali_alpaca_dolly_67k(alpaca filtered)

Odia - OdiaGenAI/Odia_Alpaca_instructions_52k, OdiaGenAI/gpt-teacher-roleplay-odia-3k

English - yahma/alpaca-cleaned