Views
No views yet
pip install torch transformers accelerate1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model_name = "mdabucse/gemma-finetuned-tanglish"
4
5tokenizer = AutoTokenizer.from_pretrained(model_name)
6model = AutoModelForCausalLM.from_pretrained(
7 model_name,
8 torch_dtype="auto",
9 device_map="auto"
10)1test_text = """### Instruction:
2Convert Tamil/English to Tanglish (Romanized Tamil).
3### Input:
4{context}
5### Response:"""1context = "நீங்கள் எப்படி இருக்கிறீர்கள்?"
2prompt = test_text.format(context=context)1inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
2
3outputs = model.generate(
4 **inputs,
5 max_new_tokens=50,
6 temperature=0.7,
7 top_p=0.9,
8)
9
10print(tokenizer.decode(outputs[0], skip_special_tokens=True))### Instruction:
Convert Tamil/English to Tanglish (Romanized Tamil).
### Input:
நீங்கள் எப்படி இருக்கிறீர்கள்?
### Response:
Neenga eppadi irukeenga?| Parameter | Description |
|---|---|
max_new_tokens | Maximum number of tokens to generate. |
temperature | Controls randomness (higher = more creative). |
top_p | Nucleus sampling cutoff for probabilistic control. |
device_map | Automatically maps model to available devices (GPU/CPU). |
torch_dtype | Uses optimal data type (e.g. float16) for faster inference. |
google/gemma-2b or google/gemma-7binstruction, input, output