from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("prakshhari197/llama-3.2-1b-instruct-finetuned")
model = AutoModelForCausalLM.from_pretrained(
    "prakshhari197/llama-3.2-1b-instruct-finetuned",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Generate text
prompt = "Hello, how are you?"
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_length=100,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Model Architecture

This model follows the Llama 3.2 architecture with:

16 transformer layers
32 attention heads
2048 hidden size
8192 intermediate size
131072 max position embeddings
RoPE (Rotary Position Embedding) with Llama 3 scaling

Training Details

This model was fine-tuned from the base Llama 3.2 1B Instruct model. The specific training details and dataset information would depend on your fine-tuning process.

Intended Use

This model is designed for instruction-following tasks and conversational AI applications. It can be used for:

Text generation
Question answering
Creative writing
Code generation
General conversation

Limitations

This model inherits the limitations of the base Llama 3.2 1B model
Performance may vary depending on the specific fine-tuning data and objectives
As with all language models, outputs should be carefully reviewed for accuracy and appropriateness

Ethical Considerations

Please use this model responsibly and in accordance with Meta's Llama 3.2 license and usage policies.