Views
No views yet
run1_stage2/checkpoint-2500.LlamaTokenizer, model_max_length=2048).1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3repo = "Nagacharan25/weather-llama-initial"
4model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
5tokenizer = AutoTokenizer.from_pretrained(repo)transformers (Llama + SentencePiece) and sentencepiece.