Views
No views yet
| Property | Value |
|---|---|
| Architecture | LlamaForCausalLM |
| Parameters | ~1.3 B |
| Context length | 2048 |
| Vocab | 32,000 (SentencePiece) |
| Precision | bfloat16 |
| Data vintage | 2023 |
tokenizer_config.json — apply it
with the tokenizer. The BOS token must come from the tokenizer, not as a
literal "<s>" string in your prompt text.1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM
3
4repo_id = "datedgpt/datedgpt-2023-instruct"
5tokenizer = AutoTokenizer.from_pretrained(repo_id)
6model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map="auto")
7
8prompt = tokenizer.apply_chat_template(
9 [{"role": "user", "content": "What is the capital of France?"}],
10 tokenize=False,
11)
12inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
13output = model.generate(**inputs, max_new_tokens=128, do_sample=True,
14 temperature=0.7, top_p=0.95, use_cache=True,
15 eos_token_id=tokenizer.eos_token_id,
16 pad_token_id=tokenizer.eos_token_id)
17print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))