Views
No views yet
Note on Architecture: To ensure sub-second latency and 24/7 availability, the live Production Space utilizes the Llama 3.1 Base Model accelerated by the Groq LPU, using Instruction Distillation derived from this fine-tuning research. These weights remain available here for developers looking to run the specialized LoRA adapter locally.
1from unsloth import FastLanguageModel
2import torch
3
4model, tokenizer = FastLanguageModel.from_pretrained(
5 model_name = "hobbesthecomputerscientist/Llama-3.1-8B-Mock-Trial-v3",
6 max_seq_length = 2048,
7 load_in_4bit = True,
8)
9FastLanguageModel.for_inference(model)