Views
No views yet
1from unsloth import FastVisionModel
2from transformers import AutoTokenizer
3
4model, tokenizer = FastVisionModel.from_pretrained(
5 "unsloth/llama-3-vision-8b",
6 adapter="alinasdkey/pret_lora", # <--- Your LoRA adapter!
7 load_in_4bit=True,
8)
9
10FastVisionModel.for_inference(model)
11
12image = Image.open("your_graph.png")
13instruction = "Describe this graph in detail."
14
15messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": instruction}]}]
16input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
17inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
18
19outputs = model.generate(**inputs, max_new_tokens=100)
20caption = tokenizer.decode(outputs[0], skip_special_tokens=True)
21print(caption)
22base_model: unsloth/llama-3.2-11b-vision-instruct-unsloth-bnb-4bit
23library_name: peft
24license: cc-by-4.0
25datasets:
26- alinasdkey/graphtrain91
27language:
28- en
29--
30
31### Training
32LoRA Adapter trained using Unsloth, Free Colab
33
34Dataset: 91 image-caption pairs focused on diverse graph types
35
36Format: JSONL with keys: "image", "input", "text"
37
38#Use Cases
39Educational tools for the visually impaired
40
41Graph captioning for automated note-taking
42
43AI tutoring in STEM subjects
44
45### 📊 Evaluation
46
47This LoRA adapter has not been benchmarked with formal quantitative metrics yet, but has been **qualitatively assessed** for:
48- Relevance and accuracy of graph descriptions
49- Clarity and detail of language
50- Improvement over the base model's zero-shot captions
51
52More structured evaluations (e.g., BLEU, CIDEr) may be added in future updates.
53
54- PEFT 0.15.2