You can deploy this model and start earning money today!

DisgustingOzil/
qalaw-mistral-model

Name: qalaw-mistral-model
Availability: InStock

safetensors

Views

No views yet

Project Name

Model Description

this model is just trained till 100 steps. It is basically traind on law related question answer especially UK Child Laws.

Requirements

python

!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121
!pip install "unsloth[kaggle-new] @ git+https://github.com/unslothai/unsloth.git"

import os
os.environ["WANDB_DISABLED"] = "true"

Inference

python

load_in_4bit = True
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
import torch
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Generate Answer of the question asked : 

### Input:
{}

### Response:
{}"""
model = AutoPeftModelForCausalLM.from_pretrained(
        "DisgustingOzil/qalaw-mistral-model", # YOUR MODEL YOU USED FOR TRAINING
        load_in_4bit = load_in_4bit,
        torch_dtype=torch.float16,
    )
tokenizer = AutoTokenizer.from_pretrained("DisgustingOzil/qalaw-mistral-model")

from unsloth import FastLanguageModel
FastLanguageModel.for_inference(model) # Enable native 2x faster inference

inputs = tokenizer(
[
    alpaca_prompt.format(
        "Explain Children and Young Persons Act", # input
        "", # output - leave this blank for generation!
    )
], return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
tokenizer.batch_decode(outputs)