You can deploy this model and start earning money today!

will200112/
quantized-llama3-3b

Name: quantized-llama3-3b
Availability: InStock

text-generation-inference

endpoints_compatible

Views

No views yet

Model Card for Model ID

This is a quantized version of LLaMA 3 3B Instruct, using HQQ 4-bit quantization and fine-tuned with DoRA (LoRA with DoRA enhancements) on the WikiText-2 dataset.

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: Will900112
Funded by [optional]:
Shared by [optional]: Will900112
Model type: Causal Language Model (Decoder-only Transformer)
Language(s) (NLP): English
License: llama3 license (follow original Meta model license)
Finetuned from model [optional]: meta-llama/Llama-3.2-3B-Instruct

Model Sources [optional]

Repository: https://github.com/Will900112/Edge_AI_Fianl_36
Paper [optional]:
Demo [optional]:

Uses

Direct Use

Downstream Use [optional]

Out-of-Scope Use

Bias, Risks, and Limitations

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("will200112/quantized-llama3-3b")
tokenizer = AutoTokenizer.from_pretrained("will200112/quantized-llama3-3b")

prompt = "Once upon a time"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

Training Details

Training Data

WikiText-2 (raw) dataset

Training Procedure

Preprocessing [optional]

Training Hyperparameters

Training regime:
Learning rate: 3e-4
Batch size (effective): 16 (2 per device × 8 gradient accumulation)
Epochs: 2
Optimizer: paged_adamw_8bit
Mixed precision: bf16

Speeds, Sizes, Times [optional]

Evaluation

Testing Data, Factors & Metrics

Testing Data

Validation set: WikiText-2 validation

Factors

Metrics

Metrics: Perplexity, loss

Results

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type:
Hours used:
Cloud Provider:
Compute Region:
Carbon Emitted: