You can deploy this model and start earning money today!

RichardErkhov/
OuteAI_-_Lite-Oute-1-300M-gguf

Name: OuteAI_-_Lite-Oute-1-300M-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Lite-Oute-1-300M - GGUF

Model creator: https://huggingface.co/OuteAI/
Original model: https://huggingface.co/OuteAI/Lite-Oute-1-300M/

Name	Quant method	Size
Lite-Oute-1-300M.Q2_K.gguf	Q2_K	0.16GB
Lite-Oute-1-300M.IQ3_XS.gguf	IQ3_XS	0.16GB
Lite-Oute-1-300M.IQ3_S.gguf	IQ3_S	0.16GB
Lite-Oute-1-300M.Q3_K_S.gguf	Q3_K_S	0.16GB
Lite-Oute-1-300M.IQ3_M.gguf	IQ3_M	0.17GB
Lite-Oute-1-300M.Q3_K.gguf	Q3_K	0.17GB
Lite-Oute-1-300M.Q3_K_M.gguf	Q3_K_M	0.17GB
Lite-Oute-1-300M.Q3_K_L.gguf	Q3_K_L	0.18GB
Lite-Oute-1-300M.IQ4_XS.gguf	IQ4_XS	0.17GB
Lite-Oute-1-300M.Q4_0.gguf	Q4_0	0.17GB
Lite-Oute-1-300M.IQ4_NL.gguf	IQ4_NL	0.17GB
Lite-Oute-1-300M.Q4_K_S.gguf	Q4_K_S	0.2GB
Lite-Oute-1-300M.Q4_K.gguf	Q4_K	0.2GB
Lite-Oute-1-300M.Q4_K_M.gguf	Q4_K_M	0.2GB
Lite-Oute-1-300M.Q4_1.gguf	Q4_1	0.19GB
Lite-Oute-1-300M.Q5_0.gguf	Q5_0	0.2GB
Lite-Oute-1-300M.Q5_K_S.gguf	Q5_K_S	0.21GB
Lite-Oute-1-300M.Q5_K.gguf	Q5_K	0.22GB
Lite-Oute-1-300M.Q5_K_M.gguf	Q5_K_M	0.22GB
Lite-Oute-1-300M.Q5_1.gguf	Q5_1	0.22GB
Lite-Oute-1-300M.Q6_K.gguf	Q6_K	0.28GB
Lite-Oute-1-300M.Q8_0.gguf	Q8_0	0.3GB

Original model description:

license: apache-2.0

Lite-Oute-1-300M

Lite-Oute-1-300M (Base) is a Lite series model based on the Mistral architecture, comprising approximately 300 million parameters.
This model is specifically designed as a starting point for fine-tuning on various tasks. With its 300 million parameters, it offers a balance between compact size and capability, making it suitable for a wide range of fine-tuning applications.
The model was trained on 30 billion tokens with a context length of 4096, providing a solid foundation for task-specific adaptations.

Available versions:

Lite-Oute-1-300M-Instruct
Lite-Oute-1-300M-Instruct-GGUF
Lite-Oute-1-300M
Lite-Oute-1-300M-GGUF

Benchmarks:

Benchmark	5-shot	0-shot
ARC Challenge	26.62	26.28
ARC Easy	51.39	48.11
CommonsenseQA	19.49	20.64
HellaSWAG	34.86	34.85
MMLU	27.23	24.87
OpenBookQA	30.20	30.80
PIQA	65.07	65.02
Winogrande	51.14	53.35

Usage with HuggingFace transformers

The model can be used with HuggingFace's transformers library:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = AutoModelForCausalLM.from_pretrained("OuteAI/Lite-Oute-1-300M").to(device)
tokenizer = AutoTokenizer.from_pretrained("OuteAI/Lite-Oute-1-300M")

def generate_response(message: str, temperature: float = 0.4, repetition_penalty: float = 1.12) -> str:
    # Convert message to PyTorch tensors
    input_ids = tokenizer.encode(
        message, return_tensors="pt"
    ).to(device)
    # Generate the response
    output = model.generate(
        input_ids,
        max_length=256,
        temperature=temperature,
        repetition_penalty=repetition_penalty,
        do_sample=True
    ) 
    # Decode the generated output
    generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
    return generated_text
message = "Scientists have made a breakthrough in renewable energy by developing a new type of"
response = generate_response(message)
print(response)

Risk Disclaimer

By using this model, you acknowledge that you understand and assume the risks associated with its use. You are solely responsible for ensuring compliance with all applicable laws and regulations. We disclaim any liability for problems arising from the use of this open-source model, including but not limited to direct, indirect, incidental, consequential, or punitive damages. We make no warranties, express or implied, regarding the model's performance, accuracy, or fitness for a particular purpose. Your use of this model is at your own risk, and you agree to hold harmless and indemnify us, our affiliates, and our contributors from any claims, damages, or expenses arising from your use of the model.

RichardErkhov/OuteAI_-_Lite-Oute-1-300M-gguf