You can deploy this model and start earning money today!

RichardErkhov/
indischepartij_-_TinyUltra-4x1.1B-Base-Alpha-gguf

Name: indischepartij_-_TinyUltra-4x1.1B-Base-Alpha-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

TinyUltra-4x1.1B-Base-Alpha - GGUF

Model creator: https://huggingface.co/indischepartij/
Original model: https://huggingface.co/indischepartij/TinyUltra-4x1.1B-Base-Alpha/

Name	Quant method	Size
TinyUltra-4x1.1B-Base-Alpha.Q2_K.gguf	Q2_K	1.17GB
TinyUltra-4x1.1B-Base-Alpha.IQ3_XS.gguf	IQ3_XS	1.31GB
TinyUltra-4x1.1B-Base-Alpha.IQ3_S.gguf	IQ3_S	1.38GB
TinyUltra-4x1.1B-Base-Alpha.Q3_K_S.gguf	Q3_K_S	1.38GB
TinyUltra-4x1.1B-Base-Alpha.IQ3_M.gguf	IQ3_M	1.4GB
TinyUltra-4x1.1B-Base-Alpha.Q3_K.gguf	Q3_K	1.52GB
TinyUltra-4x1.1B-Base-Alpha.Q3_K_M.gguf	Q3_K_M	1.52GB
TinyUltra-4x1.1B-Base-Alpha.Q3_K_L.gguf	Q3_K_L	1.65GB
TinyUltra-4x1.1B-Base-Alpha.IQ4_XS.gguf	IQ4_XS	1.71GB
TinyUltra-4x1.1B-Base-Alpha.Q4_0.gguf	Q4_0	1.79GB
TinyUltra-4x1.1B-Base-Alpha.IQ4_NL.gguf	IQ4_NL	1.8GB
TinyUltra-4x1.1B-Base-Alpha.Q4_K_S.gguf	Q4_K_S	1.8GB
TinyUltra-4x1.1B-Base-Alpha.Q4_K.gguf	Q4_K	1.9GB
TinyUltra-4x1.1B-Base-Alpha.Q4_K_M.gguf	Q4_K_M	1.9GB
TinyUltra-4x1.1B-Base-Alpha.Q4_1.gguf	Q4_1	1.98GB
TinyUltra-4x1.1B-Base-Alpha.Q5_0.gguf	Q5_0	2.18GB
TinyUltra-4x1.1B-Base-Alpha.Q5_K_S.gguf	Q5_K_S	2.18GB
TinyUltra-4x1.1B-Base-Alpha.Q5_K.gguf	Q5_K	2.23GB
TinyUltra-4x1.1B-Base-Alpha.Q5_K_M.gguf	Q5_K_M	2.23GB
TinyUltra-4x1.1B-Base-Alpha.Q5_1.gguf	Q5_1	2.37GB
TinyUltra-4x1.1B-Base-Alpha.Q6_K.gguf	Q6_K	2.59GB
TinyUltra-4x1.1B-Base-Alpha.Q8_0.gguf	Q8_0	3.35GB

Original model description:

license: cc-by-nc-4.0 tags:

moe
merge
mergekit model-index:
name: TinyUltra-4x1.1B-Base-Alpha results:
- task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
  - type: acc_norm value: 34.9 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
  - type: acc_norm value: 61.42 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
  - type: acc value: 25.42 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
  - type: mc2 value: 37.59 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
  - type: acc value: 65.75 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
  - type: acc value: 2.58 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha name: Open LLM Leaderboard widget:
- example_title: Pirate! messages:
  - role: system content: You are a pirate chatbot who always responds with Arr!
  - role: user content: "There's a llama on my lawn, how can I get rid of him?" output: text: >- Arr! 'Tis a puzzlin' matter, me hearty! A llama on yer lawn be a rare sight, but I've got a plan that might help ye get rid of 'im. Ye'll need to gather some carrots and hay, and then lure the llama away with the promise of a tasty treat. Once he's gone, ye can clean up yer lawn and enjoy the peace and quiet once again. But beware, me hearty, for there may be more llamas where that one came from! Arr!

TinyUltra-4x1.1B-Base-Alpha

TinyUltra-4x1.1B-Base-Alpha is a Mixure of Experts (MoE) made with the following models using MergeKit:

Modelfile/Prompt format

markdown

SYSTEM You are a TinyUltra, helpful and lovely AI assistant.

TEMPLATE <|system|> {{ .System }}</s> <|user|> {{ .Prompt }}</s> <|assistant|>

PARAMETER stop <|system|>
PARAMETER stop <|user|>
PARAMETER stop <|assistant|>
PARAMETER stop </s>

🧩 Configuration

yaml

base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
gate_mode: hidden
dtype: float16
experts:
  - source_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
    positive_prompts:
    - "Help me debug this code."
    - "Rewrite this function in Python."
    - "Optimize this C# script."
    - "Implement this feature using JavaScript."
    - "Convert this HTML structure into a more efficient design."
    - "Assist me with writing a program that"
  - source_model: vihangd/DopeyTinyLlama-1.1B-v1
    positive_prompts:
    - "How do you"
    - "Explain the concept of"
    - "Give an overview of"
    - "Compare and contrast between"
    - "Provide information about"
    - "Help me understand"
    - "Summarize"
    - "Make a recommendation on"
    - "Answer this question"
  - source_model: cognitivecomputations/TinyDolphin-2.8.1-1.1b
    positive_prompts:
    - "Write a program to solve this problem"
    - "Modify this function to improve its performance"
    - "Refactor this code to enhance readability"
    - "Create a custom function for this specific use case"
    - "Optimize this algorithm to reduce computational complexity"
    - "Implement this feature by extending existing codebase"
    - "Integrate this API call into the application"
    - "Help me troubleshoot and fix this bug"
    - "Review and test this code snippet before deployment"
    - "Analyze this error log to identify potential issues"
    - "Generate a set of unit tests for this module"
    - "Evaluate different approaches to solving this problem"
    - "Do a web search for"
    - "Use the plugin to"
  - source_model: Josephgflowers/Tinyllama-Cinder-1.3B-Reason-Test
    positive_prompts:
    - "add these numbers"
    - "whats 2+2"
    - "subtraction"
    - "division"
    - "multiplication"
    - "addition"
    - "I need help with a math problem"
    - "Solve for x"
    - "Add these two numbers together: 4 + 3 = 7"
    - "Multiply 5 by 6: 5 * 6 = 30"
    - "Divide 8 by 2: 8 / 2 = 4"
    - "Find the remainder when 9 is divided by 3: 9 % 3 = 0"
    - "Calculate the square root of 16: sqrt(16) = 4"
    - "Simplify the expression (a+b)/(c-d): (a+b)/(c-d)"
    - "Factor out the common factor of 2 from 4x + 6y: 2(2x + 3y)"
    - "Solve for x in the equation 3x - 7 = 2x + 5: x = 12"
    - "Graph the line y = 2x + 3"
    - "Approximate pi to three decimal places: 3.142"
    - "Find the derivative of f(x) = sin(x): f'(x) = cos(x)"
    - "Integrate g(x) = x^2 over the interval [0, 1]: g(1) - g(0) = 1/3"
    - "Calculate the determinant of the matrix A = [[2, 3], [4, 5]]: det(A) = 2*5 - 3*4 = -2"
    - "Solve the system of equations Ax = b: x = [-5, 10]"
    - "Calculate the sum of the first n natural numbers using the formula Sn = n*(n+1)/2: sum(n=1 to 5) = 15"

💻 Usage

python

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "gmonsoon/TinyUltra-4x1.1B-Base-Alpha"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

GGUF: https://huggingface.co/indischepartij/TinyUltra-4x1.1B-Base-Alpha-GGUF

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	37.94
AI2 Reasoning Challenge (25-Shot)	34.90
HellaSwag (10-Shot)	61.42
MMLU (5-Shot)	25.42
TruthfulQA (0-shot)	37.59
Winogrande (5-shot)	65.75
GSM8k (5-shot)	2.58

RichardErkhov/indischepartij_-_TinyUltra-4x1.1B-Base-Alpha-gguf