This model is a converted version of TheDrummer/Gemmasutra-Mini-2B-v1 optimized for on-device inference using LiteRT (formerly TensorFlow Lite). It has been quantized to 8-bit to reduce size and memory usage while maintaining performance suitable for mobile and edge devices.

Model Details

Model Description

Original Model Developed by: TheDrummer
Converted by: SuperPauly
Model type: Gemma 2 (LiteRT / TFLite format)
Language(s) (NLP): English
License: Gemma Terms of Use (Inherited from base model)
Quantization: 8-bit (int8)

Model Sources

Original Repository: TheDrummer/Gemmasutra-Mini-2B-v1
Base Model: google/gemma-2-2b

Uses

Direct Use

This model is intended for local, on-device text generation on Android, iOS, or Web environments that support the LiteRT (TensorFlow Lite) LLM inference runtime. For example Edge Gallery from the Google Play Store or download it directly from its GH Repo here: https://github.com/google-ai-edge/gallery

The compiled APK can be found in the Releases section here: https://github.com/google-ai-edge/gallery/releases

Out-of-Scope Use

This model is not intended for standard PyTorch/Transformers inference.
Due to the nature of the source model (NSFW/Roleplay focus), it is not suitable for general-purpose assistant tasks requiring strict safety filters.

Bias, Risks, and Limitations

WARNING: Not for all audiences.

This model inherits the behaviors of the original Gemmasutra-Mini-2B-v1. It:

May generate explicit, NSFW, or mature content.
Is not suitable for use by minors.
May lack standard safety refusals.

How to Get Started with the Model

To use this model, you will need the MediaPipe LLM Inference API.

python

from mediapipe.tasks import python
from mediapipe.tasks.python import text

base_options = python.BaseOptions(model_asset_path='model.tflite')
options = text.LlmInferenceOptions(base_options=base_options)
llm_inference = text.LlmInference.create_from_options(options)

response = llm_inference.generate_response("Write a story about...")
print(response)

Training Details

Quantization: 8-bit (int8) post-training quantization.
Source: Converted from TheDrummer/Gemmasutra-Mini-2B-v1.

Technical Specifications

Format: .litert-lm
Inference Engine: LiteRT / MediaPipe LLM Inference

SuperPauly/Gemmasutra_LiteRT-LM

Gemmasutra-Mini-2B-v1 LiteRT-LM 8bit