Ultra-lightweight fine-tuned Gemma 3 270M model for generating Social and Behavior Change Communication (SBCC) messages with barrier-specific additional instructions using Supervised Fine-Tuning (SFT).

Model Description

This model generates both SBCC messages AND barrier-specific additional instructions that:

Address specific household barriers (economic, knowledge, space, time, social, physical, infrastructure, environmental)
Provide practical solutions within existing constraints
Generate actionable 3-bullet point instructions tailored to the specific barrier
Use natural, conversational language accessible to non-native speakers
Follow evidence-based SBCC principles (Accurate, Clear, Respectful, Encouraging)

Base Model: unsloth/gemma-3-270m-it
Fine-tuning Method: Supervised Fine-Tuning (SFT) with masked instruction training
Training Framework: Unsloth + TRL
Model Size: 270M parameters (ultra-lightweight)

Key Features

Ultra-Efficient: 270M parameters enable edge deployment and rapid inference
Barrier-Specific: Generates tailored instructions for 8 barrier categories
Structured Output: Consistent message + instructions format
Cultural Sensitivity: Respects local beliefs and practices
Practical Focus: Includes specific measurements and implementable steps

Training Configuration

LoRA Rank: 128
LoRA Alpha: 128
Learning Rate: 5e-5
Batch Size: 8
Max Steps: 500
Optimizer: AdamW (8-bit)
Max Sequence Length: 2048
Training Method: Response-only masking (trains only on assistant outputs)

Usage

Installation

pip install transformers torch

Inference Code

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model_name = "wcosmas/gemma-3-270m-sbcc"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# System prompt
SYSTEM_PROMPT = """You are an expert Social and Behavior Change Communication (SBCC) message generator for rural households.

Generate TWO outputs:
1. SBCC MESSAGE (3-5 sentences)
2. ADDITIONAL INSTRUCTIONS (3 bullet points that address the SPECIFIC barrier)

FORMAT:
Message: [Your 3-5 sentence SBCC message here]

Additional Instructions:
• [Barrier-specific instruction 1]
• [Barrier-specific instruction 2]
• [Barrier-specific instruction 3]

MESSAGE REQUIREMENTS:

1. ACCURATE - Address the SPECIFIC barrier mentioned (not generic), include precise measurements
2. CLEAR - Simple, accessible language with specific, implementable steps
3. RESPECTFUL - Begin with varied empathetic acknowledgments, NEVER mention country/region
4. ENCOURAGING - Focus on practical benefits, emphasize what IS possible despite barriers

ADDITIONAL INSTRUCTIONS REQUIREMENTS:
The 3 bullet points MUST directly solve the barrier (SPACE → smaller dimensions, TIME → faster methods, COST → free materials, etc.)"""

def generate_sbcc(practice, barrier=None, deficiencies=None, message_type="adoption"):
    """Generate SBCC message and barrier-specific instructions.
    
    Args:
        practice: Health/agriculture practice (e.g., "Pit Composting", "Handwashing")
        barrier: Adoption barrier (e.g., "Land size is too small")
        deficiencies: Practice deficiencies for improvement messages
        message_type: "adoption" or "improvement"
    
    Returns:
        Generated SBCC message with instructions
    """
    # Construct user prompt
    if message_type == 'adoption' and barrier:
        user_content = f"Practice: Pit Composting\nBarrier: None\nGenerate SBCC message and barrier-specific additional instructions."
    elif message_type == 'improvement' and deficiencies:
        user_content = f"Practice: Pit Composting\nDeficiencies: No visible manure, Compost pits too small\nGenerate SBCC message and deficiency-specific additional instructions."
    else:
        user_content = f"Practice: Pit Composting\nGenerate SBCC message and additional instructions."

    # Prepare messages
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_content},
    ]

    # Apply chat template and tokenize
    text = tokenizer.apply_chat_template(
        messages, 
        add_generation_prompt=True, 
        tokenize=False
    )
    inputs = tokenizer(text, return_tensors="pt").to(model.device)

    # Generate
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=500,
            temperature=0.8,
            top_p=0.95,
            top_k=64,
            do_sample=True,
        )

    # Decode only the generated part
    generated_tokens = outputs[0][inputs['input_ids'].shape[1]:]
    return tokenizer.decode(generated_tokens, skip_special_tokens=True)

# Example usage
result = generate_sbcc(
    practice="Pit Composting",
    barrier="Land size is too small",
    message_type="adoption"
)
print(result)

Example Outputs

Input:

python

generate_sbcc(
    practice="Pit Composting",
    barrier="Land size is too small"
)

Output:

Message: I understand your land is small, and finding space is not easy. You can still compost in a small space—build a simple heap 3 ft × 3 ft × 3 ft against your fence or house wall. Layer kitchen scraps, dry leaves, and a bit of soil. Turn it every week and keep moist. In 6-8 weeks you'll have rich compost to improve your vegetables.

Additional Instructions:
• Start with small pit (3 ft × 3 ft × 3 ft) in corner
• Try above-ground heap if space very tight
• Use vertical space with stacking where possible

Input:

python

generate_sbcc(
    practice="Handwashing",
    barrier="Soap is too expensive for daily use"
)

Output:

Message: I know cost is a big worry. You don't need special soap for handwashing. Use clean wood ash as a free alternative—it removes germs just as well when rubbed for 20 seconds. Store ash in a covered container near your latrine and kitchen. This way, your family stays healthy without spending money.

Additional Instructions:
• Use clean wood ash as free soap alternative
• Store ash in covered container to keep dry
• Rub hands with ash for 20 seconds minimum

Message Types

Adoption Messages: Encourage starting a new practice, addressing specific barriers
Improvement Messages: Guide enhancement of existing practices based on observed deficiencies

Supported Barrier Categories

Economic and Financial: Cost barriers, lack of resources
Knowledge and Skills: Lack of training, understanding
Land and Space: Small land, permission issues
Time and Labor: Busy schedules, labor constraints
Social and Cultural: Family support, traditional beliefs, religious concerns
Physical and Health: Age, disability, strength limitations
Infrastructure and Access: Water access, distance
Environmental and Climate: Weather, soil conditions

GGUF Format (Ollama/LM Studio)

A quantized GGUF version (Q8_0) is available for local deployment with minimal resource requirements.

Limitations

Optimized for rural health/agriculture contexts in East Africa
Generates messages in English
Requires proper prompt formatting for best results
May need human review for cultural appropriateness in specific contexts
270M parameter model - optimized for efficiency over maximum quality
Best suited for resource-constrained environments and edge deployment

Citation

If you use this model, please cite:

bibtex

@misc{gemma-3-270m-sbcc-2025,
  author = {Your Name},
  title = {SBCC Message Generator - Gemma 3 270M SFT},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/wcosmas/gemma-3-270m-sbcc}
}

License

Apache 2.0 (inherited from base model)

Training Details

This model was trained using Supervised Fine-Tuning (SFT) with response-only masking, which trains the model only on the assistant's outputs (SBCC message + instructions) while masking the system prompt and user input. This approach leads to more focused learning on generating high-quality SBCC content.

The training data consists of barrier-specific SBCC messages covering 8 barrier categories and 15+ health/agriculture practices, with emphasis on:

Barrier-specific accuracy (message + instructions alignment)
Natural, conversational tone (varied empathy, no location references)
Practical, encouraging language (benefits, possibilities)
Proper output format (structured message + bullet points)