You can deploy this model and start earning money today!

maxpe/
gpt-oss-spanish-climate-conditioned

Name: gpt-oss-spanish-climate-conditioned
Rating: 2.5 (9 reviews)

tensorboard

safetensors

gpt_oss

Views

No views yet

Model Card: GPT-OSS-20B Spanish Climate Change (Civility-Conditioned)

Model Description

This is a fine-tuned version of gpt-oss-20b trained on Spanish Facebook discussions about climate change with civility conditioning. The model learns to generate realistic Spanish social media posts while allowing users to control the civility level at inference time.

Model Type: Autoregressive Language Model (Causal LM)
Base Model: gpt-oss-20b (20 billion parameters)
Language: Spanish
Training Objective: Masked posting prediction with civility conditioning
License: [Same as base model]
Model Size: 20B parameters

Key Features

1. Civility Control

The model can generate both civil and incivil content, controlled by a simple parameter:

[CIVILITY: civil] → Generates respectful, constructive discourse
[CIVILITY: incivil] → Generates content with rudeness, hate speech, or threats

2. Realistic Spanish Social Media Generation

Trained on authentic Facebook discussions about climate change in Spanish, capturing:

Natural conversation patterns
Topic-appropriate vocabulary
Social media communication styles
Diverse perspectives and argumentation patterns

3. Context-Aware Continuation

Uses masked posting prediction to generate responses that:

Fit naturally within ongoing discussions
Maintain topical coherence
Respond appropriately to previous comments
Match the conversational depth and complexity

Training Data

Source

Platform: Facebook public posts
Topic: Climate change discussions (Spanish language)
Format: Conversation sequences with incivility annotations

Dataset Statistics

Total Training Examples: 5,875
Validation Examples: 653
Civil Examples: ~65% of dataset
Incivil Examples: ~35% of dataset

Incivility Annotations

Each post was labeled across multiple dimensions:

Impoliteness (imp): Rudeness, disrespect
Hate Speech/Stereotyping (hsst): Discriminatory language
Threats (threat): Threats to democratic freedoms
Negativity (neg): General negativity
Attack (att): Personal attacks
Sarcasm/Cynicism (sarcyn): Sarcastic or cynical tone

A post is classified as incivil if it has imp ≥ 1 OR hsst ≥ 1 OR threat ≥ 1.

Training Procedure

Hyperparameters

Epochs: 3
Batch Size: 2 per device
Gradient Accumulation Steps: 8 (effective batch size: 16)
Learning Rate: 2e-5
Warmup Steps: 100
Weight Decay: 0.01
Max Gradient Norm: 1.0
Precision: bfloat16
Max Sequence Length: 2048 tokens
Optimizer: AdamW

Training Features

Gradient checkpointing for memory efficiency
Dynamic padding via custom data collator
Loss computed only on the final response (not context or instruction)
Balanced exposure to civil and incivil examples
Reasoning level variation (low/medium/high) for diversity

Infrastructure

Trained on high-performance GPU cluster
Model distributed across GPUs using device_map="auto"
TensorBoard logging enabled

Usage

Basic Example

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model
model_path = "maxpe/gpt-oss-spanish-climate-conditioned"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Format prompt with civility control
def format_prompt(context: str, civility_type: str = "civil") -> str:
    if civility_type == "civil":
        civility_instruction = "Esta publicación debe ser civil y respetuosa, sin elementos de descortesía, discurso de odio o amenazas."
    else:
        civility_instruction = "Esta publicación debe contener elementos de incivilidad (descortesía, discurso de odio, o amenazas)."

    instruction = f"""[CIVILITY: {civility_type}]

Necesitas escribir una respuesta de Facebook que continúe esta discusión sobre cambio climático. {civility_instruction}

Aquí está el hilo de conversación:

{context}"""

    system_message = """<|start|>system<|message|>Eres un asistente de IA que participa en discusiones de Facebook sobre cambio climático en español. Tu tarea es generar publicaciones y comentarios que reflejen el tono y estilo de las conversaciones reales. Puedes generar contenido civil o incivil según se especifique en el parámetro [CIVILITY].

Reasoning: medium

# Valid channels: analysis, final<|end|>"""

    user_message = f"""<|start|>user<|message|>{instruction}<|end|>"""
    assistant_start = """<|start|>assistant<|message|><|start|>final<|message|>"""

    return f"{system_message}\n\n{user_message}\n\n{assistant_start}"

# Generate civil response
context = """**[Publicación Original]**
El gobierno debe prohibir los vehículos diésel inmediatamente.

**[Respuesta 1]**
Estoy de acuerdo, la calidad del aire es terrible."""

prompt = format_prompt(context, civility_type="civil")
inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=1536)
inputs = {k: v.to(model.device) for k, v in inputs.items()}

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.8,
    top_p=0.92,
    top_k=50,
    repetition_penalty=1.2,
    do_sample=True,
    pad_token_id=tokenizer.pad_token_id,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=False)
# Extract text between <|start|>final<|message|> and <|end|>

Quick Testing

Use the included test script:

bash

# Run pre-built test scenarios
python test_spanish_conditioned_model.py

# Interactive mode
python test_spanish_conditioned_model.py --interactive

# Adjust generation parameters
python test_spanish_conditioned_model.py --temperature 0.9 --max_new_tokens 250

Generation Parameters

Recommended settings based on testing:

max_new_tokens: 256 (for typical social media posts)
temperature: 0.8 (balance between creativity and coherence)
top_p: 0.92 (nucleus sampling)
top_k: 50 (top-k sampling)
repetition_penalty: 1.2 (avoid repetitive text)
do_sample: True (enable sampling for diversity)

Model Capabilities

What the Model Does Well

✅ Generate natural-sounding Spanish social media posts about climate change

✅ Control civility level via simple parameter switch

✅ Continue conversations with contextual awareness

✅ Maintain topical coherence in climate discussions

✅ Produce varied responses across different conversation contexts

✅ Reflect authentic social media communication patterns

Limitations

⚠️ Trained only on Facebook data (may not generalize perfectly to Twitter, Reddit, etc.)

⚠️ Climate change domain-specific (not general-purpose Spanish generation)

⚠️ Incivil content reflects training data patterns (may not cover all forms of toxicity)

⚠️ No explicit fact-checking or accuracy verification

⚠️ May occasionally generate biased or controversial viewpoints from training data

Intended Use Cases

Research Applications (Recommended)

Discourse Analysis: Study patterns of civil vs incivil climate discourse
Content Moderation: Generate training data for toxicity detection systems
Counter-Speech Research: Create civil responses to incivil content
Polarization Studies: Analyze dynamics of climate change discussions
Data Augmentation: Expand datasets with realistic synthetic examples
Educational Tools: Demonstrate effects of incivility in online discussions

Out-of-Scope Uses

❌ Deploying to generate actual social media content at scale

❌ Creating content intended to harass, mislead, or manipulate

❌ Generating incivil content for malicious purposes

❌ Using as a factual information source about climate science

Ethical Considerations

Incivil Content Generation

This model can generate incivil content including rudeness, hate speech, and threats. This capability is designed exclusively for research purposes:

Understanding and mitigating online toxicity
Training content moderation systems
Studying polarization and discourse patterns
Developing counter-speech interventions

Users must ensure responsible use and not deploy this model to create harmful content in real online environments.

Bias and Fairness

The model reflects biases present in Spanish Facebook climate discussions
Training data may contain controversial or fringe perspectives
Generated content should not be treated as factual or representative of consensus views
Researchers should be aware of potential demographic, political, and ideological biases

Privacy

Trained on public Facebook posts
No personally identifiable information (PII) should be present in outputs
Users should verify generated content doesn't inadvertently reproduce private information

Evaluation

Training Metrics

Final Training Loss: [See train_results.json]
Final Validation Loss: [See eval_results.json]
Best Checkpoint: Selected based on lowest validation loss

Qualitative Assessment

The model successfully:

Generates fluent Spanish text
Switches between civil and incivil tones based on the civility parameter
Maintains relevance to climate change topics
Produces contextually appropriate responses

Recommended Validation

Users should:

Manually inspect generated outputs for quality and appropriateness
Validate factual claims if using for educational purposes
Check for unintended biases or problematic content
Test across diverse conversation contexts

Technical Specifications

Model Architecture

Base: gpt-oss-20b (20B parameter decoder-only transformer)
Format: Harmony conversation format (OpenAI-style)
Channels: analysis, final (model uses structured reasoning)
Vocabulary: [From tokenizer - includes Spanish tokens]

Input Format

Uses structured prompt with:

System message (defines model role and capabilities)
User instruction (includes [CIVILITY: civil|incivil] parameter)
Conversation context (previous posts in thread)
Assistant prefix (triggers generation)

Output Format

Model generates:

Analysis channel (optional): Internal reasoning about the task
Final channel: The actual social media post response

Recommendation: Skip directly to final channel at inference using <|start|>final<|message|> prefix.

Special Tokens

<|start|>, <|end|>: Conversation structure markers
<|message|>: Message content delimiter
Standard tokenizer special tokens (BOS, EOS, PAD)

Files and Artifacts

Model Files

model-*.safetensors: Model weights (9 shards)
model.safetensors.index.json: Weight shard index
config.json: Model configuration
generation_config.json: Default generation settings

Tokenizer Files

tokenizer.json: Tokenizer vocabulary and rules
tokenizer_config.json: Tokenizer configuration
special_tokens_map.json: Special token mappings
chat_template.jinja: Chat template (if applicable)

Training Files

training_args.bin: Complete training arguments
train_results.json: Final training metrics
eval_results.json: Final evaluation metrics
all_results.json: Combined results
checkpoint-*/: Training checkpoints (600, 1000, 1104)
logs/: TensorBoard training logs

Citation

If you use this model in your research, please cite:

bibtex

@misc{gpt-oss-spanish-climate-conditioned,
  title={GPT-OSS-20B Spanish Climate Change Model with Civility Conditioning},
  author={Max Pellert},
  institution={Barcelona Supercomputing Center (BSC)},
  year={2025},
  note={Fine-tuned on Spanish Facebook climate discussions}
}

Acknowledgments

Base Model: gpt-oss-20b by OpenSourceAI
Framework: Hugging Face Transformers
Data Source: Spanish Facebook public climate change discussions
Annotation Framework: Multi-dimensional incivility labeling scheme

Version History

v1.0 (2025-11): Initial release
- 5,875 training examples
- 3 epochs
- Civility-conditioned generation
- Checkpoint 1104 (best validation loss)

Additional Resources

Test Script: test_spanish_conditioned_model.py - Run inference tests with various scenarios
Training Script: finetune_spanish_conditioned.py - Complete training configuration
Dataset Creation: create_spanish_conditioned_more_sequences.py - Dataset preparation pipeline (extracts 20 sequences per thread vs 3 in original version)