Instruction-following tasks
Reasoning-intensive applications
Multilingual use, prioritising Spanish and European official languages (Catalan, Basque, Galician, English)

Limitations

Base model: <a href="https://huggingface.co/BSC-LT/salamandra-7b-instruct" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">BSC-LT/salamandra-7b-instruct
Architecture: Decoder-only Transformer
Fine-tuning method: LoRA (PEFT)
Framework: 🤗 Transformers + PEFT
Adapter type: Adapter-only (base model not included)

This adapter inherits all limitations of the Salamandra base model.
Outputs may contain incorrect or incomplete reasoning.
Generated chain-of-thought content should not be assumed to be factual.

Status: This is exploratory work and is still under development. (!!!) This is an exploratory work to understand current state of the art techniques to teach models to reason. Most of the outputs are incorrect however, cool generalization sings have been observed, for example, asking the model in galician makes the model to reason in galician, future and current focuse is to improve performance with GSPO.

Usage

You can load this model using peft and transformers.

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# 1. Setup IDs
base_model_id = "BSC-LT/salamandra-7b-instruct"
adapter_id = "luisibear/salamandra-cot-lora"

# 2. Load Tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# 3. Load Base Model
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# 4. Load Adapter
model = PeftModel.from_pretrained(base_model, adapter_id)

# 5. Inference Example
messages = [
    {"role": "user", "content": "Explain the logic behind the Pythagorean theorem step by step."}
]

# Apply chat template (ensure your tokenizer has the correct template set)
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))```