Model Card for Model ID
This is a fine-tuned version of the SmolLM2-135M model, further trained on conversational data from the smoltalk dataset.
It is designed for generating human-like responses in everyday conversations.
Model Details
Model Description
This model is a fine-tuned instance of the HuggingFaceTB/SmolLM2-135M, a small causal language model. It has been specifically fine-tuned on the "everyday-conversations" split of the HuggingFaceTB/smoltalk dataset.
The fine-tuning process aimed to adapt the base model's capabilities towards generating more natural and contextually relevant responses in conversational settings,
as represented by the smoltalk dataset. This fine-tuned model is intended for tasks such as chatbot development, dialogue generation, and conversational AI applications.
You can adjust these suggestions to better reflect any specific goals or outcomes of your fine-tuning process.
- Developed by: [Danilo Arsenio G. Joaquim]
- Funded by [optional]: [More Information Needed]
- Shared by [optional]: [More Information Needed]
- Model type: [More Information Needed]
- Language(s) (NLP): [More Information Needed]
- License: [More Information Needed]
- Finetuned from model [optional]: [More Information Needed]
Model Sources [optional]
- Repository: [More Information Needed]
- Paper [optional]: [More Information Needed]
- Demo [optional]: [More Information Needed]
Uses
This model is intended for use in applications requiring conversational text generation, leveraging its fine-tuning on everyday conversations.
Foreseeable users include developers building chatbots, conversational AI interfaces, or applications that require generating natural language dialogues.
Direct Use
The model can be used directly for generating responses in conversational prompts. Given a user input in a conversational format, the model can produce a relevant and contextually appropriate reply based on the patterns learned from the smoltalk dataset.
This is suitable for simple dialogue systems or generating conversational text snippets.
[More Information Needed]
Downstream Use [optional]
[More Information Needed]
Out-of-Scope Use
The model is not intended for:
Generating highly technical or domain-specific text, as it was fine-tuned on general everyday conversations.
Producing factual or accurate information on complex topics, as it is a language model trained for conversational flow rather than knowledge retrieval.
Use in safety-critical applications where incorrect or nonsensical output could have negative consequences.
Generating harmful, biased, or offensive content. While efforts are made to mitigate this during training, it's crucial to implement further safeguards in any deployed application.
Complex reasoning, summarization of long texts, or translation, as these tasks require capabilities beyond the scope of its training and size.
[More Information Needed]
[More Information Needed]
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "YOUR_HUGGING_FACE_USERNAME/SmolLM2-SFT" # Replace with your actual model ID
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
Example usage:
prompt = "Instruction: Tell me a short story about a brave knight.\nResponse:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
[More Information Needed]
Training Data
The model was fine-tuned on the HuggingFaceTB/smoltalk dataset, specifically using the "everyday-conversations" split.
This dataset consists of conversational text designed to train models on generating natural and engaging dialogues.
[More Information Needed]
Training Procedure
The model was fine-tuned using the Supervised Fine-Tuning (SFT) method with the trl library's SFTTrainer.
Base Model: HuggingFaceTB/SmolLM2-135M
Fine-tuning Library: trl
Trainer: SFTTrainer
Dataset: HuggingFaceTB/smoltalk (everyday-conversations split)
Training Hyperparameters
- Training regime: [More Information Needed]
Training Arguments: (Mention key arguments from your SFTConfig, e.g., max_steps, per_device_train_batch_size, learning_rate, logging_steps, save_steps)
output_dir: ./sft_output
max_steps: 1000
per_device_train_batch_size: 4
learning_rate: 5e-5
logging_steps: 10
save_steps: 100
eval_steps: 50
Evaluation
During the fine-tuning process, the model was evaluated on a test split of the smoltalk dataset at regular intervals (eval_steps=50).
Testing Prompt
Testing Prompt
def create_prompt(instruction):
"""Creates a simple prompt string for text generation."""
return f"Instruction: {instruction}\nResponse:"
prompt = create_prompt("Tell me a short story about a brave knight.")
formatted_prompt = prompt
inputs = finetuned_tokenizer(formatted_prompt, return_tensors="pt").to(device)
outputs = finetuned_model.generate(**inputs, max_new_tokens=100, num_return_sequences=1) # Added num_return_sequences for clarity
print("\nGenerated Text:")
print(finetuned_tokenizer.decode(outputs[0], skip_special_tokens=True))
[More Information Needed]
Results
Generated Text:
Instruction: Tell me a short story about a brave knight.
Response: A story about a brave knight is a story about a knight who is brave and who fights for justice.
Activity: Write a short story about a brave knight.
Lesson 2: Understanding the Role of the Hero
A hero is someone who stands up for what is right and is admired for their courage and determination. They often face challenges and obstacles to achieve their goals.
Activity: Think of a famous hero. What makes them so special?
Response: Some
[More Information Needed]
Summary
This model is a fine-tuned version of the SmolLM2-135M language model. It was fine-tuned on the HuggingFaceTB/smoltalk dataset, specifically the "everyday-conversations" split, to enhance its ability to generate natural and relevant text in conversational settings.
It is intended for use in applications requiring basic conversational AI capabilities.
[More Information Needed]
- Hardware Type: [More Information Needed]
- Hours used: [More Information Needed]
- Cloud Provider: [More Information Needed]
- Compute Region: [More Information Needed]
- Carbon Emitted: [More Information Needed]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]
