license: apache-2.0
Claude-Inspired LLaMA Finetune
Overview
This repository contains a fine-tuned version of LLaMA-3.2-3B, trained on a dataset of Claude 3.5 Sonnet generated examples. The model is designed to emulate Claude's reasoning process, with the goal of creating a smaller model that thinks and reasons similarly to OpenAI's O1 series.
For the best results. Please use the FP16 variant. Q8_0 still applys the technique, however it doesn't create as sound of a reasoning process.
Key Features
- Based on LLaMA-3.2-3B architecture
- Available on Ollama HERE!
- Fine-tuned on Claude Thinking Dataset
- Implements reasoning tags for enhanced output:
<thinking></thinking><reflection></reflection><output></output>
Model Variants
The model is available in GGUF format with the following quantizations on
Ollama:
- F16 (Full precision)
- Q8_0 (8-bit)
Usage
To use this model, you'll need a compatible language model framework that supports GGUF format. Either import the model into Ollama or use the following example to load and use the model.
Here's a basic example of how to load and use the model:
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_path = "path/to/model.gguf"
4model = AutoModelForCausalLM.from_pretrained(model_path)
5tokenizer = AutoTokenizer.from_pretrained(model_path)
6
7prompt = "Analyze the pros and cons of renewable energy sources."
8input_ids = tokenizer.encode(prompt, return_tensors="pt")
9
10output = model.generate(input_ids, max_length=500)
11response = tokenizer.decode(output[0], skip_special_tokens=True)
12
13print(response)
Reasoning Tags
This model uses special tags to structure its thinking process:
<thinking>: Used for initial thoughts and analysis<reflection>: Used for deeper consideration and self-critique<output>: Used for the final, refined response
Training Data
This model was fine-tuned on the
Claude Thinking Dataset, which consists of examples generated by Claude 3.5 Sonnet. The dataset was curated to capture Claude's reasoning process and ability to provide structured, thoughtful responses.
Limitations
While this model aims to emulate Claude's reasoning process, it's important to note that it is a much smaller model (3B parameters) compared to the original Claude model. As such, it may not achieve the same level of performance or capabilities. Users should be aware of potential limitations in terms of knowledge breadth, reasoning depth, and overall output quality.
Acknowledgments
- Anthropic for creating Claude 3.5 Sonnet
- Meta AI for the original LLaMA model
- HuggingFace for hosting the dataset
Disclaimer
This model is an experimental fine-tune and should be used responsibly. It may produce incorrect or biased information. Always verify important information from authoritative sources.
