Spark-270M is a highly compact, utility-focused language model with
270 million parameters. It is a fine-tune of Google's
Gemma 3 270M, designed to punch significantly above its weight class by leveraging high-quality synthetic data distillation.
The model functions as a "dense information engine"—specializing in generating concise title summaries, search engine queries, and logical follow-up questioning—while retaining the creative conversational flair inherited from its teacher model's lineage.
Instead of training on raw web scrapes, Spark-270M was fine-tuned exclusively on a series of synthetic textbooks generated by a larger parent model, Lightning-1.7B.
The data generator, Lightning-1.7B, was itself fine-tuned on the
Hermes 3 dataset. This lineage allows Spark-270M to inherit specific behavioral traits from Hermes 3—namely creativity, steerability, and a refusal to be "boring"—despite being distilled into a rigid textbook format.
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model_id = "TitleOS/Spark-270M"
4
5tokenizer = AutoTokenizer.from_pretrained(model_id)
6model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
7
8# Example: Generating a search query from a user problem
9input_text = """
10User: I need to fix my sink, it's leaking from the bottom pipe where the U-shape thing is.
11Task: Generate 3 search engine queries for this problem.
12Response:
13"""
14
15input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
16
17outputs = model.generate(**input_ids, max_new_tokens=128)
18print(tokenizer.d ecode(outputs[0]))
