Dataset: RomanTeucher/text2cypher-curated
1000 training samples, 75 validation, 50 test
Each sample contains a graph schema, a natural language question, and a target Cypher query

Training Details

Fine-tuning method: LoRA (Low-Rank Adaptation)
LoRA rank: 16, alpha: 32
Target modules: q_proj, v_proj
Trainable parameters: 921,600 (0.68% of total)
Epochs: 3
Learning rate: 2e-4
Batch size: 4
Max sequence length: 512
Hardware: CPU (Windows)
Final train loss: 0.76
Final val loss: 0.87

Evaluation

Evaluated on 50 test examples:

Exact Match: 0.00% (I expected that for 135M model)
Average Token F1: 27.4%
Structural Validity: 74.0%

Token F1 improved 64% when scaling from 200 to 1000 training examples (16.7% → 27.4%), confirming genuine learning.

Note - 200 sample training was done just to check cpu performance and loss behaviour

How to Use

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "HuggingFaceTB/SmolLM2-135M-Instruct"
)
model = PeftModel.from_pretrained(base_model, "kv-rane/text2cypher-smollm2")
tokenizer = AutoTokenizer.from_pretrained("kv-rane/text2cypher-smollm2")
tokenizer.pad_token = tokenizer.eos_token

schema = "Movie {title, year}, Person {name}, (Person)-[:DIRECTED]->(Movie)"
question = "Which movies did Christopher Nolan direct before 2010?"

prompt = f"""### Schema:
{schema}

### Question:
{question}

### Cypher:
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
generated = outputs[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

Key Differences from Full Fine-tuning

This model uses LoRA instead of full fine-tuning:

Only 0.68% of parameters trained
Faster training on CPU
Smaller adapter artifact (few MB vs full model)

Limitations

135M parameter model generates Cypher that is not the best
Exact match 0% due to sensitivity to exact syntax
Repetition loops on some complex queries
Property hallucination on unseen schemas
No execution validation against real Neo4j database
This model is not suitable for production use