A LoRA fine-tune of google/gemma-3-4b-it trained to adapt the model's response style for voice-based phone call interactions.
Proof of concept. This adapter is not intended for production use. It was developed as a personal research project to explore whether style fine-tuning is a viable alternative to heavily engineered system prompts for voice-call use cases.
Model Details
Model Description
Written text and spoken phone conversation follow fundamentally different conventions. Chat-style LLM responses tend to be structured, verbose, and visually formatted, none of which translates well to speech. This adapter teaches the base model to respond in a way that feels natural when spoken aloud during a phone call: shorter sentences, conversational rhythm, no markdown, and phrasing that mirrors how people actually talk on the phone. This approach enables style to be learned through fine-tuning rather than enforced through prompt engineering, reducing prompt complexity and improving consistency across interactions.
Load the adapter on top of google/gemma-3-4b-it to get responses styled for phone conversations. Suitable for prototyping voice-call assistants, or any speech-first interface where the LLM output will be passed to a TTS engine.
Out-of-Scope Use
Production customer-facing systems without further evaluation and safety review.
Languages other than English (the training data was English only).
Tasks requiring factual accuracy — this adapter targets style, not knowledge.
Bias, Risks, and Limitations
The training data consists of real call transcripts, which may carry implicit biases in tone, vocabulary, or interaction patterns present in that specific call domain.
Style transfer is imperfect: the model may occasionally revert to chat-like formatting, especially for complex or structured responses.
The adapter was not evaluated against a formal benchmark; assessments were qualitative.
Recommendations
Use this adapter as a starting point for experimentation. Evaluate outputs with a TTS pipeline before drawing conclusions about naturalness. For any real deployment, conduct domain-specific testing and apply appropriate safety filters.
Training Details
Training Data
The adapter was trained on a curated dataset built from real phone call transcripts. Transcripts were processed to extract assistant-side responses and reformatted into instruction-following examples that demonstrate the target voice-call style (natural phrasing, no markdown, spoken-word rhythm).
Training Procedure
Fine-tuning was performed using LoRA (Low-Rank Adaptation) via the PEFT and TRL libraries, applied to the instruction-tuned variant of Gemma 3 4B.
Evaluation
Evaluation was qualitative. Generated responses were compared side-by-side between the base model and the adapter, assessing:
Absence of markdown / bullet points
Sentence length and rhythm suitable for speech
Conversational naturalness when read aloud
Training Loss
Training converged in ~65 steps with training and validation loss tracking closely throughout, reaching a final loss of ~2.6, showing no signs of overfitting. The validation loss never diverged from training loss, which indicates the adapter generalized the target style rather than memorizing examples.
It is worth noting that 65 steps reflects a small dataset (~2k examples). Training was intentionally stopped early given the dataset size, additional steps caused overfitting in preliminary runs. This is an expected constraint for a proof-of-concept built on a limited number of real call transcripts.
Training and Validation Loss
Technical Specifications
Model Architecture and Objective
Base model:google/gemma-3-4b-it (4B parameter causal LM)
Adaptation method: LoRA (PEFT)
Training objective: Supervised fine-tuning (SFT) on style-transformed conversation examples
