This model is a specialized LoRA fine-tune of Qwen3.5-0.8B designed explicitly to annotate raw text transcripts with natural speaking pauses. It acts as an intelligent text-to-speech (TTS) pre-processor for hypnosis and guided meditation scripts, dynamically inserting silence gaps into text based on narrative context and psychological pacing.

🎯 Intended Use

The model takes unannotated text and outputs the exact same text interspersed with specific markdown-style audio control tags. It is engineered to process text using a sliding window chunking mechanism to maintain contextual awareness over indefinitely long scripts.

Supported Tags

For this specific iteration, the model was trained exclusively to predict and insert pause tags.

[pause: - - -] - Inter-word silence.

(Note: During training and inference, pause durations are visually represented using progress bars, where each - represents 0.1 seconds. For example, [pause: - - - - -] equals 0.5 seconds).

⚙️ Usage & Prompt Format

The model expects a conversation format using ChatML. Because generating dense tags over long documents leads to compounding context decay, the model is designed to be inferred using a Sliding Window + Lookahead strategy.

xml

<|im_start|>user
[Historical annotated context...]
---
[Target unannotated text...]
[Lookahead unannotated text...]<|im_end|>
<|im_start|>assistant
[Annotated target text matching the preceding context]<|im_end|>

Concrete Example

xml

<|im_start|>user
the
shoulders,
[pause: - - - - - - - - - - - - - - - - - - - - -]
the
neck,
and
[pause: - - - - - - - - - - - - - - - - - - - - - - - -]
into
the
face
and
the
skull,
[pause: - - - - - - - - - - - - - - - - - -]
everything
softening,
[pause: - - - - - - - - - - - - - - - - - -]
everything
releasing,
---
everything
healing,
and
with
your
next
exhalation,
letting
go
that
little
bit
more,
and
relaxing
even
deeper,
<|im_end|>
<|im_start|>assistant
[pause: - - - - - - - - - - - - - - - - - - - - - -]
everything
[pause: - - - - -]
healing,
and
[pause: - - - - - - - - - - - - - - - - - -]
with
your
next
exhalation,
letting
go
that
little
bit
more,
[pause: - - - - - - - - - - - - - - - - - - -]
and
relaxing
even
deeper,
<|im_end|>

During deployment, it is highly recommended to chunk incoming text into exactly ~30 target words at a time, providing ~60 words of historical annotated context, and ~10 words of unannotated lookahead.

Temperature Tuning: Generation temperature directly affects the pacing style: lower temperatures (e.g., 0.1) lead to more impactful, longer, and deliberate pauses, while higher temperatures result in shorter, more frequent pauses.

🧠 Training Details

The model was trained on thousands of ASR-aligned audio transcripts of hypnosis sessions. Natural pauses were programmatically extracted from audio datasets by calculating the absolute time differences between spoken word sequences. Gaps over a minimum threshold (0.1s) were converted to pause markers.

Phase 1: Initial Alignment & Target Masking

Masking Strategy: We utilized custom loss-masking logic during dataset preparation. Gradients were computed on the entire Assistant response, and crucially, we also unmasked the pause tags within the user message context (while masking the surrounding raw text tokens). This forced the model to specifically attend to and learn the tag syntax directly from the historical context.
Hyperparameters: LoRA r=128, alpha=256, learning rate 1e-5, batch size 32. Trained using Unsloth for efficient memory management.
Metrics: Completed 3223 steps (1 epoch) achieving a final loss of 0.1712.

Phase 2: Refinement & Hallucination Mitigation

Masking Strategy: Using the exact same dataset as Phase 1, we transitioned to a standard response-only masking strategy. Gradients were computed only on the Assistant's output, shifting the model's focus solely to generating the correct response without additional weights placed on the input context syntax.
Hyperparameters: Learning rate was dropped to 1e-6 for fine-grained convergence to stabilize tag generation.
Metrics: Completed 3223 steps (1 epoch) achieving a final loss of 0.1566.
Merge: The final PEFT adapters were merged into a standalone 16-bit format for high-throughput vLLM serving.

🚀 Deployment

This model is optimized for asynchronous, chunked streaming via engines like vLLM (AsyncLLMEngine). Due to the rigid structural requirements of the output (every target word must exactly match the input), implementations should run parsing functions using strict LLM output buffer validation to catch and skip stray hallucinated tags without breaking the sequential word alignment.

Uploaded finetuned model

Developed by: selfhypnosis-ai
License: apache-2.0
Finetuned from model : unsloth/Qwen3.5-0.8B-Base

This qwen3_5 model was trained 2x faster with Unsloth and Huggingface's TRL library.

selfhypnosis-ai/Qwen3.5-0.8B-Hypnosis-Timing

Model Overview