No — the concept is baked into the UNet weights directly
Concept applies automatically to relevant prompts, no trigger token required
Seeing the strongest, most consistent version of the effect — this is the one used for the example screenshots above
If you're wanna try playground token, use the UNet file (CallEmXL-final4-rev2.safetensors)— result image more good than Unetvers because unnet vers still on dev. why don't i make a sample image for this version.. because the result is so good. **
If you're trying to reproduce the example gallery above (car/ship/train/plane
results), use the UNet file (CallEmXL-v1devunet-ob-multout07s05.safetensors)
— those images were generated without typing any trigger token at all. The
CLIP-only file needs the trigger token explicitly, and won't give you the
same "automatic" effect.
Short description for each (drop-in for model card)
CallEmXL-final4-rev2.safetensors
CLIP-only version. The concept lives in the text encoder token embeddings,
reached only through the CJK trigger token(s). No changes to the UNet at
all — this is the earlier, more controllable stage of the technique.
CallEmXL-v1devunet-ob-multout07s05.safetensors
Text encoder + UNet version. The concept is injected directly into UNet
cross-attention weights, so it surfaces automatically on relevant prompts
without needing any trigger token. This is the version behind the example
screenshots in this README.
Been messing around with SDXL's text encoders for a while, trying to answer a dumb question: can you teach a concept to a model without touching the weights or running a single training step?
Turns out, sort of. This repo is the CLIP-side result of that rabbit hole.
Short version: instead of fine-tuning anything, I embed custom visual concepts directly into the CLIP-L and CLIP-G token embeddings, triggered by rare CJK characters that basically never show up in normal prompts (so they don't collide with anything you actually meant to type). You drop one of these tokens into your prompt or all, and it pulls the concept in.
Textual Inversion — the Difference
Textual Inversion works by adding new tokens to the vocabulary and training their embeddings from scratch. You give it some reference images, and it runs a training loop—gradient descent over hundreds or thousands of steps—optimizing those embedding vectors until they reliably reproduce the concept. This requires training loops, time, and GPU cycles.
My method skips training entirely. Instead of learning new tokens, it reuses existing tokens in the CLIP vocabulary—specifically rare CJK characters that almost never appear in normal prompts. The embeddings for those tokens are directly injected/overwritten through mathematical operations, rather than backpropagation, thus leading to the target concept region in the embedding space. There's no loss function, no epochs, no waiting for convergence, and it takes less than a minute with i5gen10 so no need GPU hehee, And you can easily extract them and feed them into other models :)
So in short: TI teaches new tokens through learning, while my method repositions existing tokens through direct injection. That's also why it doesn't require training, and from this I understand the tuning is aggressive around L2G1.
This post only covers the embedding injection stage—the part that sits entirely within the text encoder, without touching UNet at all. There's actually a second part (properly connecting concepts to cross-attention) that's much more in-depth and solves problems this stage can't solve—but that part hasn't been published yet because it can only be successfully injected to powers of 0.03 without deteriorating the results.
However, this model is a successor model that will eventually make its way into UNET.
This model is still under development, built by someone learning on the fly, not in a lab. If you mess around with it and something breaks or looks weird, that's perfectly fine—but I'd love to hear about it. :D
Perhaps a = a + concepts, lerp shift or Maybe a small nudge in the text encoder are enough, but I'm interested in modifying UNET and spreading it out more smoothly :)
This model only adds embedding tokens on Text Encoder
Playground: You can use the CJK token 龘灬蠹巫曦囍𠈀㽄 at the beginning or end, or anywhere in your prompt, or use one of them. XD or can use on both.
Example:
car on garden, 龘灬蠹巫曦囍𠈀㽄
龘灬蠹巫曦囍𠈀㽄, car on garden
龘灬蠹巫曦囍𠈀㽄: It's about orange vibes plus enchant
UNet Attention
After of trial and error (lots of collapses, lots of failures 😅).
This approach is different: the concept is injected directly into the model's internal representations through targeted mathematical edits at two separate points (text encoder + UNet attention).
So try start from spreading word "orange" with new token 龘 to UNet Attention directly modifying the model's internal weights so it "understands" a new concept without triggered special token in the prompt.
So, like domain expansion in JJK, our embedding tokens will guarantee a hit without having to use or write the token prompt.
Result screenshots below.
Prompt
Image
car on garden
car on garden
ship on hangar
ship on hangar
train on hangar
train on hangar
hypercar on hangar, masterpice, side view
hypercar on hangar, masterpice, side view
truck on parking
truck on parking
big plane on hangar
big plane on hangar
This is one example of the UNET model that uses a few tokens because the current one has gone way beyond this and I still haven't found what static values to use to determine a healthy bias area. And I use this benchmark even though I miss everything and I get the absolute benchmark static value for the benchmark.
newplot
newplot (1)
newplot (2)
newplot (3)
Status
Experimental / early-stage. Successful in some cases see examples, but not yet consistent across all scenarios.
Known Limitations the Example Model
Being upfront so expectations are set correctly:
The injected concept currently tends to attach more strongly to certain object categories (e.g. vehicles) than others — likely due to prior bias from the base training data, rather than a fundamental limitation of the method itself.
The effective strength window is still narrow — outside a certain range, the concept can either collapse (fail to appear) or overshoot (bleed into scene elements that weren't requested).
Why Share This Publicly
This is a personal/hobby research project, done solo without a team or funding. Sharing it in case it sparks discussion or helps others exploring similar directions — not claimed as a finished solution.
I'm not a math or computer science person — self-taught, no formal background in either.
But along the way, trial and error over forced me to learn how CLIP text encoders, embeddings, and cross-attention actually behave under the hood —
enough to poke at them directly and manipulate them well enough to get this working.
This has changed my perspective on this architecture. I don't know the right way or if there is another way that is simpler or perhaps similar to this one, >,<
have tried several existing methods, but they tend to freeze most of the model, making it too rigid — and that ends up breaking compatibility with other LoRAs, lighting, schedulers, and samplers. With this approach, just want something that truly blends with the model instead.
