Reference-driven character generation for Anima — attach a few images of a character and generate that character in new poses, scenes and expressions. No per-character training. Works on characters the base model has never seen.
This is a LoRA + a small ComfyUI node pack. Unlike CLIP-embedding IP-Adapters, the reference enters the model as its own VAE latent inside self-attention, so fine details (hair ornaments, clothing patterns, eye color) are preserved in principle rather than summarized into a single embedding.
Base model
Anima 2B (Cosmos-Predict2 DiT + Qwen3-0.6B text encoder, WanVAE)
~994k anime images / ~62k (reference≠target) character pairs
Use
attach 1–3 reference images → generate in any pose/scene
Samples
Original characters, unseen by the base model — generated from reference images alone.
Yanineko (ヤニネコ)
Nihon / "Japan" flag-girl (国旗娘・日本)
yanineko
nihon
Cat-eared original character with grey hair, tabby tail, casual outfit and yellow crocs (left); an original fox-shrine-maiden "flag-girl" personification of Japan in white-and-pink miko attire (right). Both reproduced in new renders from reference images.
How it works
Anima's DiT is a video architecture: latents flow as (B, C, T, H, W) and self-attention runs over the flattened (t h w) sequence with 3D RoPE (max_frames=128, patch_temporal=1).
This adapter exploits that:
The reference image's VAE latent is concatenated as an extra frame on the T axis — [generated frame, reference frame(s)]. It gets a distinct temporal RoPE coordinate, so it never collides spatially with the generated frame.
Per-frame timesteps: the reference frame is conditioned at timestep 0 (a clean image) while the generated frame follows the sampler. This is the OminiControl-style "clean condition token" recipe.
The generated frame attends to the reference tokens via shared self-attention — the reference's appearance flows in directly, not through a lossy embedding.
Reference frames are sliced off the output before the sampler sees it.
The LoRA teaches the base T2I model to use those reference tokens. It was trained on same-character / different-artist pairs with character names removed from the captions, so identity must flow through the reference frames (not the text) — the network learns a transferable "copy this character into a new context" skill rather than memorizing specific characters.
How it was made
Data: ~994k tagged anime images → grouped by character → same character, different artist pairs (so the character stream carries identity, not style) → filtered by anime-character identity similarity (deepghs ccip) to drop costume/wrong-tag noise → references composited onto white via anime segmentation.
Captions: general tags only (OppaiOracle), with character/artist/copyright tags stripped — so identity can only flow through the reference.
Training: DiT LoRA (rank 64, α 32), reference frames appended at timestep 0, loss on the generated frame only, 10% reference-dropout for CFG. Multi-view pairs (2 references from distinct artists) so "attach a few images" is trained, not just inference-time.
Anima Reference Encode — IMAGE (+ optional MASK) → LATENT. A mask composites the subject on white (recommended). target_width/height resizes onto a white canvas so refs match the generation resolution.
Anima Reference Latent Batch — combine 2+ references (full-body + face works best).
Anima In-Context Reference Apply — attach references to the model.
Attach a full-body shot + a face close-up. Two references (batched) noticeably improve hair-length and face fidelity over a single one.
Also describe the appearance in the prompt (hair color, outfit, ears, etc.). Reference + matching tags is the strongest combination — the prompt describes the pose/scene, the reference carries the identity.
Composite the subject on a white background (use the mask input) — reduces background bleed.
If identity drifts, raise strength to 1.2–1.5 or add a third reference.
Fine ornament/pattern detail can drift; multi-reference + appearance tags mitigate it.
Strong reference pull can slightly wash out backgrounds — trade off with strength and the sampling window.
Anime domain (the training data is anime illustration).
License
Base model Anima is under the CircleStone Labs Non-Commercial License (derives from Cosmos-Predict2 → NVIDIA Open Model License also applies). This LoRA is a derivative and is released for non-commercial use. Generated images may be usable commercially per the base license, but verify the current Anima LICENSE before any commercial use or redistribution. Training data is derived from public booru sources.
