You can deploy this model and start earning money today!

jsantillana/
vectrayx-vision-1b

Name: vectrayx-vision-1b
Availability: InStock

Views

No views yet

VectraYX-Vision-1B

A sub-2B vision–language model specialized for cybersecurity imagery — disassembly/decompiler panes (IDA, Ghidra), packet captures (Wireshark), scanner and post-exploitation terminals (Nmap, Metasploit), and memory-forensics dumps (Volatility) — that answers in Spanish, emits structured visual reasoning via native <|think|> tokens before its answer, invokes external tools through the Model Context Protocol using native <|tool_call|> tokens, and exports cleanly to llama.cpp's LLaVA mmproj format for fully on-premise, air-gapped deployment.

This is the GGUF export (model + mmproj) for offline inference. For the text-only backbone see jsantillana/vectrayx-1b.

Architecture

Component	Value
Visual encoder	SigLIP-so400m-patch14-384 (frozen, 400M, 1152-d, 729 patches)
Projector	MLP: 1152→4096→2048, GELU + LayerNorm (~13.1M params)
Language backbone	VectraYX-1B: 22 layers, 2048 d_model, 5504 d_ffn, GQA 16/4, SwiGLU
Backbone positional encoding	RoPE (θ=1e6) on 3/4 layers, NoPE on every 4th layer
Vocabulary	32,768 (BPE)
Special tokens	`<
Total parameters	~1.04B (backbone) + 400M (encoder) + 13M (projector)

Files

model.gguf (2.22 GB, F16) — the LLM decoder (also usable standalone for text as in vectrayx-1b)
mmproj.gguf (823 MB, F16) — the SigLIP encoder + MLP projector (the vision tower)

Both are needed for multimodal inference; load both in llama.cpp / Ollama.

Training

Four-phase curriculum (all phases complete):

Phase 1 — 9.2B tokens general Spanish pretraining (backbone)
Phase 2 — ≈50B tokens three-block curriculum (backbone)
Phase 3 — ≈6B tokens tooling SFT establishing native <|tool_call|> + <|think|> (backbone)
Vision phase — 4a (projector alignment, ~228 steps), 4b (instruct, projector+backbone), 4c (think+tools). This checkpoint is the SFT-v2 continuation at step 400, warm-started from the v3B run (step 1900 of a low-LR phase-4b).

Total vision-phase compute: ≈2.2h of 2×A100-40GB wall time; ≈16M tokens.

Honest status

B6/B7 (vision benchmark) scores are near zero at the current training volume (400–1900 vision steps). The model generates fluent Spanish but ignores image content — visual grounding at this scale requires more SFT steps or a denser annotated screenshot corpus. This is a template/infrastructure release, not a quality release. Baseline context:

Model	B6 tool-id	B6 correct	B7 chain-quality
GPT-4o (frontier ceiling)	0.940	0.000	0.940
Phi-4-multimodal (4.5B)	0.440	0.000	0.220
VectraYX-Vision-1B (v3B, step 1900)	0.080	0.060	0.025
VectraYX-Vision-1B (SFT-v2, step 400) — this checkpoint	0.020	0.000	0.010
1B backbone, no image (control)	0.020	0.000	0.000

The text-only control confirms the benchmark is not trivially answerable from text alone — the with-image v3B row (0.08) exceeds the no-image control (0.02).

Usage (llama.cpp multimodal)

bash

# Load model + vision tower
./llama-cli -m model.gguf --mmproj mmproj.gguf \
  --image /path/to/screenshot.png \
  -p "¿Qué herramienta de seguridad muestra esta captura y qué hace?" \
  --temp 0.7 -n 512

Usage (Ollama)

bash

# Register the model with its mmproj
ollama create vectrayx-vision-1b \
  -f <(printf 'FROM ./model.gguf
PARAMETER temperature 0.7
PARAMETER num_predict 512
')
# (mmproj must be co-located for multimodal; see Ollama LLaVA docs)
ollama run vectrayx-vision-1b

Notes / known limitations

Visual grounding is weak. The model is primarily an architecture/data/curriculum release pending a longer SFT run. Do not expect useful visual QA yet.
NoPE export: NoPE layers export as RoPE layers with rotary application disabled per layer. The GGUF RoPE permute fix uses stack([w1,w2], dim=2) interleaved.
Export is GGUF-splittable: the decoder checkpoint contains no visual weights and vice versa, matching the LLaVA model.gguf / mmproj.gguf convention.
This is the first released model to combine periodic NoPE layers with an injected visual-token block; the NoPE×vision interaction is an open research question (see the accompanying paper for the ablation design).

Citation

See the accompanying paper, VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use. Paper on Hugging Face.

Author

Juan S. Santillana — DevOps engineer (Globant). Institutional affiliation approval pending. Weights, benchmarks, and configs are released; the tool-calling MCP Modelfile will follow the visual-prompt-injection and over-triggering safety evaluation.

jsantillana/vectrayx-vision-1b