pip install -r requirements.txt1from janegpt_v2_janus.inference import JaneGPTv3NLU
2
3nlu = JaneGPTv3NLU(
4 model_path="weights/janegpt_v2_janus.pt",
5 tokenizer_path="weights/tokenizer.json",
6)
7
8state = {}
9result = nlu.predict("set volume", state=state)
10print(result)
11
12if result.get("type") == "command":
13 state = nlu.update_state(result, state)1from runtime.jane_nlu_runtime import JaneNLURuntime
2
3rt = JaneNLURuntime(base_dir=".")
4state = {}
5
6out, state = rt.handle_turn("set volume", state)
7print(out) # expected: clarify prompt for missing VALUE
8
9out, state = rt.handle_turn("55", state)
10print(out) # expected: resolved local command1python examples/demo_inference.py
2python examples/demo_runtime.py
3python examples/demo_runtime_suite.py| Component | Configuration | Details |
|---|---|---|
| Backbone Type | Transformer (GPT-style) | Bidirectional, non-causal attention |
| Vocabulary Size | 8,192 | BPE tokenization |
| Embedding Dim | 256 | Token + Rotary Position embeddings |
| Attention Heads | 8 Query, 4 KV | Grouped Query Attention (GQA) for efficiency |
| Head Dimension | 32 | per head_dim = embed_dim / num_heads |
| Transformer Blocks | 8 Layers | Each with Attn + FFN + Residuals |
| Feed-Forward Hidden | 672 | SwiGLU gate activation |
| Position Encoding | RoPE | Rotary Position Embeddings (theta=10000) |
| Normalization | RMSNorm | Pre-layer normalization |
| Max Sequence Length | 96 tokens | Approximately 60-80 words |
| Dropout Rate | 0.1 | Applied during training |
| Total Parameters | 7,949,626 | All trainable |
| Parameter Breakdown | Backbone: 7.80M, Task Heads: 146K | Efficient multitask design |
| Task | Type | Classes | Architecture |
|---|---|---|---|
| Domain Classification | Sequence-level | 10 domains | Pooled → Linear(256) → GELU → Linear(10) |
| Action Classification | Sequence-level | 33 actions | Pooled → Linear(256) → GELU → Linear(33) |
| Slot Tagging | Token-level | 15 BIO labels | Per-token → Linear(256) → Linear(15) |
| Metric | Detail | Jane v2 | Janus |
|---|---|---|---|
| Speed (mean latency) | CUDA, batch=1 | 31.60 ms | 25.31 ms |
| Throughput | CUDA, single GPU | 32 pred/sec | Stable across 82 turns, 0 errors |
| OOD F1 | BANKING77 | 94.31% | 87.80% |
| OOD F1 | CLINC OOS | 89.16% | 79.23% |
| OOD Precision | BANKING77 | 99.35% | 100.00% |
| OOD Precision | CLINC OOS | 99.14% | 100.00% |
| OOD Recall | BANKING77 | 89.75% | 78.25% |
| OOD Recall | CLINC OOS | 81.00% | 65.60% |
| Validation Accuracy | Domain (best epoch) | — | 99.83% |
| Validation Accuracy | Action (best epoch) | — | 99.87% |
| Validation Accuracy | Domain+Action pair (best epoch) | — | 99.83% |
| Slot Extraction F1 | All 15 slot types | — | 1.000 (100%) |
| Training Loss | Epoch 1 → 4 | — | 0.060 → 0.020 → 0.002 → 0.001 |
| Validation Loss | Epoch 1 → 3 | — | 0.0153 → 0.0116 → 0.0115 (stable) |
| Runtime Reliability | 82-turn conversation test | — | 0 errors, 0 crashes |
| Domain Confusion | 10 domains | — | 99%+ per-domain, minimal cross-confusion |
| Action Confusion | 33 actions | — | Perfect diagonal, no action commonly confused |
1{
2 "type": "command",
3 "domain": "apps",
4 "action": "launch",
5 "slots": {
6 "APP_NAME": {
7 "text": "chrome",
8 "start": 5,
9 "end": 11,
10 "confidence": 0.999
11 }
12 },
13 "confidence": 0.97,
14 "route": "local"
15}1{
2 "type": "clarify",
3 "question": "What value should I set it to?",
4 "debug": {
5 "domain": "volume",
6 "action": "set",
7 "reason": "missing_VALUE"
8 }
9}
1.
2|- README.md
3|- .gitattributes
4|- LICENSE
5|- requirements.txt
6|- assets/
7| |- jane-janus-glitch.webp
8|- janegpt_v2_janus/
9| |- __init__.py
10| |- architecture.py
11| |- dataset.py
12| |- inference.py
13| |- labels.py
14| |- multitask.py
15|- runtime/
16| |- jane_nlu_runtime.py
17|- examples/
18| |- demo_inference.py
19| |- demo_runtime.py
20| |- demo_runtime_suite.py
21|- weights/
22| |- janegpt_v2_janus.pt
23| |- tokenizer.json
24|- reports/
25| |- fair_benchmarks.json
26| |- fair_benchmarks.md
27| |- janus_model_report.json
28| |- janus_model_report.md
29| |- public_benchmarks.json
30| |- *.png benchmark visuals