You can deploy this model and start earning money today!

shannondata/
rapid_classifier

Name: rapid_classifier
Availability: InStock

onnx

Views

No views yet

ShannonBase Rapid Workload Classifier

A LightGBM binary classifier, exported to ONNX, that ShannonBase uses to decide whether a query should run on the primary MySQL (InnoDB / OLTP) engine or be offloaded to the Rapid secondary engine (OLAP).

It is consumed at query-prepare time by Query_arbitrator (storage/rapid_engine/ml/query_arbitrator.cpp) via ONNX Runtime, and ships with the server as extra/llm-models/shannon_rapid_classifier.onnx.

Model I/O

	Name	Type	Shape
Input	`float_input`	`float32`	`[N, 18]`
Output 0	`probabilities`	`float32`	`[N, 2]`
Output 1	`label`	`int64`	`[N]`

Class 1 = OLAP (offload to Rapid), class 0 = OLTP (keep on the primary). The engine reads output 0 and takes probabilities[1] as the offload score.

Note: output 0 must be a plain float tensor. Export with zipmap=False — a ZipMap output makes the tensor read fail and every query falls back to the primary engine.

Decision rule

The score is compared against a threshold of 0.5. For queries with at least 4 of the OLAP-shaped features set (has_group_by, has_having, has_aggregation, has_order_by, has_subquery), the threshold is scaled by 0.6 to bias toward offloading. A score above the effective threshold routes the query to Rapid.

Feature vector

Order is significant and must match training exactly.

#	Feature	Description
0	`mysql_total_ts_nrows`	Rows scanned by non-index-ref table scans
1	`mysql_cost`	Estimated primary-engine cost
2	`count_all_base_tables`	Number of base tables
3	`count_ref_index_ts`	Table scans served by an index ref
4	`base_table_sum_nrows`	Sum of base-table cardinalities
5	`are_all_ts_index_ref`	All scans are index refs (0/1)
6	`table_count`	Tables in the query block
7	`has_having`	HAVING clause present (0/1)
8	`has_group_by`	GROUP BY present (0/1)
9	`has_rollup`	ROLLUP present (0/1)
10	`has_order_by`	ORDER BY present (0/1)
11	`has_limit`	LIMIT present (0/1)
12	`has_join`	More than one table (0/1)
13	`has_subquery`	Subquery present (0/1)
14	`has_aggregation`	Aggregate function present (0/1)
15	`select_list_size`	Number of select-list items
16	`where_condition_count`	Number of top-level WHERE conditions
17	`estimated_rows`	Estimated result cardinality

Usage

bash

hf download shannondata/rapid_classifier shannon_rapid_classifier.onnx \
  --local-dir extra/llm-models/

python

import numpy as np, onnxruntime as ort

sess = ort.InferenceSession("shannon_rapid_classifier.onnx")
x = np.zeros((1, 18), dtype=np.float32)
probs = sess.run(None, {"float_input": x})[0]
print("offload score:", probs[0][1])

Training

Trained and exported by Shannon-Data/ShannonBase-Tools. To roll out a new model, replace extra/llm-models/shannon_rapid_classifier.onnx in the ShannonBase tree.