Lightweight MLP that classifies 36 American Sign Language static gestures
(letters A–Z, digits 0–9) from 21 MediaPipe hand landmarks, not raw
pixels. This makes it robust to background, lighting, and webcam distribution shift,
and fast enough for real-time CPU inference.
Inputs / Outputs
Input:float32[1, 63] — 21 hand landmarks (x, y, z) flattened.
Normalize before inference: subtract wrist (landmark 0), then divide by the
wrist→middle-finger-MCP distance ‖landmark9 − landmark0‖ (scale invariance).
Output:float32[1, 36] logits. argmax → class index; map via
mlp_classes.json (sorted 0-9, A-Z). Apply softmax for confidence.
The confusion matrix is fully diagonal — the wrist-centered, scale-normalized
landmark representation makes the 36 classes near-linearly separable. Closed-fist
signs (O, C, A, T) contribute fewer samples because MediaPipe detects them
less often (see distribution).
Usage
python
1import json, numpy as np, onnxruntime as ort
23sess = ort.InferenceSession("mlp_asl.onnx", providers=["CPUExecutionProvider"])4classes = json.load(open("mlp_classes.json"))56defnormalize(pts):7 pts = pts - pts[0]8 scale = np.linalg.norm(pts[9])or1.09return(pts / scale).reshape(1,-1).astype("float32")1011logits = sess.run(None,{"input": normalize(landmarks_21x3)})[0][0]12pred = classes[str(int(logits.argmax()))]
CC BY-NC 4.0 — free for research, education, and personal/open projects with
attribution; no commercial or enterprise resale. Please cite if you use it. The
ASL-HG dataset is owned by its original authors (cite separately).
