Research & Progress

The Scoreboard

Four architectures. One honest benchmark.

All models trained and evaluated on the same WLASL 100-class subset (739 training clips — roughly 7 examples per sign, a deliberately hard low-data regime) with a held-out test split our evaluation pipeline never trains on.

Phase 1 model evaluation — July 2026. Validation accuracy on 100 ASL sign classes; chance level is 1%.
Model	Approach	Best Val Accuracy	Status
Landmark LSTM	Pose keypoints → temporal LSTM (our patent-pending pipeline)	22.2%	Live in demo
3D CNN v1	Raw video, trained from scratch, 30 epochs	7.5%	Archived
3D CNN v2	Raw video, trained from scratch, 50 epochs	~2.5%	Controlled baseline
r3d_18 transfer	Raw video, Kinetics-400 pretrained backbone	In progress	Training

The Phase 1 finding

Keypoints beat pixels 3×.

On identical limited data, the keypoint-first architecture — extract skeletal hand and body landmarks first, then model their motion over time — outperformed every raw-video approach by roughly 3×. This is the architecture described in our patent-pending pipeline (USPTO #63/918,518), now supported by our own controlled evidence: when data is scarce, structure wins. And in sign language, where every signer, camera, and lighting condition differs, data is always scarcer than you'd like.

Methodology

How we measure

Every checkpoint runs through the same evaluation pipeline before any claim is made about it.

Held-out test set99 samples the models never see during training. Top-1 and Top-5 accuracy, macro-F1 across all 100 classes.

Calibration (ECE)When the model says "94% confident," is it right 94% of the time? Expected Calibration Error with reliability diagrams — because a confidence score you can't trust is worse than none.

Latency percentilesp50 / p95 / p99 inference time, model-only and end-to-end. Real-time communication access needs tail latency, not just averages.

Why publish this

Honest numbers are an accessibility issue.

Sign language AI has a credibility problem: systems built without Deaf people, evaluated on metrics that hide how they behave for real signers, marketed on demo reels. When a Deaf person relies on a recognition system that overstates itself, the failure lands on them.

SonZo AI is Deaf-led, and our founder's doctoral research at Gallaudet University focuses on exactly this: Deaf-centered evaluation of AI accessibility systems. Publishing our real Phase 1 numbers — including the models that plateaued — is that research philosophy in practice. 22.2% on 100 signs with 7 examples each is an early number, and we say so. The path from here is more data through the winning architecture, and every improvement will be measured by the same pipeline and published the same way.

What these numbers are, and aren't

Phase 1 is a low-data feasibility study on a 100-class WLASL subset — it establishes which architecture scales, not a production accuracy claim. The live demo at demo.sonzo.io runs the Landmark LSTM checkpoint and behaves like a 22%-accurate early model, because that's what it is. We'd rather show you a real early model than a scripted impressive one.

Follow the work.

Phase 2: scaling the keypoint pipeline's training data and class coverage — same benchmark, same published metrics.