Houjun LabCan a small tracker run on a CPU
and still be trusted?
Vision Lab finds people in video and follows each one with a stable ID, using a small open detector (YOLOX) and a published tracker (ByteTrack) rebuilt in C++. Everything runs on CPU threads inside locked-down Linux containers, the way it would on a small device. The question is not only how fast it is, but what the published numbers leave out.
MOT17-02, frame 180, nano model. Colours follow tracker IDs; trails show the last two seconds. Frame: MOTChallenge authors, CC BY-NC-SA 3.0, non-commercial research.
The answer, in three numbers
The question
Small detectors and trackers are usually judged by a speed figure and a benchmark score. Both can look better than the system really is.
Is the port faithful?
Does the C++ version give exactly the same tracks as the authors' Python code, not just similar scores?
What does it cost?
Frames per second, latency per stage and memory at 1, 2 and 4 CPU threads, and what 8-bit quantisation buys.
How does it fail?
Accuracy on footage the model never saw, and concrete cases where people are lost or swap identities.
Design
The pass/fail rules were written before any model or video was downloaded, and were never moved after a result came in.
Export and check
The authors' YOLOX nano and tiny pedestrian models are exported to ONNX and compared with the originals on fixed frames.
Rebuild in C++
Image preparation, box decoding, duplicate removal and the ByteTrack tracker are written in C++ on ONNX Runtime, then compared with the Python reference.
Separate seen from unseen
The published scores were measured on MOT17 training footage, which these models were trained on. MOT17 is therefore used only to check the port; MOT20, never used for training or tuning, measures accuracy.
Measure, then reproduce
Speed and INT8 are measured under fixed CPU budgets; the whole study is then rerun from a clean copy of the code.
Results
In one paragraph
The C++ tracker matches the official one frame for frame and reproduces the published scores. The smallest model keeps up with live video on four CPU threads. On crowded footage it never saw, accuracy falls well below the published figure. 8-bit quantisation helped the larger model a lot at a small cost, but broke the smallest one; the repair that fixed it removed the speed gain. A classical tracker of the kind used in older video systems drifts off its targets within seconds.
How to read the numbers
MOTA sums missed people, false boxes and identity swaps against the number of people present (100 is perfect; it can go below zero). IDF1 measures how consistently each person keeps one ID. Scores come from two standard evaluators that treat edge cases differently, so each figure names its evaluator.
1. The port is exact
Across all 22 sequence/model runs, 2,064,047 output rows match the authors' Python tracker in frame, ID and box. Against the published MOT17 scores the C++ pipeline lands within 0.6 points (nano 69.2 MOTA / 66.8 IDF1 against 69.0 / 66.3; tiny 77.1 / 71.6 against 77.1 / 71.5).
2. Fast enough on four threads — for the small model
Speed on CPU threads
The numbers
| Model | 1 thread | 2 threads | 4 threads | Peak memory |
|---|---|---|---|---|
| nano (0.9 M parameters) | 11.6 FPS | 19.1 FPS | 27.3 FPS | 154 MiB |
| tiny (5 M parameters) | 3.2 FPS | 5.7 FPS | 9.8 FPS | 206 MiB |
End to end, from reading the JPEG to the tracked boxes, at the model's native 608×1088 input. The bar set in advance was 25 FPS: only nano on four threads clears it. Measured on an Apple M2 Max in a Docker Linux VM, which is faster than a typical edge board.
3. Denser crowds: 56 MOTA on MOT20, against 69 on training footage
On held-out MOT20, nano scores 56.0 MOTA / 51.4 IDF1 and tiny 61.1 / 59.5 (motmetrics), against 69.2 and 77.1 on the MOT17 footage they were trained on. MOT20 is also far more crowded, so the drop cannot be blamed on training exposure alone — but the published figure is not what a new scene should expect.
Accuracy on footage it was trained on, and on crowds it never saw
4. INT8: a real gain for one model, a collapse for the other
8-bit models: accuracy lost on unseen crowds
The numbers
| Model | Speed-up | MOTA change on MOT20 | Verdict (rule: ≥1.3× and ≤1 point lost) |
|---|---|---|---|
| tiny | 1.9–2.5× | −1.06 | Just misses |
| nano | 0.9–1.3× | −56.1 | Fails |
| nano, repaired | 0.97–1.03× | −0.70 | Accurate, but no faster |
Quantised nano missed almost every person on MOT20. The cause was the first layer: its 8-bit step was too coarse and rounded every negative value to zero. The repair was chosen on MOT17 only and run once on MOT20; it restored accuracy by keeping the arithmetic in floating point, which removed the speed advantage. Full-precision nano stays the recommended setting.
5. Where it fails
- Long occlusion. A person hidden behind a pillar for more than about a second comes back with a new ID: the tracker had already given up on the old one.
- Swaps in plain view. When another person overlaps in a crowd, an ID can move from one person to the other even though both stay visible.
- Small and cut-off people. People at the edge of the frame or showing only a small part of their body are often missed.
6. Against a classical tracker
A mean-shift tracker — colour-histogram tracking, the method behind many video systems of the 2000s — was started from the annotated first box of every person. It scored −252.3 MOTA / 3.1 IDF1 on MOT20, against 62.4 / 53.3 for nano (TrackEval). Even when its boxes were removed the moment each person left the scene, it stayed at −72: the windows drift off their targets. An ID that never changes is not the same as following the right person.
7. It reproduces
Rerun from a clean copy of the code, every accuracy score and every track file came out identical. Speed varied by −2.7% to +5.8% between full reruns.
Limits
- Speed was measured on a desktop-class Apple chip inside a virtual machine, not on an edge board. Relative results (thread scaling, INT8 versus full precision) transfer better than absolute frame rates.
- The models detect pedestrians only. Nothing here measures vehicles or other objects.
- No model was trained or fine-tuned; the study uses the ByteTrack authors' published checkpoints.
- MOT17 and MOT20 are licensed for non-commercial research. Results and the image above inherit that licence.
- One INT8 scheme and one repair were tested. Other quantisation methods may behave differently.
Code and licence
The project's own code and documentation are open source under the Apache License 2.0. Videos, annotations and model weights are not part of the repository; they keep their own licences. Every run, including the failed ones, is recorded in the repository with its data. See the licence page.