On-Device Named-Entity Recognition: A Deployability Study of Accuracy, Cost, Reliability, and Confidence
A new study evaluates nine NER systems across three paradigms to determine real-world deployability for on-device applications.
Researchers tested models ranging from 13M to 8B parameters, including spaCy, GLiNER, and generative LLMs like Qwen3 and DeepSeek-R1. The study focuses on practical deployment metrics—accuracy, cost, reliability, and confidence calibration—rather than leaderboard performance, providing a framework for evaluating NER systems without human-annotated data.