The story’s score and claims

Comments on No. 50 of 62 · trius.canton.graphics · edition of Sunday 15 November 2026

TriusLD – A Small Tokenizer for LLMs with Robust Multilingual Support

Summary

TriusLD is a 65,535-slot tokenizer for LLMs that allocates vocabulary across 19 writing systems using per-script quotas plus a shared byte fallback. Its author reports 0.52 tokens per character averaged over 58 languages, versus 0.84 for GPT-4's vocabulary, while English gets 3.85 characters per token, about 18% behind a 200,000-token…

Comments

Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules