cover

The Nuts and Bolts of Token Testing: Prompt Variations and Decoding in Practice

13 May 2025

Robust, repetitive prompts and UTF-8 understanding are key for accurately verifying and diagnosing under-trained tokens across language models.

cover

Secret Tokens, Secret Trouble: The Hidden Flaws Lurking in Big-Name AIs

13 May 2025

Glitch tokens persist in closed and open models; aligning tokenizer and training data, plus targeted checks, are key to safer, more efficient LLMs.

cover

How Tokenizer Choices Shape Hidden Risks in Popular Language Models

12 May 2025

Model-specific analysis reveals GPT-2, NeoX, and OLMo tokenizers each harbor unique under-trained tokens, influenced by training data and tokenizer choices.

cover

Where Glitch Tokens Hide: Common Patterns in LLM Tokenizer Vocabularies

12 May 2025

Untrained tokens often stem from unused byte tokens, merged fragments, and special tokens-patterns found across major LLMs regardless of architecture.

cover

How Many Glitch Tokens Hide in Popular LLMs? Revelations from Large-Scale Testing

12 May 2025

Under-trained token indicators efficiently flag risky tokens in LLMs, with cross-model results showing 0.1–1% of vocabularies consistently problematic.

cover

Comprehensive Detection of Untrained Tokens in Language Model Tokenizers

12 May 2025

Presents new methods for detecting ‘glitch tokens’ in LLMs-untrained tokens that cause unwanted behavior-and tools for safer, more robust language models.

cover

Supplementary Figures and Supplementary Tables

16 Dec 2024

cover

LLMs Rely on Contextual Knowledge Over Background Knowledge

16 Dec 2024

LLMs rely on contextual knowledge over domain-specific facts, maintaining accuracy even with synthetic gene names in reasoning tasks.

cover

How Extra Information Affects AI’s Ability to Think Logically

16 Dec 2024

LLMs demonstrate varying sensitivity to increasing distractors in reasoning tasks.