AI Tokenizer
See how GPT models split your text into tokens. Each token is highlighted, with its ID one click away. Uses the o200k_base encoding behind GPT-5, GPT-4.1 and GPT-4o, and runs entirely in your browser.
- Tokens
- 0
- Characters
- 0
- Words
- 0
- Characters per token
- –
Loading the tokenizer… the count is an estimate for now. The tokenizer could not load. Check your connection and refresh the page; counts are estimates until then. Exact for GPT-5, GPT-4.1, GPT-4o and o-series models. Claude and Gemini use their own tokenizers, so expect different counts.
Tokens appear here as you type.
Showing the first tokens. The counts above cover all of your text; Copy IDs copies every token.
Spaces show as · and line breaks as ↵. A highlighted block with several IDs is one character spread over several tokens, such as an emoji.
Want the price? Compare what your prompt costs on GPT, Claude and Gemini →
About the AI Tokenizer
Language models read text as tokens: common words are one token, rarer words, numbers, emoji and non-English scripts split into several. The tokenizer shows exactly where those splits fall, with each token highlighted and its ID, using the o200k_base encoding behind GPT-4o, GPT-4.1 and GPT-5.
How to use it
- 1 Paste or type your text, or try the example.
- 2 Read the token, character and word counts, and see each token highlighted.
- 3 Switch to Token IDs to see the numbers the model reads, and copy them as JSON.
Frequently asked questions
What is a token?
A token is a chunk of text a language model reads as one unit. In English, one token is about 4 characters or three quarters of a word, but the exact split depends on the tokenizer.
Which models does this tokenizer match?
It uses o200k_base, the encoding for GPT-4o, GPT-4.1, GPT-5 and the o-series models, so counts for those are exact. Claude, Gemini and Llama use their own tokenizers, so their counts differ, usually by 10–30%.
Why do Bangla, Hindi or emoji use more tokens?
Tokenizers are trained mostly on English, so other scripts and emoji are split into smaller pieces, sometimes single bytes. The same sentence can cost two to four times as many tokens in Bangla as in English.
Why does a word with a leading space count differently?
Most tokens include the space before a word, so " hello" and "hello" are different tokens. Spaces are shown as dots so you can see this.
Is my text sent anywhere?
No. The tokenizer runs entirely in your browser, so your text never leaves your device.
Related tools
AI Output Cleaner
Clean ChatGPT and Claude text for Word, email, and WhatsApp
AI Token & Cost Calculator
Count tokens and compare GPT, Claude, and Gemini API costs
JSON Repair & LLM Output Validator
Fix broken JSON from ChatGPT, Claude, or Gemini and check it against a schema
AI Crawler Checker
See which AI crawlers, such as GPTBot and ClaudeBot, your robots.txt allows
llms.txt Generator
Create an llms.txt file that guides AI tools to your best pages
Speech to Text & Subtitle Maker
Transcribe audio and video to text and SRT subtitles in your browser