← Blog

How to find hidden characters in text: zero-width spaces, BOMs and more

✓ The technical facts in this article are checked against the Unicode database every time this site is published.

Two strings look exactly the same on screen, yet a login rejects one, a spreadsheet refuses to sum another, and a search finds nothing. When text misbehaves while looking perfect, the usual cause is a character you cannot see. This guide lists the codepoints responsible, shows how to expose them, and explains how to clean them out without breaking text that needs them.

The usual suspects

Unicode contains dozens of characters that render as nothing or as plain blank space. A handful cause almost every real-world problem:

  • U+200B Zero-Width Space — an invisible line-break hint. Web pages use it inside long words and URLs, so it rides along whenever you copy from a browser.
  • U+FEFF Zero-Width No-Break Space — better known as the byte order mark. Editors write it at the start of files; parsers that don't expect it fail on "line 1".
  • U+00A0 No-Break Space — looks like a normal space, is not one. It turns numbers into text in CSV imports and defeats find-and-replace aimed at ordinary spaces.
  • U+200C / U+200D Zero-Width Non-Joiner and Joiner — legitimate and essential in Persian, Indic scripts and emoji sequences, but surprising anywhere else.
  • U+200E / U+200F and U+202A–U+202E — direction marks and bidirectional overrides. They can make text display in a different order than it is stored.
  • U+00AD Soft Hyphen — invisible unless a line breaks at its position, then it suddenly appears as a hyphen.

Why they are more than a nuisance

Most of the time a hidden character is an accident with an annoying symptom: a coupon code that "doesn't exist", a username that is "already taken", a JSON file that "starts with an invalid token". But invisibility is also used on purpose. Some document-leak investigations rely on zero-width characters inserted as a per-copy fingerprint: each recipient receives text that reads identically but carries a unique invisible pattern. And bidirectional override characters were the basis of the 2021 Trojan Source research, which showed that source code can be made to display differently from how a compiler reads it. Knowing how to look is a practical skill in both situations.

Reveal them in seconds

Paste the suspicious text into the invisible text inspector. Every hidden codepoint appears as a tag with its number — U+200B, U+FEFF and so on — exactly where it sits between the visible letters, and the counter reports how many there are. Nothing is uploaded: the check runs entirely in your browser, which matters when the text is a password or a private document. The inspector deliberately ignores ordinary spaces, tabs and line breaks, and it does not flag the accents and vowel marks of scripts like Hindi or Arabic, which are visible characters in their own right.

Developers have a second option inside their tools. Modern code editors can highlight invisible and look-alike Unicode characters, and many linters can be configured to reject them in source files. For one-off checks of copied text, a paste-and-look inspector is faster.

Remove them without breaking anything

Deleting every invisible character is tempting and often wrong. The joiner U+200D holds multi-person and flag emoji together; strip it and a family emoji falls apart into separate faces. The non-joiner U+200C changes how Persian words are written. So clean by purpose:

  1. Data fields (codes, IDs, numbers, email addresses): remove every zero-width and direction character, and turn no-break spaces into ordinary spaces. Nothing legitimate lives there.
  2. File headers: strip a leading U+FEFF before parsing; keep the rest of the file untouched.
  3. Human-language text: remove only what you have identified, and leave joiners in place when the text contains emoji or scripts that need them.
  4. Source code: treat bidirectional override characters as errors unless a string literal genuinely requires them.

Check the result

After cleaning, paste the text into the inspector once more: the invisible count should read zero for data fields, or show only the characters you chose to keep. If you are on the other side of the problem — you need an invisible character on purpose — the character library groups all 36 of them by where they survive, and the Unicode reference covers the six that matter most in depth.