The invisible characters hiding in your AI text

Copy a paragraph out of a chat window, paste it into your CMS, and the text looks exactly the same. Byte for byte, it usually is not.

Characters with no width

Unicode contains a set of codepoints that occupy no space and render as nothing. They exist for real typographic reasons. U+200B ZERO WIDTH SPACE marks a place where a line may break inside a long word. U+200D ZERO WIDTH JOINER glues emoji together, which is how a family emoji is built from several people. U+FEFF began life as a byte order mark.

Because they render as nothing, they also survive nearly every copy-paste operation without a human noticing. That makes them a convenient channel for anything you want to travel with a piece of text but not be seen in it.

The Tags block

The most interesting range is U+E0000 to U+E007F, the Unicode Tags block. Each codepoint in it mirrors one printable ASCII character. U+E0041 corresponds to A, U+E0061 to a, and so on.

The consequence is worth stating slowly: any ASCII string at all can be re-encoded into Tags characters, appended to a visible sentence, and become completely invisible while remaining perfectly recoverable. A session id, an account handle, a timestamp. The text looks clean and carries a fingerprint.

Bidirectional controls

A third group changes reading order rather than occupying space: U+202E RIGHT-TO-LEFT OVERRIDE and its relatives. In prose they are usually harmless leftovers from a mixed-script source document. In a filename or an identifier they can make text display in an order different from the order it is stored in, which is the basis of a well-known class of spoofing trick.

Seeing them

You cannot fix what you cannot see, and no normal editor will show you these. Three practical options:

1. Search for the codepoint directly. Most editors accept \x{200B} or similar in a regular expression search.
2. Pipe the text through something that prints codepoints rather than glyphs.
3. Paste it into a tool built for the job, such as the cleaner on our home page, which lists every codepoint it finds by name and shows you where in the text it sat.

What to remove and what to keep

Not every invisible character is junk, and a cleaner that removes all of them is wrong in a way users will notice.

U+FE0F VARIATION SELECTOR-16 is the difference between a red heart and a monochrome one. A zero width joiner between two pictographs is holding an emoji together. Strip those and you have broken visible content in the name of cleanliness.

So the rule our engine follows is narrower than “remove the invisibles”. It removes the whole Tags block unconditionally, removes zero-width and format characters in prose, and leaves a joiner alone when a pictograph sits on both sides of it. Everything else it reports rather than deletes.