Context before deletion
Unicode invisible characters
Unicode includes controls, format characters, combining marks and sequence components that may have no standalone glyph. “Invisible” describes appearance, not purpose. This inspector classifies exact scalars and preserves any category it cannot safely offer for deletion.
Common groups
Zero-width space, soft hyphen and word joiner can influence wrapping or copying. Byte-order marks can appear at the start of a UTF-8 file or unexpectedly inside decoded text. C0 controls can affect terminals and parsers. Unusual spaces can look ordinary while comparing differently.
Bidirectional controls affect display order. Zero-width non-joiner and joiner affect shaping in writing systems and also participate in emoji sequences. Variation selectors choose presentation. Combining marks modify neighboring scalars. Treating those groups as interchangeable would damage legitimate text.
- Reviewable artifacts: reveal and permit an explicit, per-occurrence choice where policy allows.
- Language or display-sensitive controls: reveal prominently and preserve.
- Valid emoji sequences: recognize the sequence context and preserve it.
- Unknown future format controls: preserve and report incomplete coverage.
- Replacement characters: flag the decoding problem; never pretend deletion restores lost data.
Coordinates and visible encoding
Every finding points to one non-empty scalar occurrence. Typed text includes derived UTF-8 bytes plus retained UTF-16 and scalar coordinates. Imported files also include original raw-byte offsets. One-based line and scalar-column coordinates resolve to the same occurrence.
Evidence excerpts use a visible-scalar encoding. Literal surrounding text stays literal, while hidden/control characters become trusted tokens derived from pinned Unicode data. Input never supplies a token name, and the renderer never inserts visitor text through innerHTML.
No normalization
The launch does not apply NFC, NFD, NFKC or NFKD. Normalization can be useful in a dedicated workflow, but it can change many scalar sequences beyond the selected hidden-character occurrence and make a “removed this one character” receipt false.
Output is therefore the admitted scalar sequence minus only the selected, non-overlapping deletions. Every other scalar and the admitted line-ending choice remain intact; the workbench explicitly announces when editing pasted text rebases that input to the visible editor value.
Primary references
These sources describe the external standards or mechanisms discussed above. The property’s cleanup decisions remain narrower than those specifications.