{"uuid": "4e52958d-c862-4ae3-8c70-7db3dd62b502", "vulnerability_lookup_origin": "1a89b78e-f703-45f3-bb86-59eb712668bd", "author": "9f56dd64-161d-43a6-b9c3-555944290a09", "vulnerability": "CVE-2021-42574", "type": "seen", "source": "https://gist.github.com/LeahyCC/1a4f0b984a180fcd6fc797d5676ebf70", "content": "# Scanning a repo for hidden text markers\n\nA zero-dependency Node script that finds invisible characters, homoglyphs, and\nother steganographic markers in source code \u2014 the tricks used to fingerprint a\ncopy of a text file so it can be traced back to whoever leaked or shared it.\n\nWorks in any repo, any language, git or not. Node >= 18. No install.\n\n---\n\n## Quick start\n\nSave `scan-hidden-markers.mjs` anywhere, then:\n\n```bash\n# report only, never modifies anything\nnode scan-hidden-markers.mjs\n```\n\nRun it from your repo root. It respects `.gitignore` and skips binaries and\nlockfiles automatically. Exit code is `1` if anything is found, `0` if clean.\n\n```bash\nnode scan-hidden-markers.mjs src docs   # only these paths\nnode scan-hidden-markers.mjs --all      # ignore .gitignore, walk everything\nnode scan-hidden-markers.mjs --json     # machine-readable\nnode scan-hidden-markers.mjs --strict   # also show benign emoji selectors\n```\n\nCleaning is a separate, explicit step:\n\n```bash\nnode scan-hidden-markers.mjs --fix              # strip invisible chars\nnode scan-hidden-markers.mjs --fix --trailing   # also trim trailing whitespace\ngit diff                                        # ALWAYS review before committing\n```\n\n**Run without `--fix` first.** `--fix` only deletes characters that are\ninvisible *and* never meaningful in source. It deliberately leaves homoglyphs,\nnon-breaking spaces, and private-use codepoints alone, because those are\nsometimes legitimate and need a human decision.\n\n---\n\n## How text actually gets watermarked\n\nThis is the part that surprises people. There is no single \"tracking pixel\" for\ntext. There are about six independent families of technique, and they differ\nwildly in how detectable they are.\n\n### 1. Invisible characters (the common one)\n\nUnicode has many codepoints that render as nothing at all. Sprinkle them\nbetween words and you have a binary channel: presence/absence of a zero-width\nspace at each word boundary encodes one bit per gap. Thirty words is enough to\nuniquely tag thousands of recipients.\n\n| Family | Codepoints | Why it exists legitimately |\n|---|---|---|\n| Zero-width | `U+200B` ZWSP, `U+200C` ZWNJ, `U+200D` ZWJ, `U+2060` word joiner, `U+FEFF` | Script shaping, emoji sequences |\n| Soft hyphen | `U+00AD` | Hyphenation hints |\n| Invisible math | `U+2061`\u2013`U+2064` | MathML semantics |\n| Hangul/Braille filler | `U+3164`, `U+FFA0`, `U+2800` | Filler glyphs, blank braille cell |\n\n### 2. Unicode Tag characters \u2014 the big one\n\nThe block `U+E0000`\u2013`U+E007F` mirrors all of ASCII, but renders as absolutely\nnothing in essentially every font and editor. `U+E0041` is an invisible \"A\".\n\nThis means **an arbitrary ASCII string can be embedded invisibly inside any\nother string.** A name, an email, an employee ID, a timestamp \u2014 appended to a\nline of code and completely unreadable without a tool like this. It survives\ncopy/paste, most editors, chat apps, and diff views. This is the single most\nimportant thing to scan for, and it is why the script rates it `CRITICAL`.\n\n### 3. Variation selectors \u2014 the sneaky one\n\n`U+FE00`\u2013`U+FE0F` and `U+E0100`\u2013`U+E01EF` are modifiers that adjust how the\n*preceding* character renders. Each carries 256 possible values, so a short run\nof them encodes arbitrary bytes.\n\nThe catch: **`U+FE0F` after an emoji is completely normal.** `\u26a0\ufe0f` is `U+26A0`\nfollowed by `U+FE0F`. A naive scanner floods you with false positives, and a\nnaive auto-fixer silently breaks every emoji in your docs.\n\nThis script judges them by context \u2014 a selector after a pictographic base or in\na keycap sequence is normal text; one after a letter, a space, or another\nselector is a payload. Benign ones are counted and disclosed in a footer note,\nnot hidden.\n\n### 4. Homoglyphs\n\nCyrillic `\u0430` (`U+0430`) is visually identical to Latin `a` in most fonts.\nSwapping a few letters in identifiers or comments creates a fingerprint that is\ninvisible to the eye but trivially machine-readable. Greek, Armenian, Cherokee,\nand fullwidth Latin all provide lookalikes.\n\nThis is also a genuine **security** issue, not just a privacy one: `p\u0430ssword`\nand `password` are different identifiers, and homoglyph attacks are used to\nsneak lookalike package names and function names past code review.\n\n### 5. Bidirectional controls (\"Trojan Source\")\n\n`U+202A`\u2013`U+202E` and `U+2066`\u2013`U+2069` reorder how text displays without\nchanging what the compiler sees. A comment can be made to *look* like it ends\nwhere it doesn't, hiding live code inside what appears to be a comment. This is\n[CVE-2021-42574](https://trojansource.codes/). Treat any hit here as a security\nfinding, not a privacy one.\n\n### 6. Whitespace steganography\n\nTrailing spaces vs. tabs at line ends, non-breaking vs. regular spaces between\nwords, one blank line vs. two. All invisible, all survive most formatters, all\nencode bits. This is the oldest technique and the noisiest to detect, which is\nwhy the script rates trailing whitespace `LOW` \u2014 you will have plenty of\ninnocent hits.\n\n---\n\n## What this will NOT catch\n\nBe honest with yourself about the limits. Character-level scanning cannot see:\n\n- **Linguistic/semantic watermarking.** Choosing \"fetch\" vs. \"retrieve\" vs.\n  \"get\" in comments, or reordering independent statements. Increasingly common\n  in LLM-generated text. Effectively undetectable by any tool, and unprovable\n  without the original.\n- **Structural fingerprints.** Import order, brace style, variable naming\n  patterns, blank-line rhythm. Running a formatter (Prettier, Biome, gofmt,\n  black) destroys most of these \u2014 that's the real defense.\n- **Metadata outside the text.** Git author emails, commit timestamps, EXIF in\n  images, PDF/Office document properties, font files. Use `exiftool` for media,\n  `git log --format=fuller` for history.\n- **Anything inside binaries, minified bundles, or base64 blobs.** The script\n  skips these by design; markers there need different tooling.\n- **Server-side tracking.** A URL in a comment or a remote asset reference can\n  phone home when fetched. Grep your dependencies and asset URLs separately.\n\nA clean scan means \"no character-level markers.\" It does not mean \"definitely\nnot watermarked.\"\n\n---\n\n## Checking code from a specific person\n\nIf your concern is a particular contributor, scan only what they touched:\n\n```bash\n# every file that author has ever modified\ngit log --author=\"Their Name\" --name-only --pretty=format: \\\n  | sort -u | grep -v '^$' > /tmp/their-files.txt\n\nxargs -a /tmp/their-files.txt node scan-hidden-markers.mjs --all\n```\n\nTo check whether markers were present historically and later removed, search\nthe full object database (needs a `git` built with PCRE, i.e. `git grep -P`):\n\n```bash\ngit grep -nP \"[\\x{200B}-\\x{200D}\\x{FEFF}\\x{2060}\\x{202A}-\\x{202E}\\x{E0000}-\\x{E007F}]\" \\\n  $(git rev-list --all) -- . 2>/dev/null | head -50\n```\n\nThat last one is slow on big repos. Narrow it with `--since` or a path if\nneeded.\n\nA caution worth stating plainly: finding invisible characters is **not** proof\nof intent. Editors, CMSes, Word, Jira, Slack, and copy/paste from web pages all\ninject zero-width and non-breaking characters routinely. Confirm what the\ncharacters actually encode before drawing a conclusion about a colleague.\n\n---\n\n## Keeping it clean\n\nAdd a CI gate \u2014 the script already exits `1` on findings:\n\n```yaml\n# .github/workflows/hidden-markers.yml\nname: hidden-markers\non: [push, pull_request]\njobs:\n  scan:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: actions/setup-node@v4\n        with: { node-version: '20' }\n      - run: node scan-hidden-markers.mjs\n```\n\nOr as a pre-commit hook, scanning only what is staged:\n\n```bash\n#!/bin/sh\n# .git/hooks/pre-commit\nfiles=$(git diff --cached --name-only --diff-filter=ACM)\n[ -z \"$files\" ] && exit 0\necho \"$files\" | xargs node scan-hidden-markers.mjs --all\n```\n\nBelt and braces: an `.editorconfig` with `trim_trailing_whitespace = true` and\n`charset = utf-8`, plus a formatter on save, removes most re-introduction paths.\n\n---\n\n## License\n\nPublic domain (CC0). Use it however you like.\n", "creation_timestamp": "2026-08-12T03:30:26.662867Z"}