Combining marks
Detect zalgo text without breaking Vietnamese
Paste text to see how deep its combining marks stack, base character by base character. The naive rule — strip the marks, or allow only one — quietly corrupts Vietnamese, which needs two on a single letter.
The tool
Four lines: Vietnamese and French, which must survive any sane rule; a zalgo with two marks, which no threshold separates from the Vietnamese; and a sprawling one, which every threshold catches.
Stacked 7 marks deep — beyond any orthography.
No writing system needs this many marks on one base. disarm reports it as zalgo at the default threshold of 3.
How deep the marks stack
Every base character with the number of combining marks riding on it. Depths of one and two are unremarkable — that is where French, Vietnamese and Hebrew live. The colour only rises above that.
café1 naï1ve
Z͓̎2algo
Hͨ̊̽ͤ4e͓͔ͭ͑ͥͪͫ7
Your text at each threshold
is_zalgo flags a base carrying more than the threshold in marks.
Depths are counted after decomposition, so a precomposed ế
still counts the two marks it decomposes to.
A low threshold catches more abuse and starts rejecting real languages.
| Threshold | Verdict | Cost |
|---|---|---|
| 1 | zalgo | flags Vietnamese |
| 2 | zalgo | the capping default |
| 3 | zalgo | the default |
| 4 | zalgo | |
| 5 | zalgo |
Capped at two marks
strip_zalgo with disarm's default cap, which is what Vietnamese needs.
Tiếng Việt — Nguyễn Thị Hường café naïve Z͓̎algo Hͨ̊e͓͔
Running disarm 0.14.1, compiled to WebAssembly. Your text is never uploaded — the engine is loaded into this page and runs on your machine.
Where depth stops working
Mark depth is the only thing a counter can see, and it has a floor. These two are the same shape:
| Text | Decomposes to | Marks |
|---|---|---|
| ế — Vietnamese | U+0065 U+0302 U+0301 | 2 |
| Z͓̎ — zalgo | U+005A U+030E U+0353 | 2 |
A base and two marks in both cases, so no threshold tells them apart. Depth catches the sprawling kind reliably and the restrained kind not at all. Separating those needs to look at which marks appear and whether they form a real orthographic unit — a different question from how many.
What depth does buy you is a safe floor. Measured against disarm, with the deepest stack each sample reaches:
| Sample | Deepest stack | Flagged at threshold 1? | At 3? |
|---|---|---|---|
| Vietnamese — Tiếng Việt | 2 | yes | no |
| Hebrew with niqqud | 2 | yes | no |
| French — café naïve | 1 | no | no |
| Thai | 1 | no | no |
| Zalgo, sprawling | 5+ | yes | yes |
A threshold of one rejects ordinary Vietnamese and Hebrew. Three is disarm's default for detection, and two is its default when capping, because two is what Vietnamese needs.
The same thing in your own code
Each code block has been compiled and verified in CI. Provided under the MIT
license to illustrate disarm. There is no C here: the C ABI exposes no
is_zalgo or strip_zalgo, so there is nothing to call.
disarm on GitHub →
# Cap combining-mark stacking without corrupting Vietnamese.
# pip install disarm
from disarm import is_zalgo, strip_zalgo
# Vietnamese puts two marks on one base: ế is U+0065 U+0302 U+0301.
VIETNAMESE = "Tiếng Việt"
# Five marks on one base, which no writing system uses.
ZALGO = "Hͤͥͦͧͨ"
# The naive rule — at most one mark — rejects an ordinary Vietnamese word.
assert is_zalgo(VIETNAMESE, threshold=1), "a threshold of 1 rejects Vietnamese"
assert not is_zalgo(VIETNAMESE, threshold=3), "the default does not"
assert is_zalgo(ZALGO, threshold=3), "and still catches sprawling zalgo"
# Capping at two is what leaves Vietnamese untouched.
assert strip_zalgo(VIETNAMESE, max_marks=2) == VIETNAMESE
assert strip_zalgo(ZALGO, max_marks=2) != ZALGO
print("ok: threshold 1 rejects Vietnamese, threshold 3 does not, zalgo caught either way")
// Cap combining-mark stacking without corrupting Vietnamese.
// cargo add disarm
use disarm::api::{is_zalgo, strip_zalgo};
fn main() {
// Vietnamese puts two marks on one base: ế is U+0065 U+0302 U+0301.
let vietnamese = "Ti\u{1ebf}ng Vi\u{1ec7}t";
// Five marks on one base, which no writing system uses.
let zalgo = "H\u{0364}\u{0365}\u{0366}\u{0367}\u{0368}";
// The naive rule — at most one mark — rejects an ordinary Vietnamese word.
assert!(is_zalgo(vietnamese, 1), "a threshold of 1 rejects Vietnamese");
assert!(!is_zalgo(vietnamese, 3), "the default does not");
assert!(is_zalgo(zalgo, 3), "and still catches sprawling zalgo");
// Capping at two is what leaves Vietnamese untouched.
assert_eq!(strip_zalgo(vietnamese, 2), vietnamese);
assert_ne!(strip_zalgo(zalgo, 2), zalgo);
println!("ok: threshold 1 rejects Vietnamese, threshold 3 does not, zalgo caught either way");
}
// Cap combining-mark stacking without corrupting Vietnamese.
// npm i disarm
const assert = require("node:assert/strict");
const { isZalgo, stripZalgo } = require("disarm");
// Vietnamese puts two marks on one base: ế is U+0065 U+0302 U+0301.
const VIETNAMESE = "Tiếng Việt";
// Five marks on one base, which no writing system uses.
const ZALGO = "Hͤͥͦͧͨ";
// The naive rule — at most one mark — rejects an ordinary Vietnamese word.
assert.ok(isZalgo(VIETNAMESE, { threshold: 1 }), "a threshold of 1 rejects Vietnamese");
assert.ok(!isZalgo(VIETNAMESE, { threshold: 3 }), "the default does not");
assert.ok(isZalgo(ZALGO, { threshold: 3 }), "and still catches sprawling zalgo");
// Capping at two is what leaves Vietnamese untouched.
assert.equal(stripZalgo(VIETNAMESE, { maxMarks: 2 }), VIETNAMESE);
assert.notEqual(stripZalgo(ZALGO, { maxMarks: 2 }), ZALGO);
console.log("ok: threshold 1 rejects Vietnamese, threshold 3 does not, zalgo caught either way");
# Cap combining-mark stacking without corrupting Vietnamese.
# gem install disarm
require "disarm"
# Vietnamese puts two marks on one base: ế is U+0065 U+0302 U+0301.
VIETNAMESE = "Ti\u{1EBF}ng Vi\u{1EC7}t"
# Five marks on one base, which no writing system uses.
ZALGO = "H\u{0364}\u{0365}\u{0366}\u{0367}\u{0368}"
# The naive rule — at most one mark — rejects an ordinary Vietnamese word.
raise "a threshold of 1 rejects Vietnamese" unless Disarm.zalgo?(VIETNAMESE, threshold: 1)
raise "the default does not" if Disarm.zalgo?(VIETNAMESE, threshold: 3)
raise "and still catches sprawling zalgo" unless Disarm.zalgo?(ZALGO, threshold: 3)
# Capping at two is what leaves Vietnamese untouched.
raise "cap 2 must leave Vietnamese alone" unless Disarm.strip_zalgo(VIETNAMESE, max_marks: 2) == VIETNAMESE
raise "cap 2 must change zalgo" if Disarm.strip_zalgo(ZALGO, max_marks: 2) == ZALGO
puts "ok: threshold 1 rejects Vietnamese, threshold 3 does not, zalgo caught either way"
// Cap combining-mark stacking without corrupting Vietnamese.
// implementation("dev.disarm:disarm:0.14.1")
import dev.disarm.Disarm;
public class DetectZalgo {
public static void main(String[] args) {
// Vietnamese puts two marks on one base: ế is U+0065 U+0302 U+0301.
String vietnamese = "Tiếng Việt";
// Five marks on one base, which no writing system uses.
String zalgo = "Hͤͥͦͧͨ";
// The naive rule — at most one mark — rejects an ordinary Vietnamese word.
if (!Disarm.isZalgo(vietnamese, 1)) throw new AssertionError("a threshold of 1 rejects Vietnamese");
if (Disarm.isZalgo(vietnamese, 3)) throw new AssertionError("the default does not");
if (!Disarm.isZalgo(zalgo, 3)) throw new AssertionError("and still catches sprawling zalgo");
// Capping at two is what leaves Vietnamese untouched.
if (!Disarm.stripZalgo(vietnamese, 2).equals(vietnamese)) throw new AssertionError("cap 2 must leave Vietnamese alone");
if (Disarm.stripZalgo(zalgo, 2).equals(zalgo)) throw new AssertionError("cap 2 must change zalgo");
System.out.println("ok: threshold 1 rejects Vietnamese, threshold 3 does not, zalgo caught either way");
}
}
// Cap combining-mark stacking without corrupting Vietnamese.
// implementation("dev.disarm:disarm-kotlin:0.14.1")
import dev.disarm.kotlin.*
fun main() {
// Vietnamese puts two marks on one base: ế is U+0065 U+0302 U+0301.
val vietnamese = "Tiếng Việt"
// Five marks on one base, which no writing system uses.
val zalgo = "Hͤͥͦͧͨ"
// The naive rule — at most one mark — rejects an ordinary Vietnamese word.
check(vietnamese.isZalgo(1)) { "a threshold of 1 rejects Vietnamese" }
check(!vietnamese.isZalgo(3)) { "the default does not" }
check(zalgo.isZalgo(3)) { "and still catches sprawling zalgo" }
// Capping at two is what leaves Vietnamese untouched.
check(vietnamese.stripZalgo(2) == vietnamese) { "cap 2 must leave Vietnamese alone" }
check(zalgo.stripZalgo(2) != zalgo) { "cap 2 must change zalgo" }
println("ok: threshold 1 rejects Vietnamese, threshold 3 does not, zalgo caught either way")
}
Who this hurts
Stacked marks are usually treated as a joke, which is why the damage tends to be somewhere other than where the text is displayed.
| Where | What happens | Why depth is the wrong lever alone |
|---|---|---|
| Moderation and filters | A banned word carrying one mark per letter is no longer that string, and still reads as it. | One mark per base is depth 1, which no threshold can reject without rejecting French. |
| Layout | Marks render outside the line box, so text climbs into the rows above and below and covers them. | This is a rendering cost, not a content one: the string can be short and still wreck a page. |
| Storage and indexing | A visually short name can be thousands of codepoints, filling a column or a token budget. | Length in characters is the defence here, not depth. Cap the marks and cap the length. |
| Screen readers | Every mark may be announced, so a short name becomes minutes of speech. | An accessibility failure that a purely visual review never sees. |
| Copy and paste | Marks travel invisibly into commit messages, tickets and logs, and break alignment there. | The destination usually has no rendering budget for them at all. |
The engineering conclusion the page keeps circling: cap rather than
detect. strip_zalgo with a cap of 2 keeps every orthography
the tool above leaves alone and removes the excess from everything else, without
needing to decide whether a two-mark string was hostile. Detection is for
reporting; capping is for accepting input. Restrained zalgo stays
indistinguishable from real language by depth, and that is a property of the
signal rather than a gap in the implementation.
Found a string this gets wrong? The confusables table grew out of exactly that kind of report. Open an issue with it.
Related tools
- Truncate without breaking emoji — the same combining marks, as grapheme clusters.
- Check confusable characters — lookalike characters rather than stacked ones.
- Remove invisible characters — the invisible classes disarm strips outright.