disarm

Whole-script confusables

Detect script spoofing

A string can be written entirely in one script — nothing mixed, so every mixed-script check passes it — and still consist only of characters that look Latin. раураӏ is six Cyrillic letters and is drawn exactly like paypal. Paste any string to see it character by character; hostnames get the extra label analysis, because that is where this usually bites.

The tool

The example spells a brand entirely in Cyrillic, so every character of the label agrees with every other and a mixed-script check reports nothing — yet it folds exactly onto the Latin name. Compare it with the others below.

Whole-script spoof.

Every character of the first label is one script, and it folds to paypal.com under a Latin top-level domain. That is a domain imitating a Latin name, not a site written in another language.

Character by character

Every character that has a Latin lookalike, with its codepoint and what it folds to. This is the part the string itself cannot show you.

рU+0440 → pаU+0430 → aуU+0443 → yрU+0440 → pаU+0430 → aӏU+04CF → l.com

6 of 9 letters fold onto Latin. Scripts present: Cyrillic, Latin.

Label by label

Each label with the scripts it uses. The top-level domain is shown too, because its script is half the question.

раураӏCyrillic · folds onto Latin
.
comTLD · Latin

Folds to

The canonical form. When this is a familiar name, that is the point.

paypal.com

Every signal disarm reports

SignalValueWhat it means
suspicious true disarm's overall flag. True for spoofs and for ordinary non-Latin domains alike, which is why it cannot be acted on alone.
mixed_script false More than one script inside a single label.
whole_script_confusable true A single-script label that folds onto a Latin name. The signal that separates a spoof from a real foreign-language domain.
cross_label_script true Different labels use different scripts.
has_confusables true Contains characters with Latin lookalikes.
bidi_conflict false Right-to-left letters reverse the reading order.
bidi_control false Contains bidi control characters.
has_invisible false Contains characters that render as nothing.

Running disarm 0.14.1, compiled to WebAssembly. The hostname is never uploaded — the engine is loaded into this page and runs on your machine.

One script, entirely Latin lookalikes

The hard case is not a word with a foreign letter smuggled into it. That mixes scripts, and a mixed-script check catches it. The hard case is a word written wholly in one script where every character happens to have a Latin twin:

CharacterCodepointScriptLooks like
рU+0440Cyrillicp
аU+0430Cyrillica
уU+0443Cyrillicy
рU+0440Cyrillicp
аU+0430Cyrillica
ӏU+04CFCyrillicl

Six characters, one script, no mixing anywhere — and the word is drawn exactly like paypal. That is what whole-script confusability means, and it applies to a username or a filename as readily as to a domain.

Where it bites hardest is hostnames, because there a second question follows: which script is the top-level domain in? Measured with disarm — note the suspicious column, which cannot tell the second row from the fourth.

Hostnamesuspicious mixed_scriptwhole_script_confusable folds to
paypal.comfalsefalsefalsepaypal.com
раураӏ.comtruefalsetruepaypal.com
раураӏ.рфtruefalsetruepaypal.pф
яндекс.рфtruefalsefalseяндekc.pф
рaypal.comtruetruefalsepaypal.com

Rows two and four both report suspicious. One is a Cyrillic imitation of a payment provider; the other is Russia's largest search engine. Blocking on that flag alone rejects the second, and ignoring it lets the first through.

whole_script_confusable separates them, because the imitation folds onto a Latin name and the real domain does not. The third row shows why the top-level domain matters as well: the same Cyrillic label under .рф is what a site written in Russian looks like, so the usable policy is a label that folds onto Latin, under a Latin top-level domain — which is what per-label results let you express.

Row five is the easier case and the one browsers already handle: two scripts inside one label, which no ordinary name does.

The same thing in your own code

Each code block has been compiled and verified in CI. Provided under the MIT license to illustrate disarm. disarm on GitHub →

# Tell a spoofed domain from a real one written in another script.
#   pip install disarm
from disarm import is_suspicious_hostname

# paypal spelled entirely in Cyrillic: er, a, u, er, a, palochka. Every character
# agrees with every other, so nothing is mixed-script.
SPOOF = "раураӏ.com"
LEGIT = "яндекс.рф"          # Russia's largest search engine

spoof_flag, spoof = is_suspicious_hostname(SPOOF)
legit_flag, legit = is_suspicious_hostname(LEGIT)

# The blunt signal cannot tell them apart — it fires for both.
assert spoof_flag and legit_flag, "suspicious flags both"
assert not spoof.mixed_script, "the spoof is not mixed-script; every letter is Cyrillic"

# Whole-script confusability is what separates them.
assert spoof.whole_script_confusable, "the spoof folds onto a Latin name"
assert not legit.whole_script_confusable, "the real domain does not"
assert spoof.canonical == "paypal.com", spoof.canonical

print("ok: both flagged suspicious; whole-script confusability separates them")

Found a string this gets wrong? The confusables table grew out of exactly that kind of report. Open an issue with it.

Related tools