Skip to content

Multilingual Support

Seven languages beyond English: Spanish, French, Russian, Arabic (verified confidence) and Igbo, Yoruba, Hausa (experimental confidence). Pass language on a request (nhc classify TEXT --language es, or "language": "es" in the API body) — an unrecognized code falls back to English with an explicit note in the rationale rather than silently misclassifying.

What "verified" vs "experimental" actually means

Spanish, French, Russian, and Arabic get the same detection rigor as English — identity anchors, harm patterns, and the negation/counter-speech/benign-context suppression cues are all populated. Igbo, Yoruba, and Hausa are deliberately smaller seed vocabularies (a handful of core identity and harm terms, no negation/counter-speech heuristics yet), because the training-data depth for those three languages doesn't match the others, and none of the eight languages here — including the "verified" ones — have been reviewed by a native-speaker domain expert. language_confidence on every response makes this distinction visible in the data, not just in documentation.

Where the vocabulary lives

Each language's vocabulary lives in its own file under narrative_harm_classifier/data/patterns/ (en.yaml, es.yaml, fr.yaml, ru.yaml, ar.yaml, ig.yaml, yo.yaml, ha.yaml) rather than in code, so adding or correcting a language doesn't require touching engine.py — see Contributing for the format. A smaller, separate smoke-test suite (nhc benchmark i18n, data/i18n_smoke_tests.yaml) checks basic detection works per language — it is not a replication of the English 192-case benchmark; building adversarial negation/counter-speech/ obfuscation test cases with confidence requires a fluency that isn't equally available for all eight languages.

A real Arabic-specific bug found while building this

A plain \b(word)\b regex never matches a prefixed or suffixed Arabic word, because the definite article (ال) and plural suffixes attach directly with no space (مسلمالمسلمون) — extremely common in ordinary text, not an edge case. Every Arabic pattern explicitly allows for the attached article and common plural forms now; see the comment at the top of ar.yaml if you're extending it.