Skip to main content
ALTCHA®

Spam, profanity & injection detection

Sentinel scores submitted text for spam, profanity and disguised wording in 26 languages, and scans every form field for SQL, HTML, email-header, template and NoSQL injection attempts. Both run inside the classifier, on widget submissions and on POST /v2/classifier, without machine-learning models or network calls.

How it works

Which text is checked

The classifier builds one text from the submission and checks it with the text rules, while the injection rules look at each field separately:

InputText rules (text)Injection rules (injection)
POST /v2/classifier body textChecked when non-blankScanned
Form fieldsChecked when there is no text: the values, sorted by field name, joined with blank linesEvery key and every value scanned, even when text is given
POST /v2/classifier body emailNot checkedScanned, untrimmed
POST /v2/verifyThe text of the widget's fieldsEvery key and value of the widget's fields

When building the text from fields, Sentinel skips the email address (a value under 120 characters keyed email or shaped like an address), whitespace-only values and placeholders ("", 1, 0, true, false, null, undefined). The injection rules skip blank and placeholder values too, but always scan keys, so a key such as password[$ne] counts. Without any text, text is null; without fields, text or email, injection is null.

On POST /v2/verify, fields are only available when the widget sends them. See Enabling it.

The spam and profanity filter

The text rules come from a built-in filter that is deterministic: the same text always gets the same scores. It uses weighted dictionaries and counting heuristics, with no models, no GPU and no outbound requests.

  • Dictionaries. Spam terms and phrases are weighted and scaled down for long texts, so a long business email needs more evidence than a short message. Profanity terms are graded by severity (mild words, strong swearing, slurs and sexually explicit terms), and each distinct term counts once however often it repeats.
  • Heuristics. Counts of URLs, link markup, money amounts, !! runs, repeated letters, emojis, gibberish words and all-caps text. Most counts are scaled down for long texts. Deliberate evasion (invisible characters, mixed alphabets, disguised terms) counts fully at any length.
  • De-obfuscation. Dictionary matching sees through common disguises before it gives up on a word:
DisguiseExample
Leet speakfr33, v1agra, sh!t
Separators or spacesf.u.c.k, f u c k
Repeated lettersfuuuck
* maskingsh*t
Cyrillic or Greek look-alike lettersсаsh with a Cyrillic с
Invisible characters and full-width lettersfr​ee with a zero-width space, FREE
Arabic vowel marks and tatweelشرمـــوطة

A term found only through a disguise also triggers OBFUSCATION, because a disguised spam word is a stronger signal than a plain one. Phrases match with or without accents, and in most languages inflected forms match through a stemmer.

Languages

The dictionaries cover 26 languages:

ar Arabicbg Bulgariancs Czechda Danishde Germanel Greek
en Englishes Spanishfi Finnishfr Frenchhu Hungarianid Indonesian
it Italianko Koreannl Dutchno Norwegianpl Polishpt Portuguese
ro Romanianru Russiansh Serbo-Croatiansk Slovaksv Swedishtr Turkish
uk Ukrainianzh Chinese

A language-neutral list (international brand and drug names such as viagra, casino, and strong English swearing used everywhere) always applies, so English spam inside a German message is still caught. The Chinese list has spam terms only.

Matching every language at once would turn ordinary words of one language into false hits in another, so the filter uses only the dictionaries of the candidate languages:

  1. The body's languages on POST /v2/classifier (up to 50 codes, such as ["de-AT"]), when given.
  2. Otherwise, the language detected from the first 400 characters of the text.
  3. Otherwise (no letters, or a tie), the Accept-Language languages: of the body's headers on /v2/classifier, of the request on /v2/verify.

Only the part before - or _ counts; nb and nn map to no, and sr, hr, bs and cnr to sh. Codes without a dictionary are ignored, so a text detected as Japanese, Hindi, Thai or Vietnamese gets only the language-neutral list and the heuristics. Without candidates, or with *, every dictionary applies. text.language reports the detected language; it is null when you pass languages, when every text rule is disabled, or when nothing is detected. The same detector answers POST /v2/language; see Lookups & inspect.

Injection rules

Each of the five injection rules checks every scanned value and key, and one match is enough to trigger it. Matches are listed in injection.matches, in scan order (fields by name, then text, then email), at most one per rule and value and at most 5 per rule. Each has the rule, the pattern (kind), where it was found (source: FIELD, TEXT or EMAIL), the field name (null for TEXT and EMAIL), and a snippet of up to 64 characters.

EMAIL_HEADER_INJECTION depends on the field:

  • In the body's email, a field whose name contains mail, or a value that looks like an address, any line break counts.
  • In single-line fields (named subject, from, to, cc, bcc, replyTo or sender, or ending in name or subject, ignoring case and punctuation), a line break followed by a mail header such as Bcc: counts.
  • Other fields, such as a message body, aren't checked, because letters legitimately contain To: lines.

Effect on the score

The text rules' sum is the text category score, and the injection rules' sum the injection score. Any triggered injection rule, and any URL on the phishing list (URL_PHISHING), raise the overall score to at least 2, so the submission is BAD however clean the other categories are. A clean injection scan doesn't count toward the mean. See From rules to a verdict.

Enabling it

The text and injection rules run whenever there is text or there are fields to check; there is no server switch. The only related server setting is the phishing list behind URL_PHISHING, [classifier.phishing], which is on by default. See Classifier lists and Phishing detection.

From your backend, send text or fields to POST /v2/classifier (see Using the API).

From the widget, have it send the form's fields to POST /v2/verify, either with fields=1 on the challenge URL or with a security-group rule, which takes precedence:

{
  "name": "preset:forms:classifier",
  "action": "set",
  "set": [{ "field": "classifier.fields", "value": "true" }]
}

This is the rule the Form fields template writes. Sentinel accepts up to 200 fields (names up to 200 characters, values up to 16000), and signs the text and injection results into verificationData as text.score, text.triggeredRules, injection.score and injection.triggeredRules. See Widget integration and Verifying submissions.

To turn individual rules off, list them in classifier.disableRules or the request's disableRules. See Choosing rules.

Rules & signals

Scores and exact triggers are in Classifier rules & scores.

Text rules

The 13 filter signals, each a score from the filter, plus URL_PHISHING:

RuleMeaning
SPAM_WORDSSpam dictionary terms and phrases
PROFANITYProfanity dictionary terms, graded by severity
URLURLs
LINK_MARKUPLink markup: href=, [url, [link
CURRENCYCurrency symbols next to a digit, such as $5 or 5 €
EXCLAMATIONRuns of two or more !
REPEATED_CHARSThe same letter four or more times in a row
INVISIBLE_CHARSZero-width or other invisible characters between letters or digits
MIXED_SCRIPTWords that mix Latin with Cyrillic or Greek letters
RANDOM_CHARSWords with erratic letter case, or random letter-digit codes
EMOJIEmojis
OBFUSCATIONDictionary terms found only through a disguise
CAPITALIZATIONMostly capital letters
URL_PHISHINGA URL in the text is on the phishing list; forces BAD

text.terms lists each dictionary hit once per rule: the dictionary term, its rule (SPAM_WORDS or PROFANITY), how it was found (kind: EXACT, PHRASE, INFLECTED, LEET, SEPARATED, REPEATED, MASKED or HOMOGLYPH), and its language (null for the language-neutral list). text.phishing lists the phishing URLs found.

Injection rules

RuleMeaning (kind values)
SQL_INJECTIONSQL that breaks out of a value: COMMENT_TRUNCATION, TAUTOLOGY, UNION_SELECT, STACKED_QUERY, SQL_FUNCTION
HTML_INJECTIONMarkup that runs code: DANGEROUS_TAG, EVENT_HANDLER, SCRIPT_URL
EMAIL_HEADER_INJECTIONLine breaks that could add mail headers: LINE_BREAK, HEADER_LINE
TEMPLATE_INJECTIONTemplate-engine or JNDI expressions: JNDI_LOOKUP, EXPRESSION_PROBE, TEMPLATE_CODE
NOSQL_INJECTIONMongoDB operators or JavaScript in a query value: OPERATOR_KEY, OPERATOR_OBJECT, JS_INJECTION

Using the API

Send a form's fields to POST /v2/classifier with a restricted or full key. This submission hides spam behind leet speak and has an SQL tautology in the name field:

curl -X POST 'https://sentinel.example.com/v2/classifier' \
  -u 'key_…:sec_…' \
  -H 'Content-Type: application/json' \
  -d '{
    "fields": {
      "email": "deals@example.com",
      "name": "x'\'' OR '\''1'\''='\''1",
      "message": "Make $$$ from home!!! Visit https://cheap-pills.example and get fr33 v1agra today."
    },
    "disableRules": ["MX", "DMARC"]
  }'
{
  "data": {
    "classification": "BAD",
    "score": 5.19,
    "triggeredRules": ["SPAM_WORDS", "SQL_INJECTION", "OBFUSCATION", "URL", "EXCLAMATION"],
    "device": null,
    "email": { "score": 0.0, "triggeredRules": [], … },
    "ip": null,
    "location": null,
    "rateLimit": null,
    "text": {
      "language": "en",
      "terms": [
        { "term": "$$$", "rule": "SPAM_WORDS", "kind": "EXACT", "language": null },
        { "term": "cheap", "rule": "SPAM_WORDS", "kind": "EXACT", "language": "en" },
        { "term": "pills", "rule": "SPAM_WORDS", "kind": "EXACT", "language": "en" },
        { "term": "free", "rule": "SPAM_WORDS", "kind": "LEET", "language": "en" },
        { "term": "viagra", "rule": "SPAM_WORDS", "kind": "LEET", "language": null }
      ],
      "phishing": [],
      "rules": {
        "SPAM_WORDS": { "score": 5.0 },
        "PROFANITY": { "score": 0.0 },
        "URL": { "score": 1.3 },
        "URL_PHISHING": { "score": 0.0 },
        "EXCLAMATION": { "score": 1.11 },
        "OBFUSCATION": { "score": 3.16 },
        …
      },
      "score": 10.57,
      "time": 21.979,
      "triggeredRules": ["SPAM_WORDS", "OBFUSCATION", "URL", "EXCLAMATION"]
    },
    "injection": {
      "matches": [
        { "rule": "SQL_INJECTION", "kind": "TAUTOLOGY", "source": "FIELD", "field": "name", "snippet": "' OR '1'='1" }
      ],
      "rules": {
        "SQL_INJECTION": { "score": 5.0 },
        "HTML_INJECTION": { "score": 0.0 },
        "EMAIL_HEADER_INJECTION": { "score": 0.0 },
        "TEMPLATE_INJECTION": { "score": 0.0 },
        "NOSQL_INJECTION": { "score": 0.0 }
      },
      "score": 5.0,
      "time": 0.111,
      "triggeredRules": ["SQL_INJECTION"]
    },
    "similarity": { … },
    "his": null,
    "ai": null,
    "timeZone": null
  }
}

$$$ and viagra come from the language-neutral list; fr33 and v1agra were found through leet speak, which also triggers OBFUSCATION. CURRENCY didn't trigger because $$$ isn't next to a digit. The email address was taken out of the text, while the name, with its SQL, is part of it. The injection match alone makes the submission BAD.

To check a single text, send text instead of fields, and add languages when you know the language: {"text": "…", "languages": ["de"]}. See the POST /v2/classifier API reference.

Tuning

  • Read the evidence first. text.terms shows which dictionary terms matched and how; injection.matches shows the field and snippet. In verificationData and request logs you get the rule names (reasons, triggeredRules).
  • Disable rules that fit your form's normal content. A support form for a shop sees prices (CURRENCY), a chat sees emojis (EMOJI), and a developer forum sees pasted code (HTML_INJECTION, SQL_INJECTION, TEMPLATE_INJECTION). Disable them for that form's security group, or per request.
  • Pass languages when you know them. Detection is less reliable on short texts and between close languages. Passing the form's language gives better dictionary coverage than detection, and passing several only adds more chances of a false hit.
  • Add meaning-based checks for what dictionaries miss. Compare text with your own spam and ham examples using Similarity, or ask a model with the AI decider.

Limitations

  • The filter favors precision over recall: it is tuned to keep false positives on business mail low, so it misses spam that uses ordinary wording. Combine it with similarity matching or the AI decider for that.
  • Dictionaries are fixed per release; you can't add your own terms. For custom wording, use training data.
  • Only the single most likely detected language is used. A text that mixes two languages gets the other language's terms only from the language-neutral list, unless you pass both in languages.
  • Texts in languages without a dictionary get only the language-neutral list and the heuristics. Chinese has no profanity list.
  • text is limited to 16000 characters and each field value to 16000.
  • Injection rules are a signal that a client is probing your forms, not protection. Keep using parameterized queries, output escaping and safe mail libraries in your application.
  • POST /v2/eml runs the text rules on the message, but no injection rules. See Email analysis.

Start typing to search...

↑ ↓ Navigate ↵ Select