Spam, profanity & injection detection
Sentinel scores submitted text for spam, profanity and disguised wording in 26 languages, and scans every form field for SQL, HTML, email-header, template and NoSQL injection attempts. Both run inside the classifier, on widget submissions and on POST /v2/classifier, without machine-learning models or network calls.
How it works
Which text is checked
The classifier builds one text from the submission and checks it with the text rules, while the injection rules look at each field separately:
| Input | Text rules (text) | Injection rules (injection) |
|---|---|---|
POST /v2/classifier body text | Checked when non-blank | Scanned |
Form fields | Checked when there is no text: the values, sorted by field name, joined with blank lines | Every key and every value scanned, even when text is given |
POST /v2/classifier body email | Not checked | Scanned, untrimmed |
POST /v2/verify | The text of the widget's fields | Every key and value of the widget's fields |
When building the text from fields, Sentinel skips the email address (a value under 120 characters keyed email or shaped like an address), whitespace-only values and placeholders ("", 1, 0, true, false, null, undefined). The injection rules skip blank and placeholder values too, but always scan keys, so a key such as password[$ne] counts. Without any text, text is null; without fields, text or email, injection is null.
On POST /v2/verify, fields are only available when the widget sends them. See Enabling it.
The spam and profanity filter
The text rules come from a built-in filter that is deterministic: the same text always gets the same scores. It uses weighted dictionaries and counting heuristics, with no models, no GPU and no outbound requests.
- Dictionaries. Spam terms and phrases are weighted and scaled down for long texts, so a long business email needs more evidence than a short message. Profanity terms are graded by severity (mild words, strong swearing, slurs and sexually explicit terms), and each distinct term counts once however often it repeats.
- Heuristics. Counts of URLs, link markup, money amounts,
!!runs, repeated letters, emojis, gibberish words and all-caps text. Most counts are scaled down for long texts. Deliberate evasion (invisible characters, mixed alphabets, disguised terms) counts fully at any length. - De-obfuscation. Dictionary matching sees through common disguises before it gives up on a word:
| Disguise | Example |
|---|---|
| Leet speak | fr33, v1agra, sh!t |
| Separators or spaces | f.u.c.k, f u c k |
| Repeated letters | fuuuck |
* masking | sh*t |
| Cyrillic or Greek look-alike letters | саsh with a Cyrillic с |
| Invisible characters and full-width letters | free with a zero-width space, FREE |
| Arabic vowel marks and tatweel | شرمـــوطة |
A term found only through a disguise also triggers OBFUSCATION, because a disguised spam word is a stronger signal than a plain one. Phrases match with or without accents, and in most languages inflected forms match through a stemmer.
Languages
The dictionaries cover 26 languages:
ar Arabic | bg Bulgarian | cs Czech | da Danish | de German | el Greek |
en English | es Spanish | fi Finnish | fr French | hu Hungarian | id Indonesian |
it Italian | ko Korean | nl Dutch | no Norwegian | pl Polish | pt Portuguese |
ro Romanian | ru Russian | sh Serbo-Croatian | sk Slovak | sv Swedish | tr Turkish |
uk Ukrainian | zh Chinese |
A language-neutral list (international brand and drug names such as viagra, casino, and strong English swearing used everywhere) always applies, so English spam inside a German message is still caught. The Chinese list has spam terms only.
Matching every language at once would turn ordinary words of one language into false hits in another, so the filter uses only the dictionaries of the candidate languages:
- The body's
languagesonPOST /v2/classifier(up to 50 codes, such as["de-AT"]), when given. - Otherwise, the language detected from the first 400 characters of the text.
- Otherwise (no letters, or a tie), the
Accept-Languagelanguages: of the body'sheaderson/v2/classifier, of the request on/v2/verify.
Only the part before - or _ counts; nb and nn map to no, and sr, hr, bs and cnr to sh. Codes without a dictionary are ignored, so a text detected as Japanese, Hindi, Thai or Vietnamese gets only the language-neutral list and the heuristics. Without candidates, or with *, every dictionary applies. text.language reports the detected language; it is null when you pass languages, when every text rule is disabled, or when nothing is detected. The same detector answers POST /v2/language; see Lookups & inspect.
Injection rules
Each of the five injection rules checks every scanned value and key, and one match is enough to trigger it. Matches are listed in injection.matches, in scan order (fields by name, then text, then email), at most one per rule and value and at most 5 per rule. Each has the rule, the pattern (kind), where it was found (source: FIELD, TEXT or EMAIL), the field name (null for TEXT and EMAIL), and a snippet of up to 64 characters.
EMAIL_HEADER_INJECTION depends on the field:
- In the body's
email, a field whose name containsmail, or a value that looks like an address, any line break counts. - In single-line fields (named
subject,from,to,cc,bcc,replyToorsender, or ending innameorsubject, ignoring case and punctuation), a line break followed by a mail header such asBcc:counts. - Other fields, such as a message body, aren't checked, because letters legitimately contain
To:lines.
Effect on the score
The text rules' sum is the text category score, and the injection rules' sum the injection score. Any triggered injection rule, and any URL on the phishing list (URL_PHISHING), raise the overall score to at least 2, so the submission is BAD however clean the other categories are. A clean injection scan doesn't count toward the mean. See From rules to a verdict.
Enabling it
The text and injection rules run whenever there is text or there are fields to check; there is no server switch. The only related server setting is the phishing list behind URL_PHISHING, [classifier.phishing], which is on by default. See Classifier lists and Phishing detection.
From your backend, send text or fields to POST /v2/classifier (see Using the API).
From the widget, have it send the form's fields to POST /v2/verify, either with fields=1 on the challenge URL or with a security-group rule, which takes precedence:
{
"name": "preset:forms:classifier",
"action": "set",
"set": [{ "field": "classifier.fields", "value": "true" }]
}This is the rule the Form fields template writes. Sentinel accepts up to 200 fields (names up to 200 characters, values up to 16000), and signs the text and injection results into verificationData as text.score, text.triggeredRules, injection.score and injection.triggeredRules. See Widget integration and Verifying submissions.
To turn individual rules off, list them in classifier.disableRules or the request's disableRules. See Choosing rules.
Rules & signals
Scores and exact triggers are in Classifier rules & scores.
Text rules
The 13 filter signals, each a score from the filter, plus URL_PHISHING:
| Rule | Meaning |
|---|---|
SPAM_WORDS | Spam dictionary terms and phrases |
PROFANITY | Profanity dictionary terms, graded by severity |
URL | URLs |
LINK_MARKUP | Link markup: href=, [url, [link |
CURRENCY | Currency symbols next to a digit, such as $5 or 5 € |
EXCLAMATION | Runs of two or more ! |
REPEATED_CHARS | The same letter four or more times in a row |
INVISIBLE_CHARS | Zero-width or other invisible characters between letters or digits |
MIXED_SCRIPT | Words that mix Latin with Cyrillic or Greek letters |
RANDOM_CHARS | Words with erratic letter case, or random letter-digit codes |
EMOJI | Emojis |
OBFUSCATION | Dictionary terms found only through a disguise |
CAPITALIZATION | Mostly capital letters |
URL_PHISHING | A URL in the text is on the phishing list; forces BAD |
text.terms lists each dictionary hit once per rule: the dictionary term, its rule (SPAM_WORDS or PROFANITY), how it was found (kind: EXACT, PHRASE, INFLECTED, LEET, SEPARATED, REPEATED, MASKED or HOMOGLYPH), and its language (null for the language-neutral list). text.phishing lists the phishing URLs found.
Injection rules
| Rule | Meaning (kind values) |
|---|---|
SQL_INJECTION | SQL that breaks out of a value: COMMENT_TRUNCATION, TAUTOLOGY, UNION_SELECT, STACKED_QUERY, SQL_FUNCTION |
HTML_INJECTION | Markup that runs code: DANGEROUS_TAG, EVENT_HANDLER, SCRIPT_URL |
EMAIL_HEADER_INJECTION | Line breaks that could add mail headers: LINE_BREAK, HEADER_LINE |
TEMPLATE_INJECTION | Template-engine or JNDI expressions: JNDI_LOOKUP, EXPRESSION_PROBE, TEMPLATE_CODE |
NOSQL_INJECTION | MongoDB operators or JavaScript in a query value: OPERATOR_KEY, OPERATOR_OBJECT, JS_INJECTION |
Using the API
Send a form's fields to POST /v2/classifier with a restricted or full key. This submission hides spam behind leet speak and has an SQL tautology in the name field:
curl -X POST 'https://sentinel.example.com/v2/classifier' \
-u 'key_…:sec_…' \
-H 'Content-Type: application/json' \
-d '{
"fields": {
"email": "deals@example.com",
"name": "x'\'' OR '\''1'\''='\''1",
"message": "Make $$$ from home!!! Visit https://cheap-pills.example and get fr33 v1agra today."
},
"disableRules": ["MX", "DMARC"]
}'{
"data": {
"classification": "BAD",
"score": 5.19,
"triggeredRules": ["SPAM_WORDS", "SQL_INJECTION", "OBFUSCATION", "URL", "EXCLAMATION"],
"device": null,
"email": { "score": 0.0, "triggeredRules": [], … },
"ip": null,
"location": null,
"rateLimit": null,
"text": {
"language": "en",
"terms": [
{ "term": "$$$", "rule": "SPAM_WORDS", "kind": "EXACT", "language": null },
{ "term": "cheap", "rule": "SPAM_WORDS", "kind": "EXACT", "language": "en" },
{ "term": "pills", "rule": "SPAM_WORDS", "kind": "EXACT", "language": "en" },
{ "term": "free", "rule": "SPAM_WORDS", "kind": "LEET", "language": "en" },
{ "term": "viagra", "rule": "SPAM_WORDS", "kind": "LEET", "language": null }
],
"phishing": [],
"rules": {
"SPAM_WORDS": { "score": 5.0 },
"PROFANITY": { "score": 0.0 },
"URL": { "score": 1.3 },
"URL_PHISHING": { "score": 0.0 },
"EXCLAMATION": { "score": 1.11 },
"OBFUSCATION": { "score": 3.16 },
…
},
"score": 10.57,
"time": 21.979,
"triggeredRules": ["SPAM_WORDS", "OBFUSCATION", "URL", "EXCLAMATION"]
},
"injection": {
"matches": [
{ "rule": "SQL_INJECTION", "kind": "TAUTOLOGY", "source": "FIELD", "field": "name", "snippet": "' OR '1'='1" }
],
"rules": {
"SQL_INJECTION": { "score": 5.0 },
"HTML_INJECTION": { "score": 0.0 },
"EMAIL_HEADER_INJECTION": { "score": 0.0 },
"TEMPLATE_INJECTION": { "score": 0.0 },
"NOSQL_INJECTION": { "score": 0.0 }
},
"score": 5.0,
"time": 0.111,
"triggeredRules": ["SQL_INJECTION"]
},
"similarity": { … },
"his": null,
"ai": null,
"timeZone": null
}
}$$$ and viagra come from the language-neutral list; fr33 and v1agra were found through leet speak, which also triggers OBFUSCATION. CURRENCY didn't trigger because $$$ isn't next to a digit. The email address was taken out of the text, while the name, with its SQL, is part of it. The injection match alone makes the submission BAD.
To check a single text, send text instead of fields, and add languages when you know the language: {"text": "…", "languages": ["de"]}. See the POST /v2/classifier API reference.
Tuning
- Read the evidence first.
text.termsshows which dictionary terms matched and how;injection.matchesshows the field and snippet. InverificationDataand request logs you get the rule names (reasons,triggeredRules). - Disable rules that fit your form's normal content. A support form for a shop sees prices (
CURRENCY), a chat sees emojis (EMOJI), and a developer forum sees pasted code (HTML_INJECTION,SQL_INJECTION,TEMPLATE_INJECTION). Disable them for that form's security group, or per request. - Pass
languageswhen you know them. Detection is less reliable on short texts and between close languages. Passing the form's language gives better dictionary coverage than detection, and passing several only adds more chances of a false hit. - Add meaning-based checks for what dictionaries miss. Compare text with your own spam and ham examples using Similarity, or ask a model with the AI decider.
Limitations
- The filter favors precision over recall: it is tuned to keep false positives on business mail low, so it misses spam that uses ordinary wording. Combine it with similarity matching or the AI decider for that.
- Dictionaries are fixed per release; you can't add your own terms. For custom wording, use training data.
- Only the single most likely detected language is used. A text that mixes two languages gets the other language's terms only from the language-neutral list, unless you pass both in
languages. - Texts in languages without a dictionary get only the language-neutral list and the heuristics. Chinese has no profanity list.
textis limited to 16000 characters and each field value to 16000.- Injection rules are a signal that a client is probing your forms, not protection. Keep using parameterized queries, output escaping and safe mail libraries in your application.
POST /v2/emlruns the text rules on the message, but no injection rules. See Email analysis.