# Spam, profanity & injection detection

> **Sentinel v2 (beta):** This page documents Sentinel v2, a beta rewrite that may change before the stable release. It does not apply to Sentinel v1, the current stable version: see the [Sentinel v1 documentation](/docs/sentinel.md).

Sentinel scores submitted text for spam, profanity and disguised wording in 26 languages, and scans every form field for SQL, HTML, email-header, template and NoSQL injection attempts. Both run inside the [classifier](/docs/sentinel/v2/features/classifier/), on widget submissions and on `POST /v2/classifier`, without machine-learning models or network calls.

## How it works

### Which text is checked

The classifier builds one text from the submission and checks it with the text rules, while the injection rules look at each field separately:

| Input | Text rules (`text`) | Injection rules (`injection`) |
| --- | --- | --- |
| `POST /v2/classifier` body `text` | Checked when non-blank | Scanned |
| Form `fields` | Checked when there is no `text`: the values, sorted by field name, joined with blank lines | Every key and every value scanned, even when `text` is given |
| `POST /v2/classifier` body `email` | Not checked | Scanned, untrimmed |
| `POST /v2/verify` | The text of the widget's `fields` | Every key and value of the widget's `fields` |

When building the text from `fields`, Sentinel skips the email address (a value under 120 characters keyed `email` or shaped like an address), whitespace-only values and placeholders (`""`, `1`, `0`, `true`, `false`, `null`, `undefined`). The injection rules skip blank and placeholder values too, but always scan keys, so a key such as `password[$ne]` counts. Without any text, `text` is `null`; without fields, text or email, `injection` is `null`.

On `POST /v2/verify`, fields are only available when the widget sends them. See [Enabling it](#enabling-it).

### The spam and profanity filter

The text rules come from a built-in filter that is deterministic: the same text always gets the same scores. It uses weighted dictionaries and counting heuristics, with no models, no GPU and no outbound requests.

- **Dictionaries.** Spam terms and phrases are weighted and scaled down for long texts, so a long business email needs more evidence than a short message. Profanity terms are graded by severity (mild words, strong swearing, slurs and sexually explicit terms), and each distinct term counts once however often it repeats.
- **Heuristics.** Counts of URLs, link markup, money amounts, `!!` runs, repeated letters, emojis, gibberish words and all-caps text. Most counts are scaled down for long texts. Deliberate evasion (invisible characters, mixed alphabets, disguised terms) counts fully at any length.
- **De-obfuscation.** Dictionary matching sees through common disguises before it gives up on a word:

| Disguise | Example |
| --- | --- |
| Leet speak | `fr33`, `v1agra`, `sh!t` |
| Separators or spaces | `f.u.c.k`, `f u c k` |
| Repeated letters | `fuuuck` |
| `*` masking | `sh*t` |
| Cyrillic or Greek look-alike letters | `саsh` with a Cyrillic `с` |
| Invisible characters and full-width letters | `fr​ee` with a zero-width space, `ＦＲＥＥ` |
| Arabic vowel marks and tatweel | `شرمـــوطة` |

A term found only through a disguise also triggers `OBFUSCATION`, because a disguised spam word is a stronger signal than a plain one. Phrases match with or without accents, and in most languages inflected forms match through a stemmer.

### Languages

The dictionaries cover 26 languages:

| | | | | | |
| --- | --- | --- | --- | --- | --- |
| `ar` Arabic | `bg` Bulgarian | `cs` Czech | `da` Danish | `de` German | `el` Greek |
| `en` English | `es` Spanish | `fi` Finnish | `fr` French | `hu` Hungarian | `id` Indonesian |
| `it` Italian | `ko` Korean | `nl` Dutch | `no` Norwegian | `pl` Polish | `pt` Portuguese |
| `ro` Romanian | `ru` Russian | `sh` Serbo-Croatian | `sk` Slovak | `sv` Swedish | `tr` Turkish |
| `uk` Ukrainian | `zh` Chinese | | | | |

A language-neutral list (international brand and drug names such as `viagra`, `casino`, and strong English swearing used everywhere) always applies, so English spam inside a German message is still caught. The Chinese list has spam terms only.

Matching every language at once would turn ordinary words of one language into false hits in another, so the filter uses only the dictionaries of the *candidate languages*:

1. The body's `languages` on `POST /v2/classifier` (up to 50 codes, such as `["de-AT"]`), when given.
2. Otherwise, the language detected from the first 400 characters of the text.
3. Otherwise (no letters, or a tie), the `Accept-Language` languages: of the body's `headers` on `/v2/classifier`, of the request on `/v2/verify`.

Only the part before `-` or `_` counts; `nb` and `nn` map to `no`, and `sr`, `hr`, `bs` and `cnr` to `sh`. Codes without a dictionary are ignored, so a text detected as Japanese, Hindi, Thai or Vietnamese gets only the language-neutral list and the heuristics. Without candidates, or with `*`, every dictionary applies. `text.language` reports the detected language; it is `null` when you pass `languages`, when every text rule is disabled, or when nothing is detected. The same detector answers `POST /v2/language`; see [Lookups & inspect](/docs/sentinel/v2/features/lookups/).

### Injection rules

Each of the five injection rules checks every scanned value and key, and one match is enough to trigger it. Matches are listed in `injection.matches`, in scan order (fields by name, then text, then email), at most one per rule and value and at most 5 per rule. Each has the `rule`, the pattern (`kind`), where it was found (`source`: `FIELD`, `TEXT` or `EMAIL`), the `field` name (`null` for `TEXT` and `EMAIL`), and a `snippet` of up to 64 characters.

`EMAIL_HEADER_INJECTION` depends on the field:

- In the body's `email`, a field whose name contains `mail`, or a value that looks like an address, any line break counts.
- In single-line fields (named `subject`, `from`, `to`, `cc`, `bcc`, `replyTo` or `sender`, or ending in `name` or `subject`, ignoring case and punctuation), a line break followed by a mail header such as `Bcc:` counts.
- Other fields, such as a message body, aren't checked, because letters legitimately contain `To:` lines.

### Effect on the score

The text rules' sum is the `text` category score, and the injection rules' sum the `injection` score. Any triggered injection rule, and any URL on the phishing list (`URL_PHISHING`), raise the overall score to at least `2`, so the submission is `BAD` however clean the other categories are. A clean injection scan doesn't count toward the mean. See [From rules to a verdict](/docs/sentinel/v2/features/classifier/#from-rules-to-a-verdict).

<Callout type="note" title="Changed in v2">

`SQL_INJECTION` and `HTML_INJECTION` moved from the text rules into the `injection` category, which also adds `EMAIL_HEADER_INJECTION`, `TEMPLATE_INJECTION` and `NOSQL_INJECTION` and scans each field's key and value. The text rules now come from the built-in spam and profanity filter. See [Breaking changes: Classifier](/docs/sentinel/v2/upgrade/breaking-changes/#classifier).

## Enabling it

The text and injection rules run whenever there is text or there are fields to check; there is no server switch. The only related server setting is the phishing list behind `URL_PHISHING`, `[classifier.phishing]`, which is on by default. See [Classifier lists](/docs/sentinel/v2/reference/configuration/#classifier-lists) and [Phishing detection](/docs/sentinel/v2/features/phishing/).

**From your backend**, send `text` or `fields` to `POST /v2/classifier` (see [Using the API](#using-the-api)).

**From the widget**, have it send the form's fields to `POST /v2/verify`, either with `fields=1` on the challenge URL or with a security-group rule, which takes precedence:

```json
{
"name": "preset:forms:classifier",
"action": "set",
"set": [{ "field": "classifier.fields", "value": "true" }]
}
```

This is the rule the **Form fields** template writes. Sentinel accepts up to 200 fields (names up to 200 characters, values up to 16000), and signs the text and injection results into `verificationData` as `text.score`, `text.triggeredRules`, `injection.score` and `injection.triggeredRules`. See [Widget integration](/docs/sentinel/v2/integrate/widget/) and [Verifying submissions](/docs/sentinel/v2/integrate/verification/).

To turn individual rules off, list them in `classifier.disableRules` or the request's `disableRules`. See [Choosing rules](/docs/sentinel/v2/features/classifier/#choosing-rules).

## Rules & signals

Scores and exact triggers are in [Classifier rules & scores](/docs/sentinel/v2/reference/classifier-rules/#text).

### Text rules

The 13 filter signals, each a score from the filter, plus `URL_PHISHING`:

| Rule | Meaning |
| --- | --- |
| `SPAM_WORDS` | Spam dictionary terms and phrases |
| `PROFANITY` | Profanity dictionary terms, graded by severity |
| `URL` | URLs |
| `LINK_MARKUP` | Link markup: `href=`, `[url`, `[link` |
| `CURRENCY` | Currency symbols next to a digit, such as `$5` or `5 €` |
| `EXCLAMATION` | Runs of two or more `!` |
| `REPEATED_CHARS` | The same letter four or more times in a row |
| `INVISIBLE_CHARS` | Zero-width or other invisible characters between letters or digits |
| `MIXED_SCRIPT` | Words that mix Latin with Cyrillic or Greek letters |
| `RANDOM_CHARS` | Words with erratic letter case, or random letter-digit codes |
| `EMOJI` | Emojis |
| `OBFUSCATION` | Dictionary terms found only through a disguise |
| `CAPITALIZATION` | Mostly capital letters |
| `URL_PHISHING` | A URL in the text is on the phishing list; forces `BAD` |

`text.terms` lists each dictionary hit once per rule: the dictionary `term`, its `rule` (`SPAM_WORDS` or `PROFANITY`), how it was found (`kind`: `EXACT`, `PHRASE`, `INFLECTED`, `LEET`, `SEPARATED`, `REPEATED`, `MASKED` or `HOMOGLYPH`), and its `language` (`null` for the language-neutral list). `text.phishing` lists the phishing URLs found.

### Injection rules

| Rule | Meaning (`kind` values) |
| --- | --- |
| `SQL_INJECTION` | SQL that breaks out of a value: `COMMENT_TRUNCATION`, `TAUTOLOGY`, `UNION_SELECT`, `STACKED_QUERY`, `SQL_FUNCTION` |
| `HTML_INJECTION` | Markup that runs code: `DANGEROUS_TAG`, `EVENT_HANDLER`, `SCRIPT_URL` |
| `EMAIL_HEADER_INJECTION` | Line breaks that could add mail headers: `LINE_BREAK`, `HEADER_LINE` |
| `TEMPLATE_INJECTION` | Template-engine or JNDI expressions: `JNDI_LOOKUP`, `EXPRESSION_PROBE`, `TEMPLATE_CODE` |
| `NOSQL_INJECTION` | MongoDB operators or JavaScript in a query value: `OPERATOR_KEY`, `OPERATOR_OBJECT`, `JS_INJECTION` |

## Using the API

Send a form's fields to `POST /v2/classifier` with a `restricted` or `full` key. This submission hides spam behind leet speak and has an SQL tautology in the name field:

```bash
curl -X POST 'https://sentinel.example.com/v2/classifier' \
-u 'key_…:sec_…' \
-H 'Content-Type: application/json' \
-d '{
"fields": {
  "email": "deals@example.com",
  "name": "x'\'' OR '\''1'\''='\''1",
  "message": "Make $$$ from home!!! Visit https://cheap-pills.example and get fr33 v1agra today."
},
"disableRules": ["MX", "DMARC"]
}'
```

```json
{
"data": {
"classification": "BAD",
"score": 5.19,
"triggeredRules": ["SPAM_WORDS", "SQL_INJECTION", "OBFUSCATION", "URL", "EXCLAMATION"],
"device": null,
"email": { "score": 0.0, "triggeredRules": [], … },
"ip": null,
"location": null,
"rateLimit": null,
"text": {
  "language": "en",
  "terms": [
    { "term": "$$$", "rule": "SPAM_WORDS", "kind": "EXACT", "language": null },
    { "term": "cheap", "rule": "SPAM_WORDS", "kind": "EXACT", "language": "en" },
    { "term": "pills", "rule": "SPAM_WORDS", "kind": "EXACT", "language": "en" },
    { "term": "free", "rule": "SPAM_WORDS", "kind": "LEET", "language": "en" },
    { "term": "viagra", "rule": "SPAM_WORDS", "kind": "LEET", "language": null }
  ],
  "phishing": [],
  "rules": {
    "SPAM_WORDS": { "score": 5.0 },
    "PROFANITY": { "score": 0.0 },
    "URL": { "score": 1.3 },
    "URL_PHISHING": { "score": 0.0 },
    "EXCLAMATION": { "score": 1.11 },
    "OBFUSCATION": { "score": 3.16 },
    …
  },
  "score": 10.57,
  "time": 21.979,
  "triggeredRules": ["SPAM_WORDS", "OBFUSCATION", "URL", "EXCLAMATION"]
},
"injection": {
  "matches": [
    { "rule": "SQL_INJECTION", "kind": "TAUTOLOGY", "source": "FIELD", "field": "name", "snippet": "' OR '1'='1" }
  ],
  "rules": {
    "SQL_INJECTION": { "score": 5.0 },
    "HTML_INJECTION": { "score": 0.0 },
    "EMAIL_HEADER_INJECTION": { "score": 0.0 },
    "TEMPLATE_INJECTION": { "score": 0.0 },
    "NOSQL_INJECTION": { "score": 0.0 }
  },
  "score": 5.0,
  "time": 0.111,
  "triggeredRules": ["SQL_INJECTION"]
},
"similarity": { … },
"his": null,
"ai": null,
"timeZone": null
}
}
```

`$$$` and `viagra` come from the language-neutral list; `fr33` and `v1agra` were found through leet speak, which also triggers `OBFUSCATION`. `CURRENCY` didn't trigger because `$$$` isn't next to a digit. The email address was taken out of the text, while the name, with its SQL, is part of it. The injection match alone makes the submission `BAD`.

To check a single text, send `text` instead of `fields`, and add `languages` when you know the language: `{"text": "…", "languages": ["de"]}`. See the [`POST /v2/classifier` API reference](/sentinel-api-docs-v2.html#tag/classifier/POST/v2/classifier).

## Tuning

- **Read the evidence first.** `text.terms` shows which dictionary terms matched and how; `injection.matches` shows the field and snippet. In `verificationData` and request logs you get the rule names (`reasons`, `triggeredRules`).
- **Disable rules that fit your form's normal content.** A support form for a shop sees prices (`CURRENCY`), a chat sees emojis (`EMOJI`), and a developer forum sees pasted code (`HTML_INJECTION`, `SQL_INJECTION`, `TEMPLATE_INJECTION`). Disable them for that form's security group, or per request.
- **Pass `languages` when you know them.** Detection is less reliable on short texts and between close languages. Passing the form's language gives better dictionary coverage than detection, and passing several only adds more chances of a false hit.
- **Add meaning-based checks for what dictionaries miss.** Compare text with your own spam and ham examples using [Similarity](/docs/sentinel/v2/features/similarity/), or ask a model with the [AI decider](/docs/sentinel/v2/features/ai-decider/).

## Limitations

- The filter favors precision over recall: it is tuned to keep false positives on business mail low, so it misses spam that uses ordinary wording. Combine it with similarity matching or the AI decider for that.
- Dictionaries are fixed per release; you can't add your own terms. For custom wording, use [training data](/docs/sentinel/v2/features/similarity/).
- Only the single most likely detected language is used. A text that mixes two languages gets the other language's terms only from the language-neutral list, unless you pass both in `languages`.
- Texts in languages without a dictionary get only the language-neutral list and the heuristics. Chinese has no profanity list.
- `text` is limited to 16000 characters and each field value to 16000.
- Injection rules are a signal that a client is probing your forms, not protection. Keep using parameterized queries, output escaping and safe mail libraries in your application.
- `POST /v2/eml` runs the text rules on the message, but no injection rules. See [Email analysis](/docs/sentinel/v2/features/email-analysis/).

## Related

- [Classifier](/docs/sentinel/v2/features/classifier/)
- [Classifier rules & scores](/docs/sentinel/v2/reference/classifier-rules/)
- [Similarity](/docs/sentinel/v2/features/similarity/)
- [AI decider](/docs/sentinel/v2/features/ai-decider/)
- [Phishing detection](/docs/sentinel/v2/features/phishing/)
- [Widget integration](/docs/sentinel/v2/integrate/widget/)
- [Presets: Form fields](/docs/sentinel/v2/policies/presets/#form-fields)
