# Similarity & Training Data

Similarity & Training Data catches spam, abuse, and banned content that's been reworded to dodge exact-match filtering — comparing submitted text against known example phrases by meaning, not just words. It's a good fit for chat, forums, comments, and other user-generated-content moderation.

## How it works

Similarity is computed as **cosine similarity** between text embeddings, generated by the open-source [`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) sentence-embedding model.

- Input longer than 256 tokens is automatically truncated.
- Accuracy is strongest for English; non-English performance varies.
- When run through the [Classifier](/docs/sentinel/features/classifier/), a match raises the overall spam score.

## Setup

1. Finish installing Sentinel.
2. Create an [API Key](/docs/sentinel/configure/api-keys/) on a Security Group with **Restricted** access — this isn't a public endpoint.

## Matching against examples directly

Pass example phrases inline, with no group setup required:

```json
{
"text": "Claim your exclusive prize now by clicking the link below!",
"examples": [
"Claim your exclusive reward now by clicking the link below!",
"Get your exclusive prize now by visiting this link!"
]
}
```

## Matching against a predefined group

Create reusable training data via `POST /v1/training-data`, then reference it by group name instead of repeating examples on every request:

```json
{
"text": "Your message here",
"groups": ["chat_spam"],
"partial": true
}
```

## Response

```json
{
"matches": {
"examples": {
  "matches": [{ "example": "phrase1", "score": 0.85 }],
  "time": 24.833
}
}
}
```

## Parameters

| Parameter | Type | Purpose |
|---|---|---|
| `text` | string | The content to evaluate. |
| `examples` | array | Example phrases to match against directly. |
| `groups` | array | Predefined training-data set names to match against instead. |
| `partial` | boolean | Match phrases within longer text, rather than requiring a close full-text match. |
| `stopOnMatch` | number | Stop processing further examples once a match scores above this threshold (0.0–1.0). |
| `threshold` | number | Minimum score to count as a match. |
| `weight` | number | Multiplier applied to this match's contribution to the overall score. |

## Interpreting scores

As a starting point:

| Score | Interpretation |
|---|---|
| 0.7 and above | Likely spam |
| 0.4–0.7 | Possibly suspicious |
| Below 0.4 | Usually safe |

For partial matches (short phrases, 1–5 words), a lower threshold — around 0.6 — is often more appropriate than the full-text guidance above.

## Best practices

- Keep examples and input text within the 256-token limit for best accuracy.
- Use `partial: true` for short phrases rather than full sentences.
- Review user-submitted training data manually before trusting it — bad examples degrade matching quality.
- Tune `threshold` and `stopOnMatch` against your own false-positive tolerance rather than using the defaults blindly.

## Related

- [Classifier](/docs/sentinel/features/classifier/#using-the-api) — `similarity` can be checked inline as part of a classification request.
- [Forms](/docs/sentinel/configure/forms/) — attaching this to submitted fields.
- [Chat & Forum Moderation](/docs/sentinel/use-cases/chat-forum-moderation/) — a usage example.
- [Data Sources](/docs/sentinel/integrations/data-sources/)