Similarity & Training Data

Similarity & Training Data catches spam, abuse, and banned content that's been reworded to dodge exact-match filtering — comparing submitted text against known example phrases by meaning, not just words. It's a good fit for chat, forums, comments, and other user-generated-content moderation.

How it works

Similarity is computed as cosine similarity between text embeddings, generated by the open-source all-MiniLM-L6-v2 sentence-embedding model.

Input longer than 256 tokens is automatically truncated.
Accuracy is strongest for English; non-English performance varies.
When run through the Classifier, a match raises the overall spam score.

Setup

Finish installing Sentinel.
Create an API Key on a Security Group with Restricted access — this isn't a public endpoint.

Matching against examples directly

Pass example phrases inline, with no group setup required:

{
  "text": "Claim your exclusive prize now by clicking the link below!",
  "examples": [
    "Claim your exclusive reward now by clicking the link below!",
    "Get your exclusive prize now by visiting this link!"
  ]
}

Matching against a predefined group

Create reusable training data via POST /v1/training-data, then reference it by group name instead of repeating examples on every request:

{
  "text": "Your message here",
  "groups": ["chat_spam"],
  "partial": true
}

Response

{
  "matches": {
    "examples": {
      "matches": [{ "example": "phrase1", "score": 0.85 }],
      "time": 24.833
    }
  }
}

Parameters

Parameter	Type	Purpose
`text`	string	The content to evaluate.
`examples`	array	Example phrases to match against directly.
`groups`	array	Predefined training-data set names to match against instead.
`partial`	boolean	Match phrases within longer text, rather than requiring a close full-text match.
`stopOnMatch`	number	Stop processing further examples once a match scores above this threshold (0.0–1.0).
`threshold`	number	Minimum score to count as a match.
`weight`	number	Multiplier applied to this match's contribution to the overall score.

Interpreting scores

As a starting point:

Score	Interpretation
0.7 and above	Likely spam
0.4–0.7	Possibly suspicious
Below 0.4	Usually safe

For partial matches (short phrases, 1–5 words), a lower threshold — around 0.6 — is often more appropriate than the full-text guidance above.

Best practices

Keep examples and input text within the 256-token limit for best accuracy.
Use partial: true for short phrases rather than full sentences.
Review user-submitted training data manually before trusting it — bad examples degrade matching quality.
Tune threshold and stopOnMatch against your own false-positive tolerance rather than using the defaults blindly.

Classifier — similarity can be checked inline as part of a classification request.
Forms — attaching this to submitted fields.
Chat & Forum Moderation — a usage example.
Data Sources