Similarity and Diff¶
2 endpoints for pairwise text comparison and line-level diffing.
| Method |
Endpoint |
Purpose |
POST |
/v1/text/similarity |
Compute similarity with Levenshtein, cosine, or Jaccard |
POST |
/v1/text/diff |
Return unified and structured text differences |
REST API Examples¶
Compare text similarity¶
curl -X POST "https://textanalysis.toolkitapi.io/v1/text/similarity" \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{"a": "The cat sat on the mat", "b": "The cat sat on a rug", "method": "cosine"}'
const resp = await fetch("https://textanalysis.toolkitapi.io/v1/text/similarity", {
method: "POST",
headers: { "X-API-Key": "YOUR_KEY", "Content-Type": "application/json" },
body: JSON.stringify({ a: "Hello world", b: "Hello earth", method: "cosine" }),
});
const data = await resp.json();
console.log(`Similarity: ${data.similarity}`);
Python SDK Examples¶
Similarity by method¶
from toolkitapi import TextAnalysis
a = "The release includes search improvements and bug fixes."
b = "This release adds better search and fixes several bugs."
with TextAnalysis(api_key="tk_...") as ta:
for method in ["levenshtein", "cosine", "jaccard"]:
result = ta.similarity(a=a, b=b, method=method)
print(method, result["similarity"])
Unified diff¶
from toolkitapi import TextAnalysis
original = "line one\nline two\nline three\n"
modified = "line one\nline 2\nline three\nline four\n"
with TextAnalysis(api_key="tk_...") as ta:
diff = ta.diff(a=original, b=modified, context_lines=2)
print(diff["is_identical"])
print(diff["statistics"])
print(diff["unified_diff"])
Request Parameters¶
POST /v1/text/similarity¶
| Parameter |
Type |
Description |
a |
string |
First text, max 1048576 characters |
b |
string |
Second text, max 1048576 characters |
method |
string |
levenshtein (default), cosine, jaccard, or semantic (GPU-backed; returns 500 if the worker is unavailable) |
POST /v1/text/diff¶
| Parameter |
Type |
Description |
a |
string |
Original text, max 1048576 characters |
b |
string |
Modified text, max 1048576 characters |
context_lines |
integer |
Context lines in unified diff, 0 to 50 |
Response Fields¶
Similarity¶
| Field |
Type |
Description |
method |
string |
Algorithm used |
similarity |
number |
Similarity score in range 0 to 1 |
distance |
integer |
Levenshtein distance (Levenshtein only) |
max_length |
integer |
Max input length (Levenshtein only) |
Diff¶
| Field |
Type |
Description |
unified_diff |
string |
Unified diff text |
changes |
array |
Structured operations (insert, delete, replace) |
statistics.additions |
integer |
Added line count |
statistics.deletions |
integer |
Deleted line count |
statistics.unchanged |
integer |
Unchanged line count |
is_identical |
boolean |
True when no changes detected |
Tip
Use cosine similarity for semantic overlap in longer prose, and Levenshtein when you need edit-distance sensitivity for short strings such as titles or identifiers.
Response Shape Varies by Method¶
POST /v1/text/similarity does not return a fixed object shape — the fields depend on method. Code that reads distance unconditionally will KeyError on three of the four methods.
method |
Fields returned |
levenshtein |
method, distance, similarity, max_length |
cosine |
method, similarity |
jaccard |
method, similarity |
semantic |
method, similarity |
Always read similarity — it is present for every method — and reach for distance only after checking the method:
result = ta.similarity(a=a, b=b, method="levenshtein")
score = result["similarity"] # always present, 0..1
edits = result.get("distance") # only for levenshtein
How each score is computed¶
levenshtein — character-level edit distance, then normalised as 1 - distance / max(len(a), len(b)), rounded to 6 decimals. max_length is that denominator, returned so you can recover the raw distance. A score of 1.0 means identical.
cosine — cosine similarity over token frequency vectors, rounded to 6 decimals. Word order is ignored, so "cat sat on mat" and "mat on sat cat" score 1.0.
jaccard — size of the token intersection divided by the token union, rounded to 6 decimals. Harsher than cosine on long texts because it ignores counts, only presence.
semantic — embeds both strings with all-MiniLM-L6-v2 and compares vectors. This is the only method that requires the GPU worker; if the worker is unreachable the request fails with 500 and no partial result is returned.
Edge cases¶
| Situation |
Result |
a and b both empty |
levenshtein → similarity: 1.0, distance: 0, max_length: 0 |
| One string empty |
similarity: 0.0 for all methods |
| Identical inputs, any method |
similarity: 1.0 |
No shared tokens (cosine/jaccard) |
0.0 |
| Input over 1,048,576 characters |
422 — max_length is enforced per field, not on the pair |
Choosing a method¶
- Short strings with typos (titles, slugs, product names, identifiers) →
levenshtein. It is the only method sensitive to character order and edit count.
- Long prose, relevance ranking, near-duplicate detection →
cosine.
- Set overlap / tag comparison →
jaccard.
- Paraphrase detection where wording differs →
semantic. It costs a GPU round-trip, so batch it rather than calling it in a tight loop.
All four are symmetric: similarity(a, b) == similarity(b, a).