Similarity and Diff

2 endpoints for pairwise text comparison and line-level diffing.

Method Endpoint Purpose
POST /v1/text/similarity Compute similarity with Levenshtein, cosine, or Jaccard
POST /v1/text/diff Return unified and structured text differences

REST API Examples

Compare text similarity

curl -X POST "https://textanalysis.toolkitapi.io/v1/text/similarity" \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"a": "The cat sat on the mat", "b": "The cat sat on a rug", "method": "cosine"}'
const resp = await fetch("https://textanalysis.toolkitapi.io/v1/text/similarity", {
  method: "POST",
  headers: { "X-API-Key": "YOUR_KEY", "Content-Type": "application/json" },
  body: JSON.stringify({ a: "Hello world", b: "Hello earth", method: "cosine" }),
});
const data = await resp.json();
console.log(`Similarity: ${data.similarity}`);

Python SDK Examples

Similarity by method

from toolkitapi import TextAnalysis

a = "The release includes search improvements and bug fixes."
b = "This release adds better search and fixes several bugs."

with TextAnalysis(api_key="tk_...") as ta:
    for method in ["levenshtein", "cosine", "jaccard"]:
        result = ta.similarity(a=a, b=b, method=method)
        print(method, result["similarity"])

Unified diff

from toolkitapi import TextAnalysis

original = "line one\nline two\nline three\n"
modified = "line one\nline 2\nline three\nline four\n"

with TextAnalysis(api_key="tk_...") as ta:
    diff = ta.diff(a=original, b=modified, context_lines=2)

print(diff["is_identical"])
print(diff["statistics"])
print(diff["unified_diff"])

Request Parameters

POST /v1/text/similarity

Parameter Type Description
a string First text, max 1048576 characters
b string Second text, max 1048576 characters
method string levenshtein (default), cosine, jaccard, or semantic (GPU-backed; returns 500 if the worker is unavailable)

POST /v1/text/diff

Parameter Type Description
a string Original text, max 1048576 characters
b string Modified text, max 1048576 characters
context_lines integer Context lines in unified diff, 0 to 50

Response Fields

Similarity

Field Type Description
method string Algorithm used
similarity number Similarity score in range 0 to 1
distance integer Levenshtein distance (Levenshtein only)
max_length integer Max input length (Levenshtein only)

Diff

Field Type Description
unified_diff string Unified diff text
changes array Structured operations (insert, delete, replace)
statistics.additions integer Added line count
statistics.deletions integer Deleted line count
statistics.unchanged integer Unchanged line count
is_identical boolean True when no changes detected

Tip

Use cosine similarity for semantic overlap in longer prose, and Levenshtein when you need edit-distance sensitivity for short strings such as titles or identifiers.

Response Shape Varies by Method

POST /v1/text/similarity does not return a fixed object shape — the fields depend on method. Code that reads distance unconditionally will KeyError on three of the four methods.

method Fields returned
levenshtein method, distance, similarity, max_length
cosine method, similarity
jaccard method, similarity
semantic method, similarity

Always read similarity — it is present for every method — and reach for distance only after checking the method:

result = ta.similarity(a=a, b=b, method="levenshtein")
score = result["similarity"]                       # always present, 0..1
edits = result.get("distance")                     # only for levenshtein

How each score is computed

  • levenshtein — character-level edit distance, then normalised as 1 - distance / max(len(a), len(b)), rounded to 6 decimals. max_length is that denominator, returned so you can recover the raw distance. A score of 1.0 means identical.
  • cosine — cosine similarity over token frequency vectors, rounded to 6 decimals. Word order is ignored, so "cat sat on mat" and "mat on sat cat" score 1.0.
  • jaccard — size of the token intersection divided by the token union, rounded to 6 decimals. Harsher than cosine on long texts because it ignores counts, only presence.
  • semantic — embeds both strings with all-MiniLM-L6-v2 and compares vectors. This is the only method that requires the GPU worker; if the worker is unreachable the request fails with 500 and no partial result is returned.

Edge cases

Situation Result
a and b both empty levenshtein → similarity: 1.0, distance: 0, max_length: 0
One string empty similarity: 0.0 for all methods
Identical inputs, any method similarity: 1.0
No shared tokens (cosine/jaccard) 0.0
Input over 1,048,576 characters 422 — max_length is enforced per field, not on the pair

Choosing a method

  • Short strings with typos (titles, slugs, product names, identifiers) → levenshtein. It is the only method sensitive to character order and edit count.
  • Long prose, relevance ranking, near-duplicate detection → cosine.
  • Set overlap / tag comparison → jaccard.
  • Paraphrase detection where wording differs → semantic. It costs a GPU round-trip, so batch it rather than calling it in a tight loop.

All four are symmetric: similarity(a, b) == similarity(b, a).