Two support tickets can describe the same underlying issue with almost no words in common. An exact-word search may miss that “the bill PDF has no text” and “the invoice opens as a blank page” point to the same problem.

To find similar support tickets, encode every archive summary and new message with the same sentence-embedding model, normalize the vectors, and rank archive rows by cosine similarity. Set a minimum score from labeled examples, suggest the best match only when it clears that threshold, and send weaker cases for human review.

This tutorial builds that workflow with eight archived issues and ten incoming messages. It finds related wording rather than assigning a fixed queue. If your goal is to predict categories such as billing or account, the support ticket classification tutorial teaches that different task.

Before Matching: Setup, Data, and Boundaries

You need Python 3.11 or newer and basic knowledge of Python lists and table columns. You do not need previous natural language processing experience. Natural language processing, or NLP, means using software to work with human language. A pandas DataFrame is a table with named rows and columns.

Create a virtual environment so this lesson has an isolated package set. On macOS or Linux, run:

python -m venv .venv
source .venv/bin/activate
python -m pip install sentence-transformers pandas matplotlib

On Windows PowerShell, activate the environment with .venv\Scripts\Activate.ps1, then run the same pip install command. The executed build used Python 3.13.2, sentence-transformers 5.5.0, PyTorch 2.12.0, Transformers 5.8.1, NumPy 2.5.1, pandas 3.0.3, and Matplotlib 3.11.0 on a CPU.

Download these three files and place them beside your Python file:

• archived_tickets.csv
• duplicate_validation_pairs.csv
• incoming_tickets.csv

They are original synthetic datasets released under CC0-1.0. Synthetic means they were written as teaching data. Every message, identifier, issue, and expected match is fictional, so no customer text is present.

The lesson uses sentence-transformers/all-MiniLM-L6-v2. Its official model card describes a 384-dimensional English sentence encoder for similarity, clustering, and search. The model is licensed under Apache-2.0. Review the model card, training-data notes, and your own data rules before using any pretrained model in a real system.

The first model load downloads files from Hugging Face and saves them in a local cache. Later loads can reuse that cached copy. Do not send private support text to an external service unless your organization has approved that data flow.

How Sentence Embedding Search Works

A sentence embedding is a fixed-length list of numbers that represents a complete piece of text. This model returns 384 numbers whether the input has five words or twenty words.

Imagine each number as one coordinate. The 384 coordinates place a message at one point in a 384-dimensional space. The model is designed to make texts with related meanings point in similar directions, even when their wording differs.

We compare directions with cosine similarity:

cosine similarity = (a · b) / (length(a) × length(b))

The dot in the numerator means: multiply matching coordinates and add the products. The result can range from -1 to 1. A larger value means the directions are more alike for this model. It does not mean “the probability of a duplicate.”

This lesson asks the encoder to normalize each embedding. Normalization rescales a vector to length 1 without changing its direction. When both vectors have length 1, their dot product equals their cosine similarity:

normalized cosine similarity = a · b

That lets one matrix multiplication compare every incoming ticket with every archive row. The model encodes each text independently; the comparison happens afterward.

Encode the Archive and Check the Vectors

Start by loading the three tables. Inspect a small archive preview before the column names disappear inside numeric arrays:

import numpy as np
import pandas as pd
from sentence_transformers import SentenceTransformer

archive = pd.read_csv("archived_tickets.csv", keep_default_na=False)
validation = pd.read_csv(
    "duplicate_validation_pairs.csv",
    keep_default_na=False,
)
incoming = pd.read_csv("incoming_tickets.csv", keep_default_na=False)

print(archive.head(3).to_string(index=False))

The executed preview is:

archive_id                                                                summary
     A-101    Downloaded invoice PDF opens as a blank page with no billing lines.
     A-102 Password reset link reports that it has expired before it can be used.
     A-103         Each account alert email is delivered twice to the same inbox.

Each archive row is one resolved issue summary. A production archive may also contain product, language, date, permissions, and status fields. Keep those fields beside the embedding; the vector should not become the only record of the ticket.

Now load the encoder and convert the eight summaries. normalize_embeddings=True makes every output row have length 1. show_progress_bar=False keeps this small run quiet:

MODEL_NAME = "sentence-transformers/all-MiniLM-L6-v2"

model = SentenceTransformer(MODEL_NAME, device="cpu")
archive_embeddings = model.encode(
    archive["summary"].tolist(),
    batch_size=8,
    show_progress_bar=False,
    normalize_embeddings=True,
)

print("shape:", archive_embeddings.shape)
print(
    "norms:",
    np.round(np.linalg.norm(archive_embeddings, axis=1), 6),
)

shape: (8, 384)
norms: [1. 1. 1. 1. 1. 1. 1. 1.]

The first axis contains eight archive rows. The second contains 384 embedding coordinates. Every norm is 1, so a dot product can be read as cosine similarity.

The current encode() documentation lists batching, NumPy conversion, precision, devices, and normalization options. Use one model version and one normalization rule for both sides of a comparison.

Rank One Ticket Before Processing the Batch

Work through one message before adding thresholds. Encode incoming ticket N-201, multiply it by the transposed archive matrix, and sort from largest score to smallest:

example_text = incoming.loc[0, "message"]
example_embedding = model.encode(
    [example_text],
    show_progress_bar=False,
    normalize_embeddings=True,
)

example_scores = example_embedding @ archive_embeddings.T
top_positions = np.argsort(example_scores[0])[::-1][:3]

example_ranking = pd.DataFrame({
    "archive_id": archive.iloc[top_positions]["archive_id"].to_numpy(),
    "score": example_scores[0, top_positions],
})
print(example_ranking.round({"score": 3}).to_string(index=False))

archive_id  score
     A-101  0.616
     A-106  0.329
     A-105  0.326

The incoming text says, “The bill PDF I downloaded has no text or charges.” Its best archive result, A-101, describes a blank invoice PDF. The terms are not identical, but the sentence encoder puts their meanings closer than the other choices.

The ranking alone is not a decision. Even an unrelated message has a highest score because argmax must return some position. We need a rule that can reject the entire archive.

Calibrate a Rejection Threshold with Labeled Pairs

A threshold is the minimum score required for an automatic suggestion. Do not choose it because 0.5 or 0.8 looks familiar. Similarity ranges depend on the model, domain, text length, and writing style.

The validation CSV contains 16 message-candidate pairs. Eight are labeled duplicates and eight describe different issues. These rows are separate from the ten incoming tickets used later.

Encode each validation message and its specified archive candidate. Pairwise cosine similarity is the sum of coordinate products because both arrays are normalized:

archive_lookup = archive.set_index("archive_id")["summary"]

validation_messages = model.encode(
    validation["message"].tolist(),
    batch_size=8,
    show_progress_bar=False,
    normalize_embeddings=True,
)
validation_candidates = model.encode(
    validation["candidate_id"].map(archive_lookup).tolist(),
    batch_size=8,
    show_progress_bar=False,
    normalize_embeddings=True,
)

validation["cosine_similarity"] = np.sum(
    validation_messages * validation_candidates,
    axis=1,
)

Next, evaluate thresholds from 0.30 through 0.80. Precision asks what fraction of suggested duplicates are correct. Recall asks what fraction of the known duplicates were found. F1 combines both into one score.

The build counts true and false decisions at each threshold, calculates those three measures, and selects the row with the highest F1. Ties prefer higher recall and then the higher threshold:

def score_threshold(labels, scores, threshold):
    predicted = scores >= threshold
    positive = labels == 1
    tp = np.sum(predicted & positive)
    fp = np.sum(predicted & ~positive)
    fn = np.sum(~predicted & positive)
    tn = np.sum(~predicted & ~positive)

    precision = tp / (tp + fp) if tp + fp else 0.0
    recall = tp / (tp + fn) if tp + fn else 0.0
    f1 = (
        2 * precision * recall / (precision + recall)
        if precision + recall
        else 0.0
    )
    return threshold, tp, fp, fn, tn, precision, recall, f1


threshold_rows = [
    score_threshold(
        validation["is_duplicate"].to_numpy(),
        validation["cosine_similarity"].to_numpy(),
        threshold,
    )
    for threshold in np.arange(0.30, 0.81, 0.05)
]
thresholds = pd.DataFrame(
    threshold_rows,
    columns=["threshold", "tp", "fp", "fn", "tn", "precision", "recall", "f1"],
)
best = thresholds.sort_values(
    ["f1", "recall", "threshold"],
    ascending=False,
).iloc[0]

print(best[["threshold", "precision", "recall", "f1"]].round(3))

threshold    0.550
precision    0.889
recall       1.000
f1           0.941

At 0.55, the validation set finds all eight duplicates and incorrectly accepts one of eight different-issue pairs. That false positive matters: a refund-button fault scores 0.635 against a summary about refunds missing from revenue totals. Shared topic words can produce a high score even when the required fix differs.

This is a small teaching set, so 0.55 is not a reusable support-industry constant. A real threshold needs representative labels, enough hard negatives, separate final evaluation data, and review whenever the model or ticket mix changes.

Suggest a Match or Send the Ticket for Review

Encode all incoming messages in one batch. The matrix product has ten rows and eight columns: one score for each incoming-archive pair.

incoming_embeddings = model.encode(
    incoming["message"].tolist(),
    batch_size=8,
    show_progress_bar=False,
    normalize_embeddings=True,
)
similarity_matrix = incoming_embeddings @ archive_embeddings.T

top_positions = similarity_matrix.argmax(axis=1)
top_scores = similarity_matrix.max(axis=1)
threshold = float(best["threshold"])

results = incoming[["incoming_id", "expected_match_id"]].copy()
results["top_match_id"] = archive.iloc[top_positions][
    "archive_id"
].to_numpy()
results["top_score"] = top_scores
results["decision"] = np.where(
    top_scores >= threshold,
    "suggest match",
    "manual review",
)
results["predicted_match_id"] = np.where(
    results["decision"].eq("suggest match"),
    results["top_match_id"],
    "",
)

print(
    results[
        ["incoming_id", "top_match_id", "top_score", "decision"]
    ].round({"top_score": 3}).to_string(index=False)
)

incoming_id top_match_id  top_score      decision
      N-201        A-101      0.616 suggest match
      N-202        A-102      0.775 suggest match
      N-203        A-103      0.756 suggest match
      N-204        A-104      0.777 suggest match
      N-205        A-105      0.833 suggest match
      N-206        A-106      0.722 suggest match
      N-207        A-106      0.087 manual review
      N-208        A-104      0.380 manual review
      N-209        A-107      0.611 suggest match
      N-210        A-108      0.646 suggest match

Eight tickets clear the validation-based threshold. N-207, which asks about adding a workspace member, has no related archive summary. N-208 mentions a calendar but describes an incorrect event time rather than stopped synchronization. Topic overlap raises its best score to 0.380, but the threshold keeps it out.

The expected-match column is used only to check this controlled example. It does not enter embedding, ranking, or threshold decisions. All ten outcomes match their author-specified answers, but ten synthetic rows cannot estimate production accuracy.

Read the left panel first. Most duplicate scores sit to the right of most different-issue scores, but one orange point crosses the red threshold. The right panel applies the same fixed rule to incoming tickets; orange bars are not accepted even though each still has a top archive ID.

The complete executed output is available in similar_ticket_results.csv. The published chart is a real Matplotlib SVG produced by the same build, using the non-interactive Agg backend.

Add Checks Before Trusting the Ranking

Tests should cover rules that must stay true even if messages change. Check vector width, normalization, matrix shape, score order, and rejection behavior:

assert archive_embeddings.shape[1] == model.get_embedding_dimension()
assert np.allclose(
    np.linalg.norm(archive_embeddings, axis=1),
    1.0,
    atol=1e-5,
)
assert similarity_matrix.shape == (len(incoming), len(archive))
assert np.allclose(top_scores, similarity_matrix.max(axis=1))
assert results.loc[
    results["decision"].eq("manual review"),
    "predicted_match_id",
].eq("").all()

These checks catch mixed embedding sizes, forgotten normalization, a transposed matrix, and weak matches that accidentally receive IDs. They do not prove that a suggested ticket is truly a duplicate. Meaning still needs labeled evaluation and, for costly decisions, human confirmation.

Save the model name, package versions, normalization setting, archive version, and threshold with each result batch. Re-encoding only half the archive with a new model creates scores that are not comparable.

What Can Go Wrong and How to Fix It

Treating cosine similarity as probability. A score of 0.72 does not mean a 72% chance of duplication. Calibrate decisions with labeled data from the intended domain. Use probability calibration only if you have a suitable evaluation design.

Always returning the top result. argmax produces an answer even when all choices are poor. Add a rejection threshold and a manual-review route.

Choosing the threshold on final test rows. That makes the reported result too optimistic. Use development data for workflow changes, validation data for the threshold, and a separate test period for the final estimate.

Using only easy negatives. Real errors often share product names and actions. Include close but different problems, such as “refund action unavailable” versus “refund missing from totals.”

Mixing normalized and raw embeddings. Dot product equals cosine similarity only when both sides have length 1. Normalize every compared vector, or call a documented cosine function such as the Sentence Transformers cos_sim utility.

Changing the model for only new messages. Embedding coordinates belong to a specific model and version. Re-encode the archive when the encoder changes, then recalibrate the threshold.

Ignoring truncation. The model card states that inputs longer than 256 word pieces are truncated by default. Important details near the end may disappear. Use concise issue summaries, inspect token lengths, or select and test a model designed for longer text.

Assuming English behavior transfers to every language. This model card identifies an English model. Use a documented multilingual model and labeled examples for every language you plan to support.

Using old resolutions without access checks. A similar ticket may contain private text or an outdated fix. Filter the searchable archive by permissions, product version, and current status before showing results.

Reading this synthetic result as a promise. The ten incoming cases were designed to explain the workflow. Real data has spelling errors, several problems in one message, changing products, incomplete summaries, and uncertain labels.

Recap: Encode, Rank, Calibrate, and Review

Sentence embeddings turn variable-length messages into equal-width numeric vectors. Normalization makes a dot product act as cosine similarity, so one matrix multiplication can rank an archive for many incoming tickets.

Ranking is only the first half of a safe matcher. Use labeled duplicate and different-issue pairs to select a threshold, then keep low scores out of the automatic path. In this executed example, the validation data selected 0.55, eight incoming tickets received suggested matches, and two went to manual review.

For a next step, collect reviewer decisions without exposing private text. Build a time-based test set, measure precision and recall at several thresholds, and examine errors by product and language. The durable workflow is not “accept the nearest vector.” It is “retrieve the nearest candidate, test whether the score is strong enough, and preserve a review path when it is not.”