Under the Hood of Similarity: How doc2vec Actually Powers ServiceNow Matching

You have almost certainly trained a similarity solution: "find similar incidents," "show related cases," the recommendation that helps an agent resolve faster. You clicked through the wizard, pointed it at a table, and trained it. But when the matches come back noisy or irrelevant, most admins have no idea how to diagnose it, because they never learned what is happening underneath. So let us actually open it up, because once you understand the mechanism, tuning stops being guesswork.

What a similarity solution is doing. At its core, the model measures textual similarity between records. It reads the fields you train it on, builds a numerical representation of each record's text, and then, at prediction time, finds the records whose representations are closest to the one you are asking about. "Closest" is a mathematical distance between vectors. The whole game is turning messy human text into numbers that capture meaning well enough that mathematically-near means actually-related.

Building the vocabulary. The first thing training does is build a vocabulary of words from your trained records. This matters more than it sounds. The model can only reason about words it has seen during training. If your corpus is small or narrow, the vocabulary is thin, and records that use different-but-synonymous language will not match well because the model never learned they are related. Corpus size and quality directly determine ceiling performance.

doc2vec, and why it is more than keyword matching. ServiceNow's similarity uses Paragraph-Vector, commonly known as doc2vec, which is built on the word2vec algorithm. Here is the intuition without the math. word2vec learns vector embeddings for individual words such that words used in similar contexts end up near each other in vector space; "password" and "credential" drift close together because they appear in similar sentences. doc2vec extends this to whole documents (your records), producing a single vector per record that captures its overall meaning, not just its keywords. This is why a good similarity model can match two incidents that share almost no exact words but describe the same underlying problem. It is matching meaning, not string overlap.

Why match quality degrades, and how to read the symptoms. Several predictable failure modes. A noisy corpus, where the trained fields contain boilerplate, signatures, stack traces, or templated text, teaches the model that the boilerplate is the signal, so everything looks similar to everything. Very short text gives doc2vec too little to work with, so vectors are unstable and matches are random. And dominant common words, if not handled, wash out the distinctive terms that actually carry meaning. If your matches are "technically related but useless," suspect noise in the corpus. If they are "random," suspect text that is too short or a vocabulary that is too thin.

Tuning with the word corpus. The lever most admins never touch is the word corpus, where you can improve results by managing which words count. Pruning high-frequency noise words and ensuring domain-specific terms are well represented sharpens the vectors. The principle: help the model spend its representational budget on the words that distinguish records, not the words every record shares.

Evaluate before you trust. Do not ship a similarity solution on vibes. Take a set of records where you know the correct related items, run prediction, and check whether the model surfaces them near the top. Measure precision at the top few results, because that is what users actually see. If precision is poor, go back to the corpus and the training fields before you blame the algorithm. The algorithm is sound; the input usually is not.

Understanding doc2vec changes how you build. You stop thinking "which fields should I dump in" and start thinking "which text best captures what makes these records meaningfully different," which is exactly the question that produces good matches.