RAG Inside a Custom Skill: Wiring AI Search Retrievers the Right Way

Retrieval-augmented generation is the single highest-leverage technique you have for making a Now Assist skill accurate instead of merely fluent. And on this platform, RAG is not an abstract concept you implement from scratch. It is a concrete component: a retriever that infuses AI Search content into a custom skill. Getting it right is mostly about understanding where retrieval quality leaks, because the generation half is rarely the problem. The retrieval half almost always is.

The mechanism. A retriever is a tool you attach to a custom skill (built in the Now Assist Skill Kit) that, at runtime, queries an AI Search source, pulls back the most relevant chunks, and injects them into the prompt context before the model generates. The model then answers grounded in that retrieved content rather than from its parametric memory. The quality of the final answer is bounded by the quality of what the retriever pulls. If the right chunk never makes it into context, no amount of prompt cleverness saves you.

Step one: get the index right. AI Search is doing the heavy lifting. Before you touch the skill, confirm what is actually indexed and how. Verify the search source includes the tables and knowledge bases you intend to ground on, that the relevant fields are searchable, and that security is reflected in the index so retrieval respects the user's access. A retriever that returns content the requesting user should not see is a data-leak incident waiting to happen.

Step two: scope the retriever narrowly. The most common mistake is pointing a retriever at "everything." Broad scope dilutes relevance: the retriever returns loosely related chunks, those chunks crowd the context window, and the model stitches together a plausible but wrong answer. Scope each retriever to the specific knowledge a skill needs. A VPN-troubleshooting skill retrieves from VPN and network documentation, not the entire enterprise knowledge base. Narrow retrieval is the highest-impact tuning lever you have.

Step three: mind chunking and freshness. Two quiet killers. Chunking determines how source documents are split for retrieval; chunks that are too large bury the relevant sentence in noise, chunks that are too small lose the context that makes them meaningful. And freshness matters because a retriever happily returns a chunk from a knowledge article that was accurate two years ago. If your indexed content is stale, your grounded answers are confidently stale. Retrieval does not fix bad source data; it operationalizes it.

Step four: test retrieval separately from generation. This is the discipline that separates people who debug RAG from people who guess at it. Before you evaluate the skill's final answer, evaluate retrieval in isolation: for a set of representative queries, does the retriever return the chunks that contain the correct answer? If retrieval precision is low, fix that first; tuning the prompt while retrieval is broken is wasted effort. Only once the right content reliably lands in context should you tune how the model uses it.

Step five: design the no-result path. What does the skill do when retrieval comes back empty or low-confidence? The wrong answer is to let the model improvise, because that is exactly when it hallucinates. Design an explicit fallback: respond that the information is not available, route to a human, or ask a clarifying question. A skill that says "I do not have that documented" is infinitely more trustworthy than one that invents a procedure.

Done well, a retriever turns a generic LLM into something that answers from your knowledge, with your access controls, and admits when it does not know. That is the whole point of RAG, and it lives or dies in the retrieval layer, not the model.