Skip to main content

Agentic retrieval

Regular RAG is a 2-steps process: first find relevant matches according the user question, then pass the best matches to an LLM to phrase an answer. It works great when the question is not too complex, so all the needed information can be retrieved with a single search. But when it comes to questions implying some kind of synthesis, or intermediary results, it may not be efficient enough. For example, when asking "Compare the discoveries of Marie Curie and Cecilia Payne", if your Knowkledge Box contains different documents detailing the discoveries of these two scientists but none comparing them together, it is very likely that the retrieved results for this query will be pretty weak and you will not get a proper answer.

Agentic Retrieval is an extra layer on top of RAG. It uses a Smart Agent to decide autonomously how to answer the question.

It may decide to make intermediary questions (typically in the previous example, it would make a first query to get the discoveries of Marie Curie, and another one for the discoveries of Cecilia Payne), and then make a synthesis to answer the initial question. It is able to evaluate how relevant are the retrieved information and then decide if that's suffucient to answer or if it needs to iterate, making further queries until the needed information is obtained.

It can also access external information sources, like MCP servers, Gemini or Perplexity.

Using Agentic retrieval with the Knowledge Box resources​

In the Search page, when creating a search configuration, you can select the Agentic Retrieval mode. The Agentic retrieval works with sources, by default if no source were ever declared before, the UI will offer to turn the current Knowledge Box as a source. You must provide a title and a description for the Knowledge Box content. The existing Knowledge Box title and description might be good enough, but do not hesitate to make them more explicit as these information will be used by the Smart Agent to decide how to manage the query.

Then you need to choose the planning mode:

  • Reactive: The Smart Agent will decide what to do first, and will plan the next steps based on the information it retrieves. It is expected to be faster.
  • Plan & execute: The Smart Agent will plan all the steps in advance, and will execute them. It will be slower but more accurate when processing complex questions.

For a more advanced usage, you can customize the different prompts and select different LLMs, but as a first step, keep the default.

Then you can save the configuration, and test the search widget.

Using different subsets of resources​

In some cases, you might have different kind of documents in your Knowledge Box, for example let's imagine a Knowledge box about bike repair, you will have How-tos, like "How to replace the brakes" or "How to fix a broken wheel", and Spare part providers, indicating where such or such spare part can be bought. If you just use the entire Knowledge Box as unique source, when a user will ask "My front wheel is broken, what should I do?", it is very likely that the Smart agent will retrieve the corresponding How-to and will answer based on its content, but it will probably not mention where to buy the needed spare parts. You will get better answer if you declare the Knowledge Box twice, each with different titles and descriptions and appropriate filtering to have a "How-tos" source and a "Spare part providers" source. The sources can be configured in the Connect tab of the Sync & Connect page.

Once the sources are available in the search configuration, you might just rely on the source descriptions to have the Smart agent properly instructed about how it should use such or such source, but you can also give extra explicit instructions like:

"When explaining how to repair something, always check where the corresponding spare parts can be found."

Connecting external sources​

At the moment, the Agentic Retrieval supports the following external sources:

  • Google Gemini
  • Perplexity
  • MCP servers

They can be added in the Connect tab of the Sync & Connect page.

By providing accurate descriptions, you make sure the Smart agent will pick the appropriate source when trying to retrieve information. For example, if you are implementing a Sales support tool, you could put customer personas as PDF in your Knowledge Box, and connect your CRM as MCP. So when asking "How should I present Product A to Eric?", it will use the MCP server to know what persona relates to Eric, and then search in the KB what is the most appropriate approach for this given persona.

Use Agentic retrieval through the API​

Create an agentic configuration​

We recommend to create your agentic configuration from the dashboard. But if you prefer to use the API, you can use the Configuration creation endpoint.

Use an agentic configuration on /ask​

The /ask endpoint takes an agentic_config_id parameter which allows specifying the Agentic configuration to use.

It can be used as a POST endpoint, and the behaviour is identical to the regular behaviour when using no Agentic configuration. But it can also be used as a WebSocket GET, in that case it will behave like the Hyperforge websocket