OpenAI Agent doesn't response with RAG

[andreabrgit]

Hi,

I created a OpenAI Agent with a RAG (query_engine = RetrieverQueryEngine.from_args) and others FunctionTools. I put on description the same text of Llamaindex's RAG Projeto: "Use this tool to answer any user question over any **data.". But when I ask something, the agent responses with a global knowledge, without source_nodes based on documentation.

How can I improve this behavior?

[dosubot[bot]]

To improve the behavior of your OpenAI Agent with RAG so that it responds with source_nodes based on documentation instead of global knowledge, consider implementing the following techniques:

1. Decouple Chunks for Retrieval and Synthesis: Use different chunk representations for retrieval and synthesis. This helps in retrieving relevant documents at a high level before retrieving specific chunks.

2. Structured Retrieval for Larger Document Sets: Use metadata filters and auto retrieval to tag each document with metadata and store it in a vector database. During inference, use the LLM to infer the right metadata filters to query the vector database.

3. Dynamically Retrieve Chunks Depending on Task: Use task-specific retrieval techniques, such as the router module and data agent module, to handle different types of queries like summarization or comparisons.

4. Optimize Context Embeddings: Fine-tune the embedding model to better capture the salient properties of your specific data corpus, ensuring that the embeddings are optimized for better retrieval.

5. Implement Self-RAG Query Engine: Utilize the SelfRAGQueryEngine, which includes a critic component to evaluate and score the relevance, support, and usefulness of retrieved paragraphs, ensuring responses are based on the most relevant documentation.

6. Use Multi-Document Agents: Set up a "document agent" for each document to perform QA/summarization within its document, and create a top-level agent that orchestrates across these document agents to answer queries.

7. Implement Reranking and Query Planning: Incorporate reranking during document retrieval to better filter the candidate set of documents and use a query planning tool to dynamically create plans based on the set of retrieved tools.

By applying these techniques, you can enhance the retrieval accuracy and ensure that the responses are based on the provided documentation rather than relying on global knowledge ^[1][2][3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[andreabrgit]

Could you show me more details and source code to explain with more details?