AgentGroupChat state and instance management

[vslepakov]

Hey folks!

I have a few questions about how AgentGroupChat operates:

1. Assistant Thread Management
   How does AgentGroupChat handle threads in AzureAssistantAgent? I see there's a way to retrieve an existing AzureAssistantAgent and add it to an AgentGroupChat, but what's the best way to rehydrate a specific thread? Is it more appropriate to use a single instance of AzureAssistantAgent per user?

2. Conversation History
   How does AgentGroupChat keep track of conversation history? If I add an existing AzureAssistantAgent to the chat, does it pull in the history for a particular thread automatically? Is there a way to manually provide conversation history for a ChatCompletionAgent?

3. Instance Management
   In a real-world application, should AgentGroupChat be a singleton? Or is it best to create a new AgentGroupChat for each API request, managing assistant thread clean up separately in that case?

Thanks!

[moonbox3]

@crickman, can you take a look, please?

[crickman]

Hi @vslepakov - By AzureAssistantAgent, are you refering to the Azure AI Agent SDK (https://learn.microsoft.com/azure/ai-services/agents/quickstart) or just the Azure usage of the Open AI Assistant API (OpenAIAssistantAgent)? https://platform.openai.com/docs/api-reference/assistants

Also, are you using .NET or Python?

[vslepakov]

Hi @crickman, I am referring to this one from semantic_kernel.agents.open_ai import AzureAssistantAgent described here and utilized as part of a group chat

[crickman]

Ok, thank you for the clarification.

1. Yes, you can retrieve an existing assistant definition and then add it to group chat the same as with any agent: https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/agents/open_ai/azure_assistant_agent.py#L308. Currently, there isn't a support mechanism to rehydrate an existing thread for AgentGroupChat; although, you may specifiy a thread-id if directly invoking the agent.
2. An AgentGroupChat instance defaults to creating a new thread for its scope. If a ChatCompletionAgent is present in the AgentGroupChat, the conversation will by synchronized with the assistant thread automatically.
3. AgentGroupChat is stateful. As such, I'd not expect it to be accessed as a singleton. Conversely, a transient scoped AgentGroupChat would also likely not meet expectations. Instead, definining a provider that is able to create or retrieve an AgentGroupChat based on a key or session may be the most natural approach.

[vslepakov]

Thanks, @crickman, just to clarify a bit more:

1. How can an instance of AgentGroupChat be reused in a multi-user scenario?

   • It seems I would need a separate instance of both AzureAssistantAgent and AgentGroupChat for each user since there's no way to make AgentGroupChat target a specific thread, correct?
   • Additionally, if I retrieve and add an AzureAssistantAgent instance that already contains multiple threads, which thread is selected?
   • Is support for handling multiple threads planned for the future?

2. It's great to know that AgentGroupChat synchronizes the assistant thread with the ChatCompletionAgent, but does this mean the part of the conversation contributed by the ChatCompletionAgent and the user is lost or is the whole history shared with and persisted by AzureAssistantAgent even if it is not selected on every turn?

3. By

   Instead, definining a provider that is able to create or retrieve an AgentGroupChat based on a key or session

   you basically mean retrieving the correct AzureAssistantAgent instance (currently containing only one thread) and adding it to AgentGroupChat, correct?

[crickman]

These are all really good questions based on relevant insight.

1. An agent's state isn't fundamentally associated with any specific conversation. An agent may participate in multiple conversations: either via direct invocation or group-chat. The state management for any given conversation is managed outside of the agent definition. In this regard, an Assistant agent may participate in as many threads as you like.

2. The entire history will be loaded into a new thread when an assistant joins. Assistants that are defined in the same azure endpoint will utilize a shared thread id. Assistants from different regions / endpoints will necessarily require separate threads...each of which will be syncrhonized to the entire conversation state when created. (Note: This may result in some additional latency if an assistant joins a long-running conversation).

3. No, I was referring more to managing the conversation state. Consider a basic chat-completion case without agents. A user interacting with the LLM could have a chat-identifier that is used to associate a request with the chat-history that had been built-up over multiple requests. This chat-history could even be persisted in a database and re-hydrated for a long-term / durable conversation. For .NET we are able to serialize and deserialize the group-chat...which necessarily entails continuing with any existing thread-id that may be defined. I don't believe this functionality is yet present in python.

[vslepakov]

Thanks @crickman for these insights. It helps a lot!