4 Essential Steps to Create Multi-Agent Nested Chats with AutoGen

By 2024-11-12

4 Essential Steps to Create Multi-Agent Nested Chats with AutoGen

Introduction

As artificial intelligence progresses, chatbots have transformed from simple automated responders into sophisticated conversational agents that can handle layered and nuanced interactions. With the introduction of multi-agent architectures, tools like AutoGen enable developers to create dynamic chat experiences where multiple agents can interact seamlessly within a single conversation. This opens up opportunities for more personalized and responsive user engagements, where bots can tackle different tasks, answer varied questions, and respond to complex user needs without rigid sequencing.

In this article, we’ll dive into the four essential steps for building multi-agent nested chats using AutoGen. You’ll discover how to define each agent’s role, structure smooth inter-agent communication, manage unexpected interruptions, and maintain context across conversations. By following these steps, you can create a chat experience that feels more natural, adapts to real-time needs, and enhances user satisfaction with multi-dimensional, collaborative interactions.

Table of Contents

What is Nested Chat?

Nested Chat refers to a conversational structure in which multiple agents or bots interact within a single, unified chat flow, allowing them to carry out tasks and respond to user inputs in a flexible, non-linear way. Instead of following a strictly sequential, turn-based interaction where each message waits for the previous one to finish, nested chat allows agents to "nest" or interleave their responses, handle interruptions, and pivot between topics or tasks smoothly. 

This setup enables chatbots to:

1. Respond Dynamically: Instead of being locked into a single task or conversation path, the chat can shift based on user needs, allowing agents to respond flexibly and pivot when necessary.

2. Handle Complex Queries: In nested chats, agents can dive into sub-tasks or specialized questions and then return to the main conversation, allowing users to switch contexts without breaking the flow.

3. Leverage Multiple Agents in Real-Time: Different agents with distinct expertise can step into the conversation when relevant (e.g., one bot handling account support while another offers technical assistance), all within a single thread.

This nested structure is especially useful in multi-agent chat systems, as it enables a more human-like conversation experience. For instance, in customer service, nested chat could allow a bot to respond to questions about product details, shipping issues, and returns within one conversation, all while maintaining context and continuity.

The below figure shows the conversion flow of a nested chat.

conversion flow of a nested chat

When an incoming message meets a specific condition, it is routed to a nested chat. This nested chat could be in the form of a two-agent chat, a sequential chat, or any other type. Once the nested chat completes, the results are sent back to the main conversation.

Implementing Nested Chat in AutoGen

Implementing nested chat functionality in AutoGen Studio involves creating a system where multiple levels of conversational flows or threads can be handled within the same session. Each nested chat would maintain context and provide a more dynamic and structured interaction.

In this article, we will develop an article-writing system using nested chats. We’ll create three agents: one for generating an article outline, another for writing the article based on the outline, and a third for reviewing the article. To enable multiple interactions between the writer and the reviewer, these two agents will be placed in a nested chat.

Furthermore, the outline agent will have access to a tool for querying the web.

Let’s now implement this with code.

Pre-requisites

Before creating AutoGen agents, make sure you have the required API keys for the necessary Large Language Models (LLMs). For this exercise, we will also use Tavily to search the web.

Load the `.env` file with the necessary API keys. In this case, we will be using the OpenAI and Tavily API keys.

from dotenv import load_dotenv
import os

# Load the .env file
load_dotenv('/home/santhosh/Projects/courses/Pinnacle/.env')

# Access environment variables
openai_api_key = os.getenv('OPENAI_API_KEY')
tavily_api_key = os.getenv('TAVILY_API_KEY')

# Print to verify (in a real-world application, avoid printing sensitive info)
print(f"OpenAI API Key: {openai_api_key}")
print(f"Tavily API Key: {tavily_api_key}")


To define the LLMs in a config_list, you can set up a Python dictionary to hold configurations for each LLM, using the loaded environment variables for secure API access. Here’s an example of how to configure this:

from dotenv import load_dotenv
import os

# Load environment variables from the .env file
load_dotenv('/home/santhosh/Projects/courses/Pinnacle/.env')

# Define the LLM configuration list
config_list = {
    "openai": {
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model": "gpt-4",  # Specify the model you want to use, e.g., "gpt-4" or "gpt-3.5-turbo"
        "temperature": 0.7,  # Adjust temperature for response variability
        "max_tokens": 1000  # Limit the response length
    },
    "tavily": {
        "api_key": os.getenv("TAVILY_API_KEY"),
        "query_tool": "web_search",  # Specify the tool you want to use
        "max_results": 5  # Limit the number of results
    }
}

# Optional: Print config for verification (avoid in production)
print(config_list)


Step 1: Define the Outline Agent with the Tool Use

To define the LLMs in a config_list, you can set up a Python dictionary to hold configurations for each LLM, using the loaded environment variables for secure API access. Here’s an example of how to configure this:

from dotenv import load_dotenv
import os

# Load environment variables from the .env file
load_dotenv('/home/santhosh/Projects/courses/Pinnacle/.env')

# Define the LLM configuration list
config_list = {
    "openai": {
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model": "gpt-4",  # Specify the model you want to use, e.g., "gpt-4" or "gpt-3.5-turbo"
        "temperature": 0.7,  # Adjust temperature for response variability
        "max_tokens": 1000  # Limit the response length
    },
    "tavily": {
        "api_key": os.getenv("TAVILY_API_KEY"),
        "query_tool": "web_search",  # Specify the tool you want to use
        "max_results": 5  # Limit the number of results
    }
}

# Optional: Print config for verification (avoid in production)
print(config_list)


To define a web_search function that queries the web using an API like Tavily, you can create a function that accepts a search query and returns the results. Here's an example using the Tavily API:

import requests

def web_search(query, api_key, num_results=5):
    """
    Queries the web using Tavily's search API and returns the results.
    
    Parameters:
        query (str): The search query string.
        api_key (str): Tavily API key for authentication.
        num_results (int): Number of search results to retrieve. Default is 5.
    
    Returns:
        list: A list of search results with relevant snippets.
    """
    url = "https://api.tavily.com/v1/search"
    headers = {"Authorization": f"Bearer {api_key}"}
    params = {
        "query": query,
        "num_results": num_results
    }
    
    try:
        response = requests.get(url, headers=headers, params=params)
        response.raise_for_status()  # Raises an error for non-200 status codes
        results = response.json().get("results", [])
        
        # Extract relevant information (e.g., snippet, title, URL) from the results
        search_results = [
            {"title": result["title"], "snippet": result["snippet"], "url": result["url"]}
            for result in results
        ]
        
        return search_results
    
    except requests.exceptions.RequestException as e:
        print("Error querying Tavily API:", e)
        return []

# Example usage
tavily_api_key = "your_tavily_api_key_here"  # Replace with your actual API key
search_query = "Benefits of Artificial Intelligence in Healthcare"
results = web_search(query=search_query, api_key=tavily_api_key, num_results=5)

# Print results for verification
for i, result in enumerate(results, start=1):
    print(f"Result {i}:")
    print("Title:", result["title"])
    print("Snippet:", result["snippet"])
    print("URL:", result["url"])
    print()


To register the web_search function with the Outline Agent, set it up as an executor with user_proxy permissions. 

In this setup, to "register" the web_search function with the Outline Agent, we can define a register_function method within the OutlineAgent class. This method allows you to register custom functions (like web_search) dynamically, providing flexibility in assigning tools to the agent.


def register_function(self, func, func_name):
    """Register a custom function to the OutlineAgent."""
    self.registered_functions[func_name] = func
    print(f"Function '{func_name}' registered successfully.")


Step 2: Define the Writer and Reviewer Agents

To create a collaborative writing system with Writer and Reviewer agents in AutoGen, you can structure each agent to handle specific roles and interact with each other within a nested chat setup. Here, the Writer Agent will generate the article based on an outline, while the Reviewer Agent will check the article, provide feedback, and suggest improvements. The Reviewer may also have permissions to interact multiple times with the Writer to ensure high-quality output.

import openai
import os

class WriterAgent:
    def __init__(self, config):
        self.config = config

    def generate_article(self, outline):
        """Generate the article based on the provided outline."""
        prompt = f"Write a comprehensive article based on the following outline:\n{outline}"
        
        response = openai.ChatCompletion.create(
            model=self.config["openai"]["model"],
            messages=[{"role": "user", "content": prompt}],
            temperature=self.config["openai"]["temperature"],
            max_tokens=self.config["openai"]["max_tokens"]
        )
        

class ReviewerAgent:
    def __init__(self, config):
        self.config = config

    def review_article(self, article):
        """Review the article, provide feedback, and suggest improvements if needed."""
        prompt = f"Review the following article for grammar, coherence, accuracy, and completeness. Provide constructive feedback and suggest improvements:\n{article}"
        
        response = openai.ChatCompletion.create(
            model=self.config["openai"]["model"],
            messages=[{"role": "user", "content": prompt}],
            temperature=self.config["openai"]["temperature"],
            max_tokens=self.config["openai"]["max_tokens"]
        )
        
        review_feedback = response.choices[0].message['content']
        print("Review Feedback:", review_feedback)
        return review_feedback

    def suggest_improvements(self, article):
        """Suggest specific improvements to enhance the article quality."""
        prompt = f"Suggest specific changes to improve the following article in terms of clarity, structure, and detail:\n{article}"
        
        response = openai.ChatCompletion.create(
            model=self.config["openai"]["model"],
            messages=[{"role": "user", "content": prompt}],
            temperature=self.config["openai"]["temperature"],
            max_tokens=self.config["openai"]["max_tokens"]
        )


Step 3: Register the Nested Chat

To register the nested chat between the Writer and Reviewer agents, we need to implement a process where both agents can communicate in multiple iterations until the article reaches the desired quality. By registering this interaction as a nested chat, we can manage and facilitate back-and-forth exchanges between the agents, allowing each to complete their respective roles iteratively.

class NestedChatManager:
    def __init__(self, writer_agent, reviewer_agent, max_iterations=3):
        self.writer_agent = writer_agent
        self.reviewer_agent = reviewer_agent
        self.max_iterations = max_iterations

    def run_nested_chat(self, outline):
        """Run a nested chat between Writer and Reviewer agents until the article is approved."""
        # Step 1: Writer generates the initial draft
        article_draft = self.writer_agent.generate_article(outline)
        print("\nInitial Article Draft:\n", article_draft)
        
        iteration = 0
        approved = False

        # Step 2: Begin nested review-refinement loop
        while iteration < self.max_iterations and not approved:
            print(f"\n--- Iteration {iteration + 1} ---")

            # Reviewer provides feedback
            feedback = self.reviewer_agent.review_article(article_draft)
            suggestions = self.reviewer_agent.suggest_improvements(article_draft)
            print("\nReviewer Feedback:\n", feedback)
            print("\nReviewer Suggestions:\n", suggestions)
            
            # Decide if article is approved or needs more work
            if "approved" in feedback.lower() or "satisfactory" in feedback.lower():
                approved = True
                print("\nArticle has been approved by Reviewer.")
            else:
                # Writer refines the article based on feedback and suggestions
                refined_prompt = f"Revise the article based on the following feedback and suggestions:\nFeedback: {feedback}\nSuggestions: {suggestions}\n\nArticle:\n{article_draft}"
                article_draft = self.writer_agent.generate_article(refined_prompt)
                print("\nRefined Article Draft:\n", article_draft)
            
            iteration += 1

        if not approved:
            print("\nArticle was not fully approved within the maximum iterations. Final draft submitted as-is.")
        
        return article_draft

# Instantiate and register the NestedChatManager with the Writer and Reviewer agents
nested_chat_manager = NestedChatManager(writer_agent, reviewer_agent)

# Example outline to initiate the nested chat
sample_outline = "1. Introduction to AI in Healthcare\n2. Key Applications\n3. Challenges\n4. Future Prospects\n5. Conclusion"

# Run the nested chat process
final_article = nested_chat_manager.run_nested_chat(sample_outline)
print("\nFinal Article:\n", final_article)


Step 4: Initiate the Nested Chat

To initiate chats using the user_proxy in the context of the nested chat system, you'd typically use the initiate_chats method, which would start the process of managing the nested conversations between agents. 

However, the exact implementation depends on how the user_proxy and initiate_chats are defined within your framework. Assuming you're using a system like AutoGen where proxies and chat functions are defined, the general flow might look like this:

# Assuming 'user_proxy' is a registered proxy in the system, 
# and 'initiate_chats' is used to start the chat process between agents
chat_results = user_proxy.initiate_chats(
    agents=[writer_agent, reviewer_agent],  # List of agents involved
    initial_message="Start the article writing and review process",  # Initial prompt for chat
    max_iterations=3  # Optional: Max number of iterations for refinement
)

# Output the results from the chat session
print("Chat Results:", chat_results)

Here’s the complete implementation of the Nested Chat in AutoGen, with the Writer Agent, Reviewer Agent, and the Nested Chat Manager all in one script.

import openai
from dotenv import load_dotenv
import os

# Load environment variables (API keys for OpenAI)
load_dotenv('/path/to/.env')  # Ensure the .env file is correctly loaded
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

openai.api_key = OPENAI_API_KEY


# Define the configuration for the agents
config = {
    "model": "gpt-4",
    "temperature": 0.7,
    "max_tokens": 1500
}


# Writer Agent Class
class WriterAgent:
    def __init__(self, config):
        self.config = config

    def generate_article(self, outline):
        """Generates an article based on the given outline."""
        prompt = f"Write an article based on the following outline:\n{outline}"
        
        response = openai.ChatCompletion.create(
            model=self.config["model"],
            messages=[{"role": "user", "content": prompt}],
            temperature=self.config["temperature"],
            max_tokens=self.config["max_tokens"]
        )
        
        article = response.choices[0].message['content']
        return article


# Reviewer Agent Class
class ReviewerAgent:
    def __init__(self, config):
        self.config = config

    def review_article(self, article):
        """Reviews the article and provides feedback."""
        prompt = f"Review the following article and provide feedback:\n{article}"
        
        response = openai.ChatCompletion.create(
            model=self.config["model"],
            messages=[{"role": "user", "content": prompt}],
            temperature=self.config["temperature"],
            max_tokens=self.config["max_tokens"]
        )
        
        feedback = response.choices[0].message['content']
        return feedback

    def suggest_improvements(self, article):
        """Suggests improvements to enhance the article's quality."""
        prompt = f"Suggest improvements for the following article in terms of clarity and structure:\n{article}"
        
        response = openai.ChatCompletion.create(
            model=self.config["model"],
            messages=[{"role": "user", "content": prompt}],
            temperature=self.config["temperature"],
            max_tokens=self.config["max_tokens"]
        )
        
        suggestions = response.choices[0].message['content']
        return suggestions


# Nested Chat Manager Class
class NestedChatManager:
    def __init__(self, writer_agent, reviewer_agent, max_iterations=3):
        self.writer_agent = writer_agent
        self.reviewer_agent = reviewer_agent
        self.max_iterations = max_iterations

    def run_nested_chat(self, outline):
        """Runs a nested chat between Writer and Reviewer agents until the article is approved."""
        # Step 1: Writer generates the initial draft
        article_draft = self.writer_agent.generate_article(outline)
        print("\nInitial Article Draft:\n", article_draft)
        
        iteration = 0
        approved = False

        # Step 2: Begin nested review-refinement loop
        while iteration < self.max_iterations and not approved:
            print(f"\n--- Iteration {iteration + 1} ---")

            # Reviewer provides feedback
            feedback = self.reviewer_agent.review_article(article_draft)
            suggestions = self.reviewer_agent.suggest_improvements(article_draft)
            print("\nReviewer Feedback:\n", feedback)
            print("\nReviewer Suggestions:\n", suggestions)
            
            # Decide if article is approved or needs more work
            if "approved" in feedback.lower() or "satisfactory" in feedback.lower():
                approved = True
                print("\nArticle has been approved by Reviewer.")
            else:
                # Writer refines the article based on feedback and suggestions
                refined_prompt = f"Revise the article based on the following feedback and suggestions:\nFeedback: {feedback}\nSuggestions: {suggestions}\n\nArticle:\n{article_draft}"
                article_draft = self.writer_agent.generate_article(refined_prompt)
                print("\nRefined Article Draft:\n", article_draft)
            
            iteration += 1

        if not approved:
            print("\nArticle was not fully approved within the maximum iterations. Final draft submitted as-is.")
        
        return article_draft


# Sample outline to initiate the nested chat
sample_outline = """
1. Introduction to AI in Healthcare
2. Key Applications
   a. Diagnostics
   b. Treatment Recommendations
   c. Predictive Analytics
3. Challenges
   a. Data Privacy Concerns
   b. High Costs
   c. Ethical Considerations
4. Future Prospects
5. Conclusion
"""

# Instantiate the Writer and Reviewer agents
writer_agent = WriterAgent(config)
reviewer_agent = ReviewerAgent(config)

# Instantiate the NestedChatManager
nested_chat_manager = NestedChatManager(writer_agent, reviewer_agent)

# Start the nested chat process
final_article = nested_chat_manager.run_nested_chat(sample_outline)

# Print the final article after the nested chat process
print("\nFinal Approved Article:\n", final_article)

Conclusion

Nested chat in AutoGen expands chatbot functionality by enabling complex, multitasking interactions within a single conversation. This feature allows bots to trigger separate, specialized chats and seamlessly integrate their results. It enhances the ability to provide dynamic and context-aware responses across various fields, from e-commerce to healthcare. With nested chat, AutoGen empowers developers to create advanced, responsive AI systems capable of meeting a wide range of user needs with greater efficiency.