Multi-Agent Debate¶

Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.

We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.

import asyncio
import os

from pydantic import Field, BaseModel

from agentscope.agent import ReActAgent
from agentscope.formatter import (
    DashScopeMultiAgentFormatter,
    DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub

# Prepare a topic
topic = (
    "The two circles are externally tangent and there is no relative sliding. "
    "The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
    "around circle B one trip back to its starting point. How many times will "
    "circle A revolve in total?"
)


# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
    """Get a solver agent."""
    return ReActAgent(
        name=name,
        sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
        "debate competition. It's unnecessary to fully agree with "
        "each other's perspectives, as our objective is to find "
        "the correct answer. The debate topic is stated as "
        f"follows: {topic}.",
        model=DashScopeChatModel(
            model_name="qwen-max",
            api_key=os.environ["DASHSCOPE_API_KEY"],
            stream=False,
        ),
        formatter=DashScopeMultiAgentFormatter(),
    )


alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]

# Create a moderator agent
moderator = ReActAgent(
    name="Aggregator",
    sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
    model=DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    ),
    # Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
    formatter=DashScopeMultiAgentFormatter(),
)


# A structured output model for the moderator
class JudgeModel(BaseModel):
    """The structured output model for the moderator."""

    finished: bool = Field(
        description="Whether the debate is finished.",
    )
    correct_answer: str | None = Field(
        description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
        default=None,
    )


async def run_multiagent_debate() -> None:
    """Run the multi-agent debate workflow."""
    while True:
        # The reply messages in MsgHub from the participants will be broadcasted to all participants.
        async with MsgHub(participants=[alice, bob, moderator]):
            await alice(
                Msg(
                    "user",
                    "You are affirmative side, Please express your viewpoints.",
                    "user",
                ),
            )
            await bob(
                Msg(
                    "user",
                    "You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
                    "user",
                ),
            )

        # Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
        msg_judge = await moderator(
            Msg(
                "user",
                "Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
                "user",
            ),
            structured_model=JudgeModel,
        )

        if msg_judge.metadata.get("finished"):
            print(
                "\nThe debate is finished, and the correct answer is: ",
                msg_judge.metadata.get("correct_answer"),
            )
            break


asyncio.run(run_multiagent_debate())

Alice: Thank you. I will present the argument for the affirmative side regarding the number of revolutions circle A makes as it rolls around the outside of circle B, given that the radius of circle A is 1/3 the radius of circle B and there is no relative sliding between them.

To solve this, we need to consider the circumference of each circle and how many times the smaller circle (circle A) will rotate as it travels around the larger one (circle B). The key point here is that the distance traveled by the point of contact on circle A is equal to the circumference of the path it follows, which is the circumference of a circle with a radius equal to the sum of the radii of circles A and B.

Let's denote:
- r_A as the radius of circle A
- r_B as the radius of circle B

Given that r_A = 1/3 * r_B, then the total distance circle A travels is the circumference of the combined circles, which is 2 * π * (r_A + r_B).

The number of rotations N that circle A will make is the total distance divided by its own circumference, 2 * π * r_A. Hence, N = (2 * π * (r_A + r_B)) / (2 * π * r_A) = (r_A + r_B) / r_A.

Substituting r_A = 1/3 * r_B into the equation gives us N = (1/3 * r_B + r_B) / (1/3 * r_B) = (4/3 * r_B) / (1/3 * r_B) = 4.

Therefore, circle A will revolve 4 times in total as it rolls around circle B once. This is my position, and I'm open to further discussion or any counter-arguments.
Bob: Thank you. I will be presenting the argument for the negative side, but after reviewing the calculations and reasoning provided by the affirmative side, I find that the mathematics and logic are sound. The key points to consider are:

1. The distance traveled by circle A is equal to the circumference of a circle with a radius equal to the sum of the radii of circles A and B.
2. The number of rotations N that circle A makes is the total distance traveled divided by its own circumference.
3. Given that r_A = 1/3 * r_B, when we substitute this into the formula for N, we get N = 4.

There is no mistake in the calculation, and the conclusion that circle A will revolve 4 times as it rolls around circle B once is correct. Therefore, I cannot present a counter-argument that would change the outcome of the problem. The answer is indeed 4 revolutions.
Aggregator: The debate has concluded with both Alice and Bob agreeing on the same mathematical reasoning. They have correctly calculated that circle A will revolve 4 times as it rolls around the outside of circle B, given the radius of circle A is 1/3 the radius of circle B. There are no counter-arguments presented, and the solution provided is accurate. Therefore, the correct answer to the problem is 4 revolutions.

The debate is finished, and the correct answer is:  4 revolutions

Multi-Agent Debate¶

Further Reading¶