Multi-Agent Debate¶

Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.

We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.

import asyncio
import os

from pydantic import Field, BaseModel

from agentscope.agent import ReActAgent
from agentscope.formatter import (
    DashScopeMultiAgentFormatter,
    DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub

# Prepare a topic
topic = (
    "The two circles are externally tangent and there is no relative sliding. "
    "The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
    "around circle B one trip back to its starting point. How many times will "
    "circle A revolve in total?"
)


# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
    """Get a solver agent."""
    return ReActAgent(
        name=name,
        sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
        "debate competition. It's unnecessary to fully agree with "
        "each other's perspectives, as our objective is to find "
        "the correct answer. The debate topic is stated as "
        f"follows: {topic}.",
        model=DashScopeChatModel(
            model_name="qwen-max",
            api_key=os.environ["DASHSCOPE_API_KEY"],
            stream=False,
        ),
        formatter=DashScopeMultiAgentFormatter(),
    )


alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]

# Create a moderator agent
moderator = ReActAgent(
    name="Aggregator",
    sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
    model=DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    ),
    # Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
    formatter=DashScopeMultiAgentFormatter(),
)


# A structured output model for the moderator
class JudgeModel(BaseModel):
    """The structured output model for the moderator."""

    finished: bool = Field(
        description="Whether the debate is finished.",
    )
    correct_answer: str | None = Field(
        description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
        default=None,
    )


async def run_multiagent_debate() -> None:
    """Run the multi-agent debate workflow."""
    while True:
        # The reply messages in MsgHub from the participants will be broadcasted to all participants.
        async with MsgHub(participants=[alice, bob, moderator]):
            await alice(
                Msg(
                    "user",
                    "You are affirmative side, Please express your viewpoints.",
                    "user",
                ),
            )
            await bob(
                Msg(
                    "user",
                    "You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
                    "user",
                ),
            )

        # Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
        msg_judge = await moderator(
            Msg(
                "user",
                "Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
                "user",
            ),
            structured_model=JudgeModel,
        )

        if msg_judge.metadata.get("finished"):
            print(
                "\nThe debate is finished, and the correct answer is: ",
                msg_judge.metadata.get("correct_answer"),
            )
            break


asyncio.run(run_multiagent_debate())

Alice: Thank you for the introduction. As the affirmative side, I will present my argument on the number of times circle A will revolve around its own center as it rolls around the circumference of circle B.

Given that the radius of circle A (r) is 1/3 the radius of circle B (R), we can express this relationship as r = R/3. When circle A rolls around circle B, it will travel a distance equal to the circumference of circle B, which is 2πR.

Since there is no relative sliding, the distance traveled by the point of contact on circle A is also 2πR. The circumference of circle A is 2πr, and since r = R/3, the circumference of circle A is 2π(R/3) or (2/3)πR.

To find out how many times circle A revolves, we divide the total distance traveled by the circumference of circle A:

Number of revolutions = Distance traveled / Circumference of circle A
Number of revolutions = 2πR / ((2/3)πR)
Number of revolutions = 2πR * (3/(2πR))
Number of revolutions = 3

Therefore, circle A will revolve 3 times in total as it rolls around circle B once.
Bob: Thank you, Alice, for your well-structured argument. However, I must respectfully disagree with the conclusion that circle A will revolve 3 times in total as it rolls around circle B. The key point that needs to be addressed is the concept of a rolling circle's rotation relative to an external fixed point, which is different from the number of rotations about its own center.

When circle A rolls around circle B without slipping, it does not only rotate due to its movement along the circumference of circle B but also because of the curvature of circle B. This additional rotation is often overlooked and can be understood by considering the path of a single point on the circumference of circle A.

As circle A rolls around circle B, it has to complete one full revolution around the center of circle B, which adds an extra rotation. Thus, the total number of revolutions is the sum of the revolutions due to the distance traveled (which you correctly calculated as 3) and the one extra revolution due to the circular path around circle B.

Therefore, the total number of revolutions of circle A as it rolls around circle B is 3 (from the distance traveled) + 1 (from the circular path) = 4. Hence, circle A will actually revolve 4 times in total.
Aggregator: {
    "type": "tool_use",
    "name": "generate_response",
    "input": {
        "finished": true,
        "correct_answer": "4"
    },
    "id": "call_0a88809d1a8c4e0aa8a999"
}
system: {
    "type": "tool_result",
    "id": "call_0a88809d1a8c4e0aa8a999",
    "name": "generate_response",
    "output": [
        {
            "type": "text",
            "text": "Successfully generated response."
        }
    ]
}
Aggregator: Thank you, Alice and Bob, for your insightful arguments. After carefully evaluating both sides, the correct answer is that circle A will revolve 4 times in total as it rolls around circle B once.

Alice's calculation correctly determined that circle A would make 3 revolutions based on the distance traveled along the circumference of circle B. However, Bob correctly pointed out that an additional revolution must be accounted for due to the circular path that circle A follows around circle B. This extra revolution is a result of the fact that, as circle A rolls around the larger circle, it also rotates about the center of circle B, adding one more complete turn.

Therefore, the total number of revolutions is 3 (from the rolling along the circumference) + 1 (from the rotation around the center of circle B) = 4. The final answer is 4.

The debate is finished, and the correct answer is:  4

Multi-Agent Debate¶

Further Reading¶