Multi-Agent Debate¶

Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.

We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.

import asyncio
import os

from pydantic import Field, BaseModel

from agentscope.agent import ReActAgent
from agentscope.formatter import (
    DashScopeMultiAgentFormatter,
    DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub

# Prepare a topic
topic = (
    "The two circles are externally tangent and there is no relative sliding. "
    "The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
    "around circle B one trip back to its starting point. How many times will "
    "circle A revolve in total?"
)


# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
    """Get a solver agent."""
    return ReActAgent(
        name=name,
        sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
        "debate competition. It's unnecessary to fully agree with "
        "each other's perspectives, as our objective is to find "
        "the correct answer. The debate topic is stated as "
        f"follows: {topic}.",
        model=DashScopeChatModel(
            model_name="qwen-max",
            api_key=os.environ["DASHSCOPE_API_KEY"],
            stream=False,
        ),
        formatter=DashScopeMultiAgentFormatter(),
    )


alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]

# Create a moderator agent
moderator = ReActAgent(
    name="Aggregator",
    sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
    model=DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    ),
    # Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
    formatter=DashScopeMultiAgentFormatter(),
)


# A structured output model for the moderator
class JudgeModel(BaseModel):
    """The structured output model for the moderator."""

    finished: bool = Field(
        description="Whether the debate is finished.",
    )
    correct_answer: str | None = Field(
        description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
        default=None,
    )


async def run_multiagent_debate() -> None:
    """Run the multi-agent debate workflow."""
    while True:
        # The reply messages in MsgHub from the participants will be broadcasted to all participants.
        async with MsgHub(participants=[alice, bob, moderator]):
            await alice(
                Msg(
                    "user",
                    "You are affirmative side, Please express your viewpoints.",
                    "user",
                ),
            )
            await bob(
                Msg(
                    "user",
                    "You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
                    "user",
                ),
            )

        # Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
        msg_judge = await moderator(
            Msg(
                "user",
                "Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
                "user",
            ),
            structured_model=JudgeModel,
        )

        if msg_judge.metadata.get("finished"):
            print(
                "\nThe debate is finished, and the correct answer is: ",
                msg_judge.metadata.get("correct_answer"),
            )
            break


asyncio.run(run_multiagent_debate())

Alice: Thank you. As the affirmative side, I will present the argument that when circle A, with a radius 1/3 that of circle B, rolls around the outside of circle B without sliding, it will revolve 4 times in total to return to its starting point.

Here's the reasoning:

Let's denote the radius of circle B as R and the radius of circle A as r. Given that r = R/3, we can determine the circumference of each circle. The circumference of circle B (C_B) is 2πR, and the circumference of circle A (C_A) is 2πr or 2π(R/3).

When circle A rolls around circle B, the distance it travels along the path is equal to the circumference of the larger circle plus the additional distance due to the smaller circle's own rotation. Since the circles are externally tangent, the path length for one complete trip around circle B is the circumference of a circle with a radius of R + r, which is 2π(R + r) or 2π(R + R/3) = 2π(4R/3).

The number of revolutions (N) that circle A makes is the total distance traveled divided by the circumference of circle A:

N = (path length) / C_A
N = [2π(4R/3)] / [2π(R/3)]
N = (4R/3) / (R/3)
N = 4

Therefore, circle A will revolve 4 times in total as it rolls around the outside of circle B once.
Bob: Thank you. As the negative side, I will present an argument that challenges the conclusion of 4 revolutions for circle A as it rolls around circle B. The key point to consider is the relative motion and the concept of a fixed point on circle A.

Let's re-examine the scenario with a different perspective: When circle A rolls around circle B, we can think of a fixed point on the circumference of circle A. As circle A completes its path around circle B, this fixed point will trace out a path known as a "hypocycloid" if it were rolling inside, but in our case, it's an "epicycloid" because it's rolling outside.

The number of times circle A revolves is not only dependent on the distance it travels along the path but also on the additional rotation due to the curvature of the path. In this case, the path length is indeed 2π(4R/3), as Alice correctly calculated. However, when circle A completes one full trip around circle B, it has to rotate an extra time to align with its starting orientation because of the external tangency.

To understand this, imagine a point on the edge of circle A. As it rolls, this point will touch the surface at the same relative position after each complete revolution. After traveling the full path, the point must have made one additional revolution to return to its original orientation. This means that the total number of revolutions is:

N = (path length) / C_A + 1
N = [2π(4R/3)] / [2π(R/3)] + 1
N = (4R/3) / (R/3) + 1
N = 4 + 1
N = 5

Therefore, circle A will revolve 5 times in total as it rolls around the outside of circle B once.
/home/runner/work/agentscope/agentscope/src/agentscope/model/_dashscope_model.py:232: DeprecationWarning: 'required' is not supported by DashScope API. It will be converted to 'auto'.
  warnings.warn(
Aggregator: {
    "type": "tool_use",
    "name": "generate_response",
    "input": {
        "correct_answer": "4",
        "finished": true
    },
    "id": "call_105fcd1dfc544a4c8eebfb"
}
system: {
    "type": "tool_result",
    "id": "call_105fcd1dfc544a4c8eebfb",
    "name": "generate_response",
    "output": [
        {
            "type": "text",
            "text": "Successfully generated response."
        }
    ]
}
Aggregator: The debate has concluded, and after reviewing the arguments presented by both Alice and Bob, the correct answer is that circle A will revolve 4 times in total as it rolls around the outside of circle B once.

Alice's argument correctly accounts for the relative motion and the path length traveled by circle A. The calculation she provided is accurate, and it shows that the number of revolutions (N) is indeed 4 when considering the circumference of the larger path and the circumference of circle A.

Bob's argument introduces an additional rotation due to the external tangency, which would be a valid consideration if we were discussing the orientation of a fixed point on circle A. However, in the context of counting the number of full revolutions, this extra rotation does not add to the count of complete turns. Therefore, the additional +1 in Bob's calculation is not necessary for determining the number of revolutions.

Thus, the final answer is 4 revolutions.

The debate is finished, and the correct answer is:  4

Multi-Agent Debate¶

Further Reading¶