Multi-Agent Debate¶

Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.

We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.

import asyncio
import os

from pydantic import Field, BaseModel

from agentscope.agent import ReActAgent
from agentscope.formatter import (
    DashScopeMultiAgentFormatter,
    DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub

# Prepare a topic
topic = (
    "The two circles are externally tangent and there is no relative sliding. "
    "The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
    "around circle B one trip back to its starting point. How many times will "
    "circle A revolve in total?"
)


# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
    """Get a solver agent."""
    return ReActAgent(
        name=name,
        sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
        "debate competition. It's unnecessary to fully agree with "
        "each other's perspectives, as our objective is to find "
        "the correct answer. The debate topic is stated as "
        f"follows: {topic}.",
        model=DashScopeChatModel(
            model_name="qwen-max",
            api_key=os.environ["DASHSCOPE_API_KEY"],
            stream=False,
        ),
        formatter=DashScopeMultiAgentFormatter(),
    )


alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]

# Create a moderator agent
moderator = ReActAgent(
    name="Aggregator",
    sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
    model=DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    ),
    # Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
    formatter=DashScopeMultiAgentFormatter(),
)


# A structured output model for the moderator
class JudgeModel(BaseModel):
    """The structured output model for the moderator."""

    finished: bool = Field(
        description="Whether the debate is finished.",
    )
    correct_answer: str | None = Field(
        description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
        default=None,
    )


async def run_multiagent_debate() -> None:
    """Run the multi-agent debate workflow."""
    while True:
        # The reply messages in MsgHub from the participants will be broadcasted to all participants.
        async with MsgHub(participants=[alice, bob, moderator]):
            await alice(
                Msg(
                    "user",
                    "You are affirmative side, Please express your viewpoints.",
                    "user",
                ),
            )
            await bob(
                Msg(
                    "user",
                    "You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
                    "user",
                ),
            )

        # Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
        msg_judge = await moderator(
            Msg(
                "user",
                "Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
                "user",
            ),
            structured_model=JudgeModel,
        )

        if msg_judge.metadata.get("finished"):
            print(
                "\nThe debate is finished, and the correct answer is: ",
                msg_judge.metadata.get("correct_answer"),
            )
            break


asyncio.run(run_multiagent_debate())

Alice: Thank you. As the affirmative side, I will argue that when circle A, with a radius 1/3 that of circle B, rolls around the circumference of circle B without sliding, it will revolve 4 times in total by the time it returns to its starting point.

To understand this, let's break down the problem:

- Let the radius of circle B be \( r \).
- The radius of circle A is \( \frac{1}{3}r \).

The circumference of circle B (the path that circle A follows) is \( 2\pi r \). Since circle A is rolling along this path, it must cover a distance equal to the circumference of circle B.

Now, the circumference of circle A is \( 2\pi \left(\frac{1}{3}r\right) = \frac{2\pi r}{3} \). To find out how many times circle A revolves, we divide the total distance traveled (which is the circumference of circle B) by the circumference of circle A:

\[ \text{Number of revolutions} = \frac{\text{Circumference of circle B}}{\text{Circumference of circle A}} = \frac{2\pi r}{\frac{2\pi r}{3}} = 3 \]

However, this calculation only accounts for the revolution due to the path length. We also need to consider the additional revolution that occurs because circle A is rotating around the center of circle B. This is an extra full revolution, which brings the total to 4.

Therefore, circle A will revolve 4 times in total as it rolls around circle B and returns to its starting point.
Bob: Thank you, and I will take the position of the negative side. I disagree with the assertion that circle A will revolve 4 times in total as it rolls around circle B.

Let's revisit the problem with a focus on the mechanics of the rotation:

- The radius of circle B is \( r \).
- The radius of circle A is \( \frac{1}{3}r \).

The circumference of circle B (the path that circle A follows) is indeed \( 2\pi r \). Circle A, with a circumference of \( \frac{2\pi r}{3} \), must cover this distance as it rolls without sliding.

To determine how many times circle A revolves, we calculate the number of its circumferences that fit into the circumference of circle B:

\[ \text{Number of revolutions} = \frac{\text{Circumference of circle B}}{\text{Circumference of circle A}} = \frac{2\pi r}{\frac{2\pi r}{3}} = 3 \]

This calculation correctly gives us the number of times circle A rotates about its own center as it travels along the circumference of circle B. However, there is a common misconception that an additional full revolution occurs because circle A is also rotating around the center of circle B. This is not correct, as the rolling motion already accounts for the path taken around circle B.

Therefore, circle A will only revolve 3 times in total as it rolls around circle B and returns to its starting point. There is no need to add an extra revolution, as the 3 revolutions calculated are sufficient to account for both the linear path and the rotational motion.
/home/runner/work/agentscope/agentscope/src/agentscope/model/_dashscope_model.py:232: DeprecationWarning: 'required' is not supported by DashScope API. It will be converted to 'auto'.
  warnings.warn(
Aggregator: {
    "type": "tool_use",
    "name": "generate_response",
    "input": {
        "correct_answer": "3",
        "finished": true
    },
    "id": "call_0b0520186e7d42f09722cf"
}
system: {
    "type": "tool_result",
    "id": "call_0b0520186e7d42f09722cf",
    "name": "generate_response",
    "output": [
        {
            "type": "text",
            "text": "Successfully generated response."
        }
    ]
}
Aggregator: The debate has concluded, and it is time to determine the correct answer.

Alice argued that circle A would revolve 4 times in total: 3 times due to the path length (the circumference of circle B) and an additional full revolution because circle A is rotating around the center of circle B.

Bob, on the other hand, maintained that circle A would only revolve 3 times. He pointed out that the 3 revolutions calculated from the ratio of the circumferences already account for both the linear path and the rotational motion. Bob's argument correctly addresses the mechanics of the rotation, where the rolling motion itself encompasses the complete path around circle B without the need for an extra full revolution.

After careful consideration, the correct answer is that circle A will revolve 3 times in total as it rolls around circle B and returns to its starting point. Therefore, Bob's position is the correct one.

The debate is finished, and the correct answer is:  3

Multi-Agent Debate¶

Further Reading¶