Multi-Agent Debate

Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.

We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.

import asyncio
import os

from pydantic import Field, BaseModel

from agentscope.agent import ReActAgent
from agentscope.formatter import (
    DashScopeMultiAgentFormatter,
    DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub

# Prepare a topic
topic = (
    "The two circles are externally tangent and there is no relative sliding. "
    "The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
    "around circle B one trip back to its starting point. How many times will "
    "circle A revolve in total?"
)


# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
    """Get a solver agent."""
    return ReActAgent(
        name=name,
        sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
        "debate competition. It's unnecessary to fully agree with "
        "each other's perspectives, as our objective is to find "
        "the correct answer. The debate topic is stated as "
        f"follows: {topic}.",
        model=DashScopeChatModel(
            model_name="qwen-max",
            api_key=os.environ["DASHSCOPE_API_KEY"],
            stream=False,
        ),
        formatter=DashScopeMultiAgentFormatter(),
    )


alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]

# Create a moderator agent
moderator = ReActAgent(
    name="Aggregator",
    sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
    model=DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    ),
    # Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
    formatter=DashScopeMultiAgentFormatter(),
)


# A structured output model for the moderator
class JudgeModel(BaseModel):
    """The structured output model for the moderator."""

    finished: bool = Field(
        description="Whether the debate is finished.",
    )
    correct_answer: str | None = Field(
        description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
        default=None,
    )


async def run_multiagent_debate() -> None:
    """Run the multi-agent debate workflow."""
    while True:
        # The reply messages in MsgHub from the participants will be broadcasted to all participants.
        async with MsgHub(participants=[alice, bob, moderator]):
            await alice(
                Msg(
                    "user",
                    "You are affirmative side, Please express your viewpoints.",
                    "user",
                ),
            )
            await bob(
                Msg(
                    "user",
                    "You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
                    "user",
                ),
            )

        # Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
        msg_judge = await moderator(
            Msg(
                "user",
                "Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
                "user",
            ),
            structured_model=JudgeModel,
        )

        if msg_judge.metadata.get("finished"):
            print(
                "\nThe debate is finished, and the correct answer is: ",
                msg_judge.metadata.get("correct_answer"),
            )
            break


asyncio.run(run_multiagent_debate())

Alice: Thank you, and I appreciate the opportunity to present my case. To start, let's define the problem more clearly: We have two circles, A and B, where circle A is externally tangent to circle B and rolls around it without sliding. The radius of circle A (r) is 1/3 the radius of circle B (R), so R = 3r.

When circle A rolls around circle B, it will complete one full trip around the circumference of circle B. The total distance that circle A travels along the path around circle B is equal to the circumference of the path, which is the circumference of a circle with a radius equal to the sum of the radii of A and B, i.e., R + r.

The circumference of this path is:
C_path = 2π(R + r) = 2π(3r + r) = 2π * 4r = 8πr

Now, for each revolution of circle A, it covers a distance equal to its own circumference, which is:
C_A = 2πr

To find out how many times circle A revolves, we divide the total distance traveled by the circumference of circle A:
Number of revolutions = C_path / C_A = (8πr) / (2πr) = 4

Therefore, circle A will revolve 4 times in total as it rolls around circle B back to its starting point.
Bob: Thank you for the introduction and for presenting your case, Alice. I appreciate the clear explanation of the problem and the calculations you've provided. However, I must disagree with the conclusion that circle A will revolve 4 times in total as it rolls around circle B back to its starting point.

Let's revisit the problem: We have two circles, A and B, where circle A is externally tangent to circle B and rolls around it without sliding. The radius of circle A (r) is 1/3 the radius of circle B (R), so R = 3r.

When circle A rolls around circle B, it travels along a path that has a circumference equal to 2π(R + r). As you correctly calculated, this is 8πr. Now, for each revolution of circle A, it covers a distance equal to its own circumference, which is 2πr.

However, there is an additional consideration: the relative motion between the two circles. When circle A rolls around circle B, it not only travels the distance of the path but also rotates due to the curvature of the path. This rotation is in addition to the revolutions caused by the linear distance traveled.

To find the number of rotations due to the curvature, we need to consider the ratio of the radii. Since the radius of A is 1/3 that of B, for every full revolution around B, circle A will rotate 3 times because the path length is 4 times the circumference of A, and A will make 3 additional rotations due to the difference in their radii.

Therefore, the total number of revolutions of circle A is:
- 1 revolution due to the path around B
- 3 additional revolutions due to the relative rotation

In total, circle A will revolve 4 - 1 = 3 times as it rolls around circle B back to its starting point.
/home/runner/work/agentscope/agentscope/src/agentscope/model/_dashscope_model.py:232: DeprecationWarning: 'required' is not supported by DashScope API. It will be converted to 'auto'.
  warnings.warn(
Aggregator: {
    "type": "tool_use",
    "name": "generate_response",
    "input": {
        "correct_answer": "4",
        "finished": true
    },
    "id": "call_99ada23a5ecc4cc7aabbd9"
}
system: {
    "type": "tool_result",
    "id": "call_99ada23a5ecc4cc7aabbd9",
    "name": "generate_response",
    "output": [
        {
            "type": "text",
            "text": "Successfully generated response."
        }
    ]
}
Aggregator: Thank you, Alice and Bob, for your detailed explanations and calculations. After carefully reviewing both arguments, the correct number of times that circle A will revolve as it rolls around circle B back to its starting point is 4.

Alice's calculation is accurate: The total distance that circle A travels is the circumference of the path, which is \(8\pi r\). Since each revolution of circle A covers a distance of \(2\pi r\), the number of revolutions is:

\[
\frac{8\pi r}{2\pi r} = 4
\]

Bob, while you correctly pointed out the importance of considering the relative motion between the two circles, the additional rotations due to the curvature are already accounted for in the path length. Therefore, the total number of revolutions is indeed 4, not 3.

Thus, the correct answer is 4. Thank you both for your participation and insightful contributions.

The debate is finished, and the correct answer is:  4

Further Reading

• Pipeline

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. EMNLP 2024.