Multi-Agent Debate

Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.

We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.

import asyncio
import os

from pydantic import Field, BaseModel

from agentscope.agent import ReActAgent
from agentscope.formatter import (
    DashScopeMultiAgentFormatter,
    DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub

# Prepare a topic
topic = (
    "The two circles are externally tangent and there is no relative sliding. "
    "The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
    "around circle B one trip back to its starting point. How many times will "
    "circle A revolve in total?"
)


# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
    """Get a solver agent."""
    return ReActAgent(
        name=name,
        sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
        "debate competition. It's unnecessary to fully agree with "
        "each other's perspectives, as our objective is to find "
        "the correct answer. The debate topic is stated as "
        f"follows: {topic}.",
        model=DashScopeChatModel(
            model_name="qwen-max",
            api_key=os.environ["DASHSCOPE_API_KEY"],
            stream=False,
        ),
        formatter=DashScopeMultiAgentFormatter(),
    )


alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]

# Create a moderator agent
moderator = ReActAgent(
    name="Aggregator",
    sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
    model=DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    ),
    # Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
    formatter=DashScopeMultiAgentFormatter(),
)


# A structured output model for the moderator
class JudgeModel(BaseModel):
    """The structured output model for the moderator."""

    finished: bool = Field(
        description="Whether the debate is finished.",
    )
    correct_answer: str | None = Field(
        description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
        default=None,
    )


async def run_multiagent_debate() -> None:
    """Run the multi-agent debate workflow."""
    while True:
        # The reply messages in MsgHub from the participants will be broadcasted to all participants.
        async with MsgHub(participants=[alice, bob, moderator]):
            await alice(
                Msg(
                    "user",
                    "You are affirmative side, Please express your viewpoints.",
                    "user",
                ),
            )
            await bob(
                Msg(
                    "user",
                    "You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
                    "user",
                ),
            )

        # Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
        msg_judge = await moderator(
            Msg(
                "user",
                "Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
                "user",
            ),
            structured_model=JudgeModel,
        )

        if msg_judge.metadata.get("finished"):
            print(
                "\nThe debate is finished, and the correct answer is: ",
                msg_judge.metadata.get("correct_answer"),
            )
            break


asyncio.run(run_multiagent_debate())

Alice: Thank you. As the affirmative side, I will present the argument that when a smaller circle (circle A) with a radius 1/3 that of a larger circle (circle B) rolls around the circumference of the larger circle without sliding, it will revolve 4 times in total by the time it returns to its starting point.

To understand this, let's break down the problem:

- Let the radius of circle B be \( R \).
- Then, the radius of circle A is \( \frac{R}{3} \).

The circumference of a circle is given by \( C = 2\pi r \), where \( r \) is the radius of the circle. Therefore, the circumferences of the two circles are:
- Circumference of circle B: \( C_B = 2\pi R \)
- Circumference of circle A: \( C_A = 2\pi \left(\frac{R}{3}\right) = \frac{2\pi R}{3} \)

When circle A rolls around circle B, it must travel a distance equal to the circumference of the path it follows. This path is not just the circumference of circle B but also includes the additional distance due to the radius of circle A. The center of circle A will trace out a circle with a radius of \( R + \frac{R}{3} = \frac{4R}{3} \). Hence, the length of the path that the center of circle A travels is:
- Path length: \( 2\pi \left(\frac{4R}{3}\right) = \frac{8\pi R}{3} \)

Now, to find out how many times circle A revolves, we divide the total path length by the circumference of circle A:
- Number of revolutions: \( \frac{\text{Path length}}{\text{Circumference of circle A}} = \frac{\frac{8\pi R}{3}}{\frac{2\pi R}{3}} = \frac{8\pi R}{3} \times \frac{3}{2\pi R} = 4 \)

Therefore, circle A will revolve 4 times as it rolls around circle B and returns to its starting point.
Bob: Thank you. As the negative side, I will argue that the number of times circle A revolves as it rolls around circle B and returns to its starting point is actually 3, not 4.

To clarify, let's revisit the setup and calculations:

- The radius of circle B is \( R \).
- The radius of circle A is \( \frac{R}{3} \).

The circumference of each circle:
- Circumference of circle B: \( C_B = 2\pi R \)
- Circumference of circle A: \( C_A = 2\pi \left(\frac{R}{3}\right) = \frac{2\pi R}{3} \)

When circle A rolls around circle B, the path traced by the center of circle A is a circle with a radius of \( R + \frac{R}{3} = \frac{4R}{3} \). The length of this path is:
- Path length: \( 2\pi \left(\frac{4R}{3}\right) = \frac{8\pi R}{3} \)

However, the key point here is to consider the relative motion of the circles. When circle A rolls without sliding, it must rotate an additional time due to the curvature of the path it follows. This is known as the "rolling without slipping" condition.

To find the number of revolutions, we need to consider both the linear distance traveled and the additional rotation due to the curvature. The total number of rotations is given by the ratio of the path length to the circumference of circle A, plus one additional revolution because of the rolling motion around the larger circle:

- Number of revolutions: \( \frac{\text{Path length}}{\text{Circumference of circle A}} + 1 = \frac{\frac{8\pi R}{3}}{\frac{2\pi R}{3}} + 1 = \frac{8\pi R}{3} \times \frac{3}{2\pi R} + 1 = 4 + 1 = 5 - 2 = 3 \)

Thus, when circle A completes one trip around circle B, it will have revolved 3 times in total.
/home/runner/work/agentscope/agentscope/src/agentscope/model/_dashscope_model.py:194: DeprecationWarning: 'required' is not supported by DashScope API. It will be converted to 'auto'.
  warnings.warn(
Aggregator: {
    "type": "tool_use",
    "name": "generate_response",
    "input": {
        "correct_answer": "4",
        "finished": true
    },
    "id": "call_bf5916be44aa4c08bfcb68"
}
system: {
    "type": "tool_result",
    "id": "call_bf5916be44aa4c08bfcb68",
    "name": "generate_response",
    "output": [
        {
            "type": "text",
            "text": "Successfully generated response."
        }
    ]
}
Aggregator: The debate has concluded, and after carefully considering the arguments presented by both sides, I can provide the correct answer.

Alice's argument is accurate. When circle A, with a radius 1/3 that of circle B, rolls around circle B without sliding, it will indeed revolve 4 times in total by the time it returns to its starting point. The key points in Alice's reasoning are:

- The path length traced by the center of circle A is \( \frac{8\pi R}{3} \).
- The number of revolutions is calculated by dividing this path length by the circumference of circle A, which is \( \frac{2\pi R}{3} \).
- This results in \( \frac{\frac{8\pi R}{3}}{\frac{2\pi R}{3}} = 4 \) revolutions.

Bob's argument included an additional revolution due to the curvature, but this is not necessary for the calculation as the rolling without slipping condition already accounts for the rotations along the path. Therefore, the correct number of revolutions is 4.

Thus, the correct answer is 4.

The debate is finished, and the correct answer is:  4

Further Reading

• Pipeline

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. EMNLP 2024.