Multi-Agent Debate

Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.

We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.

import asyncio
import os

from pydantic import Field, BaseModel

from agentscope.agent import ReActAgent
from agentscope.formatter import (
    DashScopeMultiAgentFormatter,
    DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub

# Prepare a topic
topic = (
    "The two circles are externally tangent and there is no relative sliding. "
    "The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
    "around circle B one trip back to its starting point. How many times will "
    "circle A revolve in total?"
)


# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
    """Get a solver agent."""
    return ReActAgent(
        name=name,
        sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
        "debate competition. It's unnecessary to fully agree with "
        "each other's perspectives, as our objective is to find "
        "the correct answer. The debate topic is stated as "
        f"follows: {topic}.",
        model=DashScopeChatModel(
            model_name="qwen-max",
            api_key=os.environ["DASHSCOPE_API_KEY"],
            stream=False,
        ),
        formatter=DashScopeMultiAgentFormatter(),
    )


alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]

# Create a moderator agent
moderator = ReActAgent(
    name="Aggregator",
    sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
    model=DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    ),
    # Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
    formatter=DashScopeMultiAgentFormatter(),
)


# A structured output model for the moderator
class JudgeModel(BaseModel):
    """The structured output model for the moderator."""

    finished: bool = Field(
        description="Whether the debate is finished.",
    )
    correct_answer: str | None = Field(
        description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
        default=None,
    )


async def run_multiagent_debate() -> None:
    """Run the multi-agent debate workflow."""
    while True:
        # The reply messages in MsgHub from the participants will be broadcasted to all participants.
        async with MsgHub(participants=[alice, bob, moderator]):
            await alice(
                Msg(
                    "user",
                    "You are affirmative side, Please express your viewpoints.",
                    "user",
                ),
            )
            await bob(
                Msg(
                    "user",
                    "You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
                    "user",
                ),
            )

        # Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
        msg_judge = await moderator(
            Msg(
                "user",
                "Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
                "user",
            ),
            structured_model=JudgeModel,
        )

        if msg_judge.metadata.get("finished"):
            print(
                "\nThe debate is finished, and the correct answer is: ",
                msg_judge.metadata.get("correct_answer"),
            )
            break


asyncio.run(run_multiagent_debate())

Alice: Thank you for the introduction. As the affirmative side, I will argue that when circle A, with a radius 1/3 that of circle B, rolls around the outside of circle B without sliding, it will complete 4 full revolutions by the time it returns to its starting point.

To understand this, let's break down the problem:

- Let the radius of circle B be \( R \).
- The radius of circle A is then \( \frac{1}{3}R \).

When circle A rolls around circle B, the path it follows is a circular path with a radius equal to the sum of the radii of both circles, which is \( R + \frac{1}{3}R = \frac{4}{3}R \). The circumference of this path (the distance circle A travels) is therefore:
\[ C_{\text{path}} = 2\pi \left(\frac{4}{3}R\right) = \frac{8}{3}\pi R \]

The circumference of circle A itself, which is the distance it covers in one revolution, is:
\[ C_A = 2\pi \left(\frac{1}{3}R\right) = \frac{2}{3}\pi R \]

Now, to find out how many times circle A revolves as it rolls around circle B, we divide the total distance traveled by the circumference of circle A:
\[ \text{Number of Revolutions} = \frac{C_{\text{path}}}{C_A} = \frac{\frac{8}{3}\pi R}{\frac{2}{3}\pi R} = \frac{8}{3} \times \frac{3}{2} = 4 \]

Therefore, circle A will revolve 4 times as it rolls around circle B and returns to its starting position.
Bob: Thank you, Alice, for your detailed explanation. I will present the negative side of the argument and provide a different perspective on the problem.

The key to understanding the number of revolutions that circle A makes as it rolls around circle B lies in considering both the translational and rotational motion of circle A. When circle A rolls around the outside of circle B without sliding, it not only moves along the path but also rotates about its own center.

Let's re-examine the given information:
- The radius of circle B is \( R \).
- The radius of circle A is \( \frac{1}{3}R \).

The path that circle A follows is a circle with a radius of \( R + \frac{1}{3}R = \frac{4}{3}R \), and the circumference of this path is:
\[ C_{\text{path}} = 2\pi \left(\frac{4}{3}R\right) = \frac{8}{3}\pi R \]

However, as circle A rolls around circle B, it also rotates around its own axis. For every full trip around circle B, circle A will have to rotate an additional time to align its starting point back to the original orientation. This extra rotation is due to the fact that the point of contact between the two circles changes continuously as circle A rolls around circle B.

Thus, the total number of rotations (or revolutions) of circle A can be calculated by considering the path length and the extra rotation:

- The number of times circle A would revolve based on the path length alone is:
\[ \text{Number of Revolutions from Path} = \frac{\frac{8}{3}\pi R}{\frac{2}{3}\pi R} = 4 \]

- Adding the one extra revolution due to the continuous change in the point of contact, we get:
\[ \text{Total Number of Revolutions} = 4 + 1 = 5 \]

Therefore, my argument is that circle A will actually complete 5 full revolutions by the time it returns to its starting point.
/home/runner/work/agentscope/agentscope/src/agentscope/model/_dashscope_model.py:231: DeprecationWarning: 'required' is not supported by DashScope API. It will be converted to 'auto'.
  warnings.warn(
Aggregator: {
    "type": "tool_use",
    "name": "generate_response",
    "input": {
        "correct_answer": "4",
        "finished": true
    },
    "id": "call_f47e5a8a5fe94410b61b8a"
}
system: {
    "type": "tool_result",
    "id": "call_f47e5a8a5fe94410b61b8a",
    "name": "generate_response",
    "output": [
        {
            "type": "text",
            "text": "Successfully generated response."
        }
    ]
}
Aggregator: Thank you, Alice and Bob, for your insightful arguments. After carefully considering both perspectives, the correct answer is that circle A will complete 4 full revolutions as it rolls around circle B and returns to its starting point.

Alice's argument correctly calculates the number of revolutions based on the path length and the circumference of circle A. The key point is that the total distance traveled by circle A along the path (which is the circumference of a circle with radius \(\frac{4}{3}R\)) divided by the circumference of circle A itself gives the number of revolutions. This calculation yields 4, and no additional revolution is needed because the problem specifies "no relative sliding," meaning the point of contact between the two circles does not affect the number of rotations.

Bob's argument introduces an extra rotation due to the continuous change in the point of contact, but this is not necessary in this context. The problem specifies that there is no relative sliding, which means the point of contact does not add an extra revolution.

Therefore, the correct answer is 4 full revolutions.

The debate is finished, and the correct answer is:  4

Further Reading

• Pipeline

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. EMNLP 2024.