Note
Go to the end to download the full example code.
Multi-Agent Debate¶
Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator. Typically, the solvers generate and exchange their answers, while the aggregator collects and summarizes the answers.
We implement the examples in EMNLP 2024, where two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history. At each round a moderator agent will decide whether the correct answer can be obtained in the current iteration.
import asyncio
import os
from pydantic import Field, BaseModel
from agentscope.agent import ReActAgent
from agentscope.formatter import (
DashScopeMultiAgentFormatter,
DashScopeChatFormatter,
)
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub
# Prepare a topic
topic = (
"The two circles are externally tangent and there is no relative sliding. "
"The radius of circle A is 1/3 the radius of circle B. Circle A rolls "
"around circle B one trip back to its starting point. How many times will "
"circle A revolve in total?"
)
# Create two debater agents, Alice and Bob, who will discuss the topic.
def create_solver_agent(name: str) -> ReActAgent:
"""Get a solver agent."""
return ReActAgent(
name=name,
sys_prompt=f"You're a debater named {name}. Hello and welcome to the "
"debate competition. It's unnecessary to fully agree with "
"each other's perspectives, as our objective is to find "
"the correct answer. The debate topic is stated as "
f"follows: {topic}.",
model=DashScopeChatModel(
model_name="qwen-max",
api_key=os.environ["DASHSCOPE_API_KEY"],
stream=False,
),
formatter=DashScopeMultiAgentFormatter(),
)
alice, bob = [create_solver_agent(name) for name in ["Alice", "Bob"]]
# Create a moderator agent
moderator = ReActAgent(
name="Aggregator",
sys_prompt=f"""You're a moderator. There will be two debaters involved in a debate competition. They will present their answer and discuss their perspectives on the topic:
``````
{topic}
``````
At the end of each round, you will evaluate both sides' answers and decide which one is correct.""",
model=DashScopeChatModel(
model_name="qwen-max",
api_key=os.environ["DASHSCOPE_API_KEY"],
stream=False,
),
# Use multiagent formatter because the moderator will receive messages from more than a user and an assistant
formatter=DashScopeMultiAgentFormatter(),
)
# A structured output model for the moderator
class JudgeModel(BaseModel):
"""The structured output model for the moderator."""
finished: bool = Field(
description="Whether the debate is finished.",
)
correct_answer: str | None = Field(
description="The correct answer to the debate topic, only if the debate is finished. Otherwise, leave it as None.",
default=None,
)
async def run_multiagent_debate() -> None:
"""Run the multi-agent debate workflow."""
while True:
# The reply messages in MsgHub from the participants will be broadcasted to all participants.
async with MsgHub(participants=[alice, bob, moderator]):
await alice(
Msg(
"user",
"You are affirmative side, Please express your viewpoints.",
"user",
),
)
await bob(
Msg(
"user",
"You are negative side. You disagree with the affirmative side. Provide your reason and answer.",
"user",
),
)
# Alice and Bob doesn't need to know the moderator's message, so moderator is called outside the MsgHub.
msg_judge = await moderator(
Msg(
"user",
"Now you have heard the answers from the others, have the debate finished, and can you get the correct answer?",
"user",
),
structured_model=JudgeModel,
)
if msg_judge.metadata.get("finished"):
print(
"\nThe debate is finished, and the correct answer is: ",
msg_judge.metadata.get("correct_answer"),
)
break
asyncio.run(run_multiagent_debate())
Alice: As the affirmative side, I will argue that circle A, with a radius 1/3 that of circle B, will revolve 4 times in total when it rolls around the circumference of circle B without sliding.
To understand this, let's break down the problem:
- Let the radius of circle B be \( R \).
- The radius of circle A is then \( \frac{1}{3}R \).
When circle A rolls around circle B, the distance it needs to travel is the circumference of the path it follows. This path is the circumference of a circle with a radius equal to the sum of the radii of circles A and B, which is \( R + \frac{1}{3}R = \frac{4}{3}R \).
The circumference of this path is:
\[ C_{path} = 2\pi \left(\frac{4}{3}R\right) = \frac{8\pi R}{3} \]
The circumference of circle A, which is the distance it covers in one revolution, is:
\[ C_A = 2\pi \left(\frac{1}{3}R\right) = \frac{2\pi R}{3} \]
To find out how many times circle A revolves, we divide the total distance traveled by the circumference of circle A:
\[ \text{Number of Revolutions} = \frac{C_{path}}{C_A} = \frac{\frac{8\pi R}{3}}{\frac{2\pi R}{3}} = \frac{8\pi R}{3} \times \frac{3}{2\pi R} = 4 \]
Therefore, circle A will revolve 4 times as it rolls around circle B.
Alice: In conclusion, when circle A, whose radius is 1/3 that of circle B, rolls around the outside of circle B without any relative sliding, it will complete 4 full revolutions.
Bob: The conclusion that circle A will complete 4 full revolutions as it rolls around the outside of circle B, whose radius is three times larger, is incorrect. Let's analyze this situation using the concept of circumference and the relationship between the two circles.
Given:
- The radius of circle A (r_A) is 1/3 the radius of circle B (r_B).
- Therefore, r_A = r_B / 3.
When circle A rolls around circle B without slipping, the distance it travels along the path is equal to the circumference of the path it follows. This path is essentially a circle with a radius equal to the sum of the radii of both circles, which is r_A + r_B.
The number of revolutions (N) that circle A makes is given by the ratio of the path's circumference to the circumference of circle A:
\[ N = \frac{ ext{Circumference of the path}}{ ext{Circumference of circle A}} \]
\[ N = \frac{2\pi(r_A + r_B)}{2\pi r_A} \]
Since r_A = r_B / 3, we can substitute this into the equation:
\[ N = \frac{2\pi((r_B / 3) + r_B)}{2\pi (r_B / 3)} \]
\[ N = \frac{(r_B / 3 + r_B)}{(r_B / 3)} \]
\[ N = \frac{(r_B / 3 + 3r_B / 3)}{(r_B / 3)} \]
\[ N = \frac{4r_B / 3}{r_B / 3} \]
\[ N = 4 \]
However, this calculation only accounts for the number of times the center of circle A would revolve around the center of circle B. We must also consider the rotation of circle A about its own center. As circle A rolls around, it will rotate once for every full circuit around circle B due to the rolling motion. Thus, in addition to the 4 revolutions calculated above, there is an extra revolution because of the rolling motion itself.
Therefore, the total number of revolutions is actually 4 (from the orbital motion) plus 1 (from the rolling motion), which gives us a total of 5 revolutions. Hence, I disagree with the affirmative side's conclusion that it is 4, and my answer is 5.
Aggregator: The debate has concluded with a thorough analysis from both sides. Bob's explanation takes into account the correct mathematical approach and the additional revolution due to the rolling motion of circle A. Therefore, the correct answer is that circle A will revolve 5 times in total as it rolls around the outside of circle B without slipping. The key point is that we must consider both the orbital revolutions (4) and the self-rotation (1) of circle A.
The debate is finished, and the correct answer is: 5
Further Reading¶
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. EMNLP 2024.
Total running time of the script: (0 minutes 44.325 seconds)