Model¶

In this tutorial, we introduce the model APIs integrated in AgentScope, how to use them and how to integrate new model APIs. The supported model APIs and providers include:

API	Class	Compatible	Streaming	Tools	Vision	Reasoning
OpenAI	`OpenAIChatModel`	vLLM, DeepSeek	✅	✅	✅	✅
DashScope	`DashScopeChatModel`		✅	✅	✅	✅
Anthropic	`AnthropicChatModel`		✅	✅	✅	✅
Gemini	`GeminiChatModel`		✅	✅	✅	✅
Ollama	`OllamaChatModel`		✅	✅	✅	✅

To provide unified model interfaces, the above model classes has the following common methods:

The first three arguments of the __call__ method are messages , tools and tool_choice, representing the input messages, JSON schema of tool functions, and tool selection mode, respectively.
The return type are either a ChatResponse instance or an async generator of ChatResponse in streaming mode.

Note

Different model APIs differ in the input message format, refer to Prompt Formatter for more details.

The ChatResponse instance contains the generated thinking/text/tool use content, identity, created time and usage information.

import asyncio
import json
import os

from agentscope.message import TextBlock, ToolUseBlock, ThinkingBlock, Msg
from agentscope.model import ChatResponse, DashScopeChatModel

response = ChatResponse(
    content=[
        ThinkingBlock(
            type="thinking",
            thinking="I should search for AgentScope on Google.",
        ),
        TextBlock(type="text", text="I'll search for AgentScope on Google."),
        ToolUseBlock(
            type="tool_use",
            id="642n298gjna",
            name="google_search",
            input={"query": "AgentScope?"},
        ),
    ],
)

print(response)

ChatResponse(content=[{'type': 'thinking', 'thinking': 'I should search for AgentScope on Google.'}, {'type': 'text', 'text': "I'll search for AgentScope on Google."}, {'type': 'tool_use', 'id': '642n298gjna', 'name': 'google_search', 'input': {'query': 'AgentScope?'}}], id='2025-09-04 07:29:16.567_9a4e01', created_at='2025-09-04 07:29:16.567', type='chat', usage=None, metadata=None)

Taking DashScopeChatModel as an example, we can use it to create a chat model instance and call it with messages and tools:

async def example_model_call() -> None:
    """An example of using the DashScopeChatModel."""
    model = DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=False,
    )

    res = await model(
        messages=[
            {"role": "user", "content": "Hi!"},
        ],
    )

    # You can directly create a ``Msg`` object with the response content
    msg_res = Msg("Friday", res.content, "assistant")

    print("The response:", res)
    print("The response as Msg:", msg_res)


asyncio.run(example_model_call())

The response: ChatResponse(content=[{'type': 'text', 'text': 'Hello! How can I assist you today?'}], id='2025-09-04 07:29:17.882_b62784', created_at='2025-09-04 07:29:17.882', type='chat', usage=ChatUsage(input_tokens=10, output_tokens=9, time=1.313802, type='chat'), metadata=None)
The response as Msg: Msg(id='kUpCyUeoFac4NqNb2KeVYY', name='Friday', content=[{'type': 'text', 'text': 'Hello! How can I assist you today?'}], role='assistant', metadata=None, timestamp='2025-09-04 07:29:17.882', invocation_id='None')

Streaming¶

To enable streaming model, set the stream parameter in the model constructor to True. When streaming is enabled, the __call__ method will return an async generator that yields ChatResponse instances as they are generated by the model.

Note

The streaming mode in AgentScope is designed to be cumulative, meaning the content in each chunk contains all the previous content plus the newly generated content.

async def example_streaming() -> None:
    """An example of using the streaming model."""
    model = DashScopeChatModel(
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=True,
    )

    generator = await model(
        messages=[
            {
                "role": "user",
                "content": "Count from 1 to 20, and just report the number without any other information.",
            },
        ],
    )
    print("The type of the response:", type(generator))

    i = 0
    async for chunk in generator:
        print(f"Chunk {i}")
        print(f"\ttype: {type(chunk.content)}")
        print(f"\t{chunk}\n")
        i += 1


asyncio.run(example_streaming())

The type of the response: <class 'async_generator'>
Chunk 0
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1'}], id='2025-09-04 07:29:18.737_ad9744', created_at='2025-09-04 07:29:18.737', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=1, time=0.853181, type='chat'), metadata=None)

Chunk 1
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n'}], id='2025-09-04 07:29:18.807_2c7f43', created_at='2025-09-04 07:29:18.807', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=4, time=0.923083, type='chat'), metadata=None)

Chunk 2
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4'}], id='2025-09-04 07:29:18.877_5011a1', created_at='2025-09-04 07:29:18.878', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=7, time=0.994009, type='chat'), metadata=None)

Chunk 3
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n'}], id='2025-09-04 07:29:18.948_63144b', created_at='2025-09-04 07:29:18.948', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=10, time=1.064585, type='chat'), metadata=None)

Chunk 4
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n'}], id='2025-09-04 07:29:19.112_62850d', created_at='2025-09-04 07:29:19.112', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=16, time=1.228862, type='chat'), metadata=None)

Chunk 5
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n1'}], id='2025-09-04 07:29:19.339_6f098f', created_at='2025-09-04 07:29:19.339', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=22, time=1.455576, type='chat'), metadata=None)

Chunk 6
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n1'}], id='2025-09-04 07:29:19.476_b0fc75', created_at='2025-09-04 07:29:19.476', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=28, time=1.592406, type='chat'), metadata=None)

Chunk 7
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n1'}], id='2025-09-04 07:29:19.752_f5126b', created_at='2025-09-04 07:29:19.752', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=34, time=1.868273, type='chat'), metadata=None)

Chunk 8
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n1'}], id='2025-09-04 07:29:20.674_07b501', created_at='2025-09-04 07:29:20.674', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=40, time=2.790403, type='chat'), metadata=None)

Chunk 9
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n1'}], id='2025-09-04 07:29:20.815_95ebed', created_at='2025-09-04 07:29:20.815', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=46, time=2.931932, type='chat'), metadata=None)

Chunk 10
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20'}], id='2025-09-04 07:29:20.977_fa06a0', created_at='2025-09-04 07:29:20.977', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=50, time=3.09326, type='chat'), metadata=None)

Chunk 11
        type: <class 'list'>
        ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20'}], id='2025-09-04 07:29:20.992_ad9909', created_at='2025-09-04 07:29:20.993', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=50, time=3.109009, type='chat'), metadata=None)

Reasoning¶

AgentScope supports reasoning models by providing the ThinkingBlock.

async def example_reasoning() -> None:
    """An example of using the reasoning model."""
    model = DashScopeChatModel(
        model_name="qwen-turbo",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        enable_thinking=True,
    )

    res = await model(
        messages=[
            {"role": "user", "content": "Who am I?"},
        ],
    )

    last_chunk = None
    async for chunk in res:
        last_chunk = chunk
    print("The final response:")
    print(last_chunk)


asyncio.run(example_reasoning())

The final response:
ChatResponse(content=[{'type': 'thinking', 'thinking': 'Okay, the user asked "Who am I?" So, first, I need to figure out what they\'re really asking. They might be curious about my identity as an AI, or maybe they\'re looking for a deeper philosophical answer. Let me start by recalling my basic information. I\'m Qwen, a large-scale language model developed by Alibaba Cloud. But the user might want more than just that.\n\nHmm, maybe they\'re testing my knowledge or want to know my capabilities. I should explain my purpose, which is to assist with information and tasks. But I should also mention that I don\'t have personal experiences or consciousness. Wait, the user could be looking for a more creative or metaphorical answer. Should I include something about being a tool for learning and exploration?\n\nI need to make sure the response is clear and not too technical. Also, since the user might be interested in how I can help them, I should invite them to ask specific questions. Let me structure the answer: start with my identity, explain my role, clarify my limitations, and offer assistance. That should cover the main points without being too lengthy.\n'}, {'type': 'text', 'text': "I am Qwen, a large-scale language model developed by Alibaba Cloud. My purpose is to assist with information, tasks, and conversations. However, I don't have personal experiences, consciousness, or a sense of self. I exist to help you explore ideas, solve problems, and learn. \n\nIf you're asking this question in a philosophical or reflective context, it might be worth considering: **Who are *you*?** What defines your identity, values, or purpose? Let me know how I can help you explore this further! 😊"}], id='2025-09-04 07:29:26.449_9afd3f', created_at='2025-09-04 07:29:26.449', type='chat', usage=ChatUsage(input_tokens=12, output_tokens=342, time=5.452648, type='chat'), metadata=None)

Tools API¶

Different model providers differ in their tools APIs, e.g. the tools JSON schema, the tool call/response format. To provide a unified interface, AgentScope solves the problem by:

Providing unified tool call block ToolUseBlock and tool response block ToolResultBlock, respectively.
Providing a unified tools interface in the __call__ method of the model classes, that accepts a list of tools JSON schemas as follows:

json_schemas = [
    {
        "type": "function",
        "function": {
            "name": "google_search",
            "description": "Search for a query on Google.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "The search query.",
                    },
                },
                "required": ["query"],
            },
        },
    },
]