.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "tutorial/task_model.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_tutorial_task_model.py: .. _model: Model ==================== In this tutorial, we introduce the model APIs integrated in AgentScope, how to use them and how to integrate new model APIs. The supported model APIs and providers include: .. list-table:: :header-rows: 1 * - API - Class - Compatible - Streaming - Tools - Vision - Reasoning * - OpenAI - ``OpenAIChatModel`` - vLLM, DeepSeek - ✅ - ✅ - ✅ - ✅ * - DashScope - ``DashScopeChatModel`` - - ✅ - ✅ - ✅ - ✅ * - Anthropic - ``AnthropicChatModel`` - - ✅ - ✅ - ✅ - ✅ * - Gemini - ``GeminiChatModel`` - - ✅ - ✅ - ✅ - ✅ * - Ollama - ``OllamaChatModel`` - - ✅ - ✅ - ✅ - ✅ To provide unified model interfaces, the above model classes has the following common methods: - The first three arguments of the ``__call__`` method are ``messages`` , ``tools`` and ``tool_choice``, representing the input messages, JSON schema of tool functions, and tool selection mode, respectively. - The return type are either a ``ChatResponse`` instance or an async generator of ``ChatResponse`` in streaming mode. .. note:: Different model APIs differ in the input message format, refer to :ref:`prompt` for more details. The ``ChatResponse`` instance contains the generated thinking/text/tool use content, identity, created time and usage information. .. GENERATED FROM PYTHON SOURCE LINES 66-91 .. code-block:: Python import asyncio import json import os from agentscope.message import TextBlock, ToolUseBlock, ThinkingBlock, Msg from agentscope.model import ChatResponse, DashScopeChatModel response = ChatResponse( content=[ ThinkingBlock( type="thinking", thinking="I should search for AgentScope on Google.", ), TextBlock(type="text", text="I'll search for AgentScope on Google."), ToolUseBlock( type="tool_use", id="642n298gjna", name="google_search", input={"query": "AgentScope?"}, ), ], ) print(response) .. rst-class:: sphx-glr-script-out .. code-block:: none ChatResponse(content=[{'type': 'thinking', 'thinking': 'I should search for AgentScope on Google.'}, {'type': 'text', 'text': "I'll search for AgentScope on Google."}, {'type': 'tool_use', 'id': '642n298gjna', 'name': 'google_search', 'input': {'query': 'AgentScope?'}}], id='2025-08-15 10:12:50.374_4a9970', created_at='2025-08-15 10:12:50.374', type='chat', usage=None) .. GENERATED FROM PYTHON SOURCE LINES 92-93 Taking ``DashScopeChatModel`` as an example, we can use it to create a chat model instance and call it with messages and tools: .. GENERATED FROM PYTHON SOURCE LINES 93-118 .. code-block:: Python async def example_model_call() -> None: """An example of using the DashScopeChatModel.""" model = DashScopeChatModel( model_name="qwen-max", api_key=os.environ["DASHSCOPE_API_KEY"], stream=False, ) res = await model( messages=[ {"role": "user", "content": "Hi!"}, ], ) # You can directly create a ``Msg`` object with the response content msg_res = Msg("Friday", res.content, "assistant") print("The response:", res) print("The response as Msg:", msg_res) asyncio.run(example_model_call()) .. rst-class:: sphx-glr-script-out .. code-block:: none The response: ChatResponse(content=[{'type': 'text', 'text': 'Hello! How can I assist you today?'}], id='2025-08-15 10:12:51.740_b8ed6e', created_at='2025-08-15 10:12:51.740', type='chat', usage=ChatUsage(input_tokens=10, output_tokens=9, time=1.365658, type='chat')) The response as Msg: Msg(id='Hw6487L3e3opbV77hrp8mR', name='Friday', content=[{'type': 'text', 'text': 'Hello! How can I assist you today?'}], role='assistant', metadata=None, timestamp='2025-08-15 10:12:51.740', invocation_id='None') .. GENERATED FROM PYTHON SOURCE LINES 119-126 Streaming ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To enable streaming model, set the ``stream`` parameter in the model constructor to ``True``. When streaming is enabled, the ``__call__`` method will return an **async generator** that yields ``ChatResponse`` instances as they are generated by the model. .. note:: The streaming mode in AgentScope is designed to be **cumulative**, meaning the content in each chunk contains all the previous content plus the newly generated content. .. GENERATED FROM PYTHON SOURCE LINES 126-156 .. code-block:: Python async def example_streaming() -> None: """An example of using the streaming model.""" model = DashScopeChatModel( model_name="qwen-max", api_key=os.environ["DASHSCOPE_API_KEY"], stream=True, ) generator = await model( messages=[ { "role": "user", "content": "Count from 1 to 20, and just report the number without any other information.", }, ], ) print("The type of the response:", type(generator)) i = 0 async for chunk in generator: print(f"Chunk {i}") print(f"\ttype: {type(chunk.content)}") print(f"\t{chunk}\n") i += 1 asyncio.run(example_streaming()) .. rst-class:: sphx-glr-script-out .. code-block:: none The type of the response: Chunk 0 type: ChatResponse(content=[{'type': 'text', 'text': '1'}], id='2025-08-15 10:12:52.669_7bef12', created_at='2025-08-15 10:12:52.669', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=1, time=0.9272, type='chat')) Chunk 1 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n'}], id='2025-08-15 10:12:52.741_dcd7f1', created_at='2025-08-15 10:12:52.741', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=4, time=0.99952, type='chat')) Chunk 2 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4'}], id='2025-08-15 10:12:52.813_4f535c', created_at='2025-08-15 10:12:52.813', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=7, time=1.071609, type='chat')) Chunk 3 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n'}], id='2025-08-15 10:12:52.991_256748', created_at='2025-08-15 10:12:52.991', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=10, time=1.24962, type='chat')) Chunk 4 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n'}], id='2025-08-15 10:12:53.412_263eca', created_at='2025-08-15 10:12:53.412', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=16, time=1.669922, type='chat')) Chunk 5 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n1'}], id='2025-08-15 10:12:53.789_ca667a', created_at='2025-08-15 10:12:53.789', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=22, time=2.047494, type='chat')) Chunk 6 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n1'}], id='2025-08-15 10:12:53.934_f75dd1', created_at='2025-08-15 10:12:53.934', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=28, time=2.192057, type='chat')) Chunk 7 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n1'}], id='2025-08-15 10:12:54.196_6cd407', created_at='2025-08-15 10:12:54.196', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=34, time=2.454762, type='chat')) Chunk 8 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n1'}], id='2025-08-15 10:12:54.325_9f727d', created_at='2025-08-15 10:12:54.326', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=40, time=2.583817, type='chat')) Chunk 9 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n1'}], id='2025-08-15 10:12:54.481_cc7a51', created_at='2025-08-15 10:12:54.481', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=46, time=2.738962, type='chat')) Chunk 10 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20'}], id='2025-08-15 10:12:55.547_8aa0f2', created_at='2025-08-15 10:12:55.547', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=50, time=3.804945, type='chat')) Chunk 11 type: ChatResponse(content=[{'type': 'text', 'text': '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20'}], id='2025-08-15 10:12:55.563_0a6389', created_at='2025-08-15 10:12:55.563', type='chat', usage=ChatUsage(input_tokens=27, output_tokens=50, time=3.821629, type='chat')) .. GENERATED FROM PYTHON SOURCE LINES 157-161 Reasoning ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ AgentScope supports reasoning models by providing the ``ThinkingBlock``. .. GENERATED FROM PYTHON SOURCE LINES 161-186 .. code-block:: Python async def example_reasoning() -> None: """An example of using the reasoning model.""" model = DashScopeChatModel( model_name="qwen-turbo", api_key=os.environ["DASHSCOPE_API_KEY"], enable_thinking=True, ) res = await model( messages=[ {"role": "user", "content": "Who am I?"}, ], ) last_chunk = None async for chunk in res: last_chunk = chunk print("The final response:") print(last_chunk) asyncio.run(example_reasoning()) .. rst-class:: sphx-glr-script-out .. code-block:: none The final response: ChatResponse(content=[{'type': 'thinking', 'thinking': 'Okay, the user asked "Who am I?" and I need to respond appropriately. First, I should clarify that I can\'t know the user\'s personal identity. But maybe they\'re asking about my role or purpose. I should explain that I\'m an AI assistant designed to help with information and tasks. Also, I should ask if there\'s something specific they need help with. Keep the tone friendly and helpful. Make sure not to make assumptions about their identity. Maybe they\'re looking for a deeper philosophical answer, but I should stick to the facts. Alright, let\'s put that together in a clear response.\n'}, {'type': 'text', 'text': 'I don\'t have access to your personal identity or private information. However, I can tell you that I am **Qwen**, a large-scale language model developed by Alibaba Group. My purpose is to assist with information, answer questions, and help with various tasks. \n\nIf you\'re asking "who am I" in a more philosophical or personal sense, that\'s a deep question that depends on your own experiences, values, and reflections. Let me know how I can assist you! 😊'}], id='2025-08-15 10:12:59.158_10c1b2', created_at='2025-08-15 10:12:59.158', type='chat', usage=ChatUsage(input_tokens=12, output_tokens=226, time=3.591076, type='chat')) .. GENERATED FROM PYTHON SOURCE LINES 187-195 Tools API ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Different model providers differ in their tools APIs, e.g. the tools JSON schema, the tool call/response format. To provide a unified interface, AgentScope solves the problem by: - Providing unified tool call block :ref:`ToolUseBlock ` and tool response block :ref:`ToolResultBlock `, respectively. - Providing a unified tools interface in the ``__call__`` method of the model classes, that accepts a list of tools JSON schemas as follows: .. GENERATED FROM PYTHON SOURCE LINES 195-216 .. code-block:: Python json_schemas = [ { "type": "function", "function": { "name": "google_search", "description": "Search for a query on Google.", "parameters": { "type": "object", "properties": { "query": { "type": "string", "description": "The search query.", }, }, "required": ["query"], }, }, }, ] .. GENERATED FROM PYTHON SOURCE LINES 217-222 Further Reading ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - :ref:`message` - :ref:`prompt` .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 8.789 seconds) .. _sphx_glr_download_tutorial_task_model.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: task_model.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: task_model.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: task_model.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_