.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "build_tutorial/streaming.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_build_tutorial_streaming.py: .. _streaming: Streaming Mode ========================= AgentScope supports streaming output for the following APIs in both terminal and AgentScope Studio. .. list-table:: :header-rows: 1 * - API - Class - Streaming * - OpenAI Chat API - `OpenAIChatWrapper` - ✓ * - DashScope Chat API - `DashScopeChatWrapper` - ✓ * - Gemini Chat API - `GeminiChatWrapper` - ✓ * - ZhipuAI Chat API - `ZhipuAIChatWrapper` - ✓ * - Ollama Chat API - `OllamaChatWrapper` - ✓ * - LiteLLM Chat API - `LiteLLMChatWrapper` - ✓ * - Anthropic Chat API - `AnthropicChatWrapper` - ✓ This section will show how to enable streaming mode in AgentScope and handle the streaming response within an agent. .. GENERATED FROM PYTHON SOURCE LINES 44-53 Enabling Streaming Output ---------------------------- AgentScope supports streaming output by providing a `stream` parameter in model wrapper class. You can directly specify the `stream` parameter in initialization or configuration. - Specifying in Initialization .. GENERATED FROM PYTHON SOURCE LINES 53-64 .. code-block:: Python from agentscope.models import DashScopeChatWrapper import os model = DashScopeChatWrapper( config_name="_", model_name="qwen-max", api_key=os.environ["DASHSCOPE_API_KEY"], stream=True, # Enabling the streaming output ) .. GENERATED FROM PYTHON SOURCE LINES 65-66 - Specifying in Configuration .. GENERATED FROM PYTHON SOURCE LINES 66-74 .. code-block:: Python model_config = { "model_type": "dashscope_chat", "config_name": "qwen_config", "model_name": "qwen-max", "stream": True, } .. GENERATED FROM PYTHON SOURCE LINES 75-79 With the above configuration, we can obtain streaming output with built-in agents in AgentScope. Next, we show how to handle the streaming output within an agent. .. GENERATED FROM PYTHON SOURCE LINES 81-86 Handling Streaming Response ------------------------------------------- Once we enable the streaming output, the returned model response will contain a generator in its `stream` field. .. GENERATED FROM PYTHON SOURCE LINES 86-92 .. code-block:: Python prompt = [{"role": "user", "content": "Hi!"}] response = model(prompt) print("The type of response.stream:", type(response.stream)) .. rst-class:: sphx-glr-script-out .. code-block:: none The type of response.stream: .. GENERATED FROM PYTHON SOURCE LINES 93-96 We can iterate over the generator to get the streaming text. A boolean value will also be yielded to indicate whether the current chunk is the last one. .. GENERATED FROM PYTHON SOURCE LINES 96-101 .. code-block:: Python for index, chunk in enumerate(response.stream): print(f"{index}.", chunk) print(f"Current text field:", response.text) .. rst-class:: sphx-glr-script-out .. code-block:: none 0. (False, 'Hello') Current text field: Hello 1. (False, 'Hello!') Current text field: Hello! 2. (False, 'Hello! How') Current text field: Hello! How 3. (False, 'Hello! How can I assist you') Current text field: Hello! How can I assist you 4. (False, 'Hello! How can I assist you today?') Current text field: Hello! How can I assist you today? 5. (True, 'Hello! How can I assist you today?') Current text field: Hello! How can I assist you today? .. GENERATED FROM PYTHON SOURCE LINES 102-109 .. note:: Note the generator is incremental and one-time. During the iterating, the `text` field in the response will concatenate sub strings automatically. To be compatible with non-streaming mode, you can also directly use `response.text` to obtain all text at once. .. GENERATED FROM PYTHON SOURCE LINES 109-114 .. code-block:: Python prompt = [{"role": "user", "content": "Hi!"}] response = model(prompt) print(response.text) .. rst-class:: sphx-glr-script-out .. code-block:: none Hello! How can I assist you today? .. GENERATED FROM PYTHON SOURCE LINES 115-140 Displaying Like Typewriter ------------------------------------------- To display the streaming text like a typewriter, AgentScope provides a `speak` function within the `AgentBase` class. If a generator is given, the `speak` function will iterate over the generator and print the text like a typewriter in terminal or AgentScope Studio. .. code-block:: python def reply(*args, **kwargs): # ... self.speak(response.stream) # ... To be compatible with both streaming and non-streaming mode, we use the following code snippet for all built-in agents in AgentScope. .. code-block:: python def reply(*args, **kwargs): # ... self.speak(response.stream or response.text) # ... .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 3.611 seconds) .. _sphx_glr_download_build_tutorial_streaming.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: streaming.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: streaming.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: streaming.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_