.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "build_tutorial/streaming.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_build_tutorial_streaming.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_build_tutorial_streaming.py:


.. _streaming:

Streaming Mode
=========================

AgentScope supports streaming output for the following APIs in both terminal
and AgentScope Studio.

.. list-table::
    :header-rows: 1

    * - API
      - Class
      - Streaming
    * - OpenAI Chat API
      - `OpenAIChatWrapper`
      - ✓
    * - DashScope Chat API
      - `DashScopeChatWrapper`
      - ✓
    * - Gemini Chat API
      - `GeminiChatWrapper`
      - ✓
    * - ZhipuAI Chat API
      - `ZhipuAIChatWrapper`
      - ✓
    * - Ollama Chat API
      - `OllamaChatWrapper`
      - ✓
    * - LiteLLM Chat API
      - `LiteLLMChatWrapper`
      - ✓
    * - Anthropic Chat API
      - `AnthropicChatWrapper`
      - ✓

This section will show how to enable streaming mode in AgentScope and handle
the streaming response within an agent.

.. GENERATED FROM PYTHON SOURCE LINES 44-53

Enabling Streaming Output
----------------------------

AgentScope supports streaming output by providing a `stream` parameter
in model wrapper class.
You can directly specify the `stream` parameter in initialization or
configuration.

- Specifying in Initialization

.. GENERATED FROM PYTHON SOURCE LINES 53-64

.. code-block:: Python


    from agentscope.models import DashScopeChatWrapper
    import os

    model = DashScopeChatWrapper(
        config_name="_",
        model_name="qwen-max",
        api_key=os.environ["DASHSCOPE_API_KEY"],
        stream=True,  # Enabling the streaming output
    )








.. GENERATED FROM PYTHON SOURCE LINES 65-66

- Specifying in Configuration

.. GENERATED FROM PYTHON SOURCE LINES 66-74

.. code-block:: Python


    model_config = {
        "model_type": "dashscope_chat",
        "config_name": "qwen_config",
        "model_name": "qwen-max",
        "stream": True,
    }








.. GENERATED FROM PYTHON SOURCE LINES 75-79

With the above configuration, we can obtain streaming output with built-in
agents in AgentScope.

Next, we show how to handle the streaming output within an agent.

.. GENERATED FROM PYTHON SOURCE LINES 81-86

Handling Streaming Response
-------------------------------------------

Once we enable the streaming output, the returned model response will
contain a generator in its `stream` field.

.. GENERATED FROM PYTHON SOURCE LINES 86-92

.. code-block:: Python


    prompt = [{"role": "user", "content": "Hi!"}]

    response = model(prompt)
    print("The type of response.stream:", type(response.stream))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    The type of response.stream: <class 'generator'>




.. GENERATED FROM PYTHON SOURCE LINES 93-96

We can iterate over the generator to get the streaming text.
A boolean value will also be yielded to indicate whether the current
chunk is the last one.

.. GENERATED FROM PYTHON SOURCE LINES 96-101

.. code-block:: Python


    for index, chunk in enumerate(response.stream):
        print(f"{index}.", chunk)
        print(f"Current text field:", response.text)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    0. (False, 'Hello')
    Current text field: Hello
    1. (False, 'Hello!')
    Current text field: Hello!
    2. (False, 'Hello! How')
    Current text field: Hello! How
    3. (False, 'Hello! How can I assist you')
    Current text field: Hello! How can I assist you
    4. (False, 'Hello! How can I assist you today?')
    Current text field: Hello! How can I assist you today?
    5. (True, 'Hello! How can I assist you today?')
    Current text field: Hello! How can I assist you today?




.. GENERATED FROM PYTHON SOURCE LINES 102-109

.. note:: Note the generator is incremental and one-time.

During the iterating, the `text` field in the response will concatenate
sub strings automatically.

To be compatible with non-streaming mode, you can also directly use
`response.text` to obtain all text at once.

.. GENERATED FROM PYTHON SOURCE LINES 109-114

.. code-block:: Python


    prompt = [{"role": "user", "content": "Hi!"}]
    response = model(prompt)
    print(response.text)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Hello! How can I assist you today?




.. GENERATED FROM PYTHON SOURCE LINES 115-140

Displaying Like Typewriter
-------------------------------------------
To display the streaming text like a typewriter, AgentScope provides a
`speak` function within the `AgentBase` class.
If a generator is given, the `speak` function will iterate over the
generator and print the text like a typewriter in terminal or AgentScope
Studio.

.. code-block:: python

  def reply(*args, **kwargs):
      # ...
      self.speak(response.stream)
      # ...

To be compatible with both streaming and non-streaming mode, we use the
following code snippet for all built-in agents in AgentScope.

.. code-block:: python

  def reply(*args, **kwargs):
      # ...
      self.speak(response.stream or response.text)
      # ...



.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 3.611 seconds)


.. _sphx_glr_download_build_tutorial_streaming.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: streaming.ipynb <streaming.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: streaming.py <streaming.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: streaming.zip <streaming.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_