agentscope.models.post_model module

Model wrapper for post-based inference apis.

class agentscope.models.post_model.PostAPIModelWrapperBase(config_name: str, api_url: str, headers: dict | None = None, max_length: int = 2048, timeout: int = 30, json_args: dict | None = None, post_args: dict | None = None, max_retries: int = 3, messages_key: str = 'messages', retry_interval: int = 1, **kwargs: Any)[source]

Bases: ModelWrapperBase, ABC

The base model wrapper for the model deployed on the POST API.

model_type: str = 'post_api': The type of the model wrapper, which is to identify the model wrapper class in model configuration.

__init__(config_name: str, api_url: str, headers: dict | None = None, max_length: int = 2048, timeout: int = 30, json_args: dict | None = None, post_args: dict | None = None, max_retries: int = 3, messages_key: str = 'messages', retry_interval: int = 1, **kwargs: Any) → None[source]

Initialize the model wrapper.

Parameters:

config_name (str) – The id of the model.
api_url (str) – The url of the post request api.
headers (dict, defaults to None) – The headers of the api. Defaults to None.
max_length (int, defaults to 2048) – The maximum length of the model.
timeout (int, defaults to 30) – The timeout of the api. Defaults to 30.
json_args (dict, defaults to None) – The json arguments of the api. Defaults to None.
post_args (dict, defaults to None) – The post arguments of the api. Defaults to None.
max_retries (int, defaults to 3) – The maximum number of retries when the parse_func raise an exception.
messages_key (str, defaults to inputs) – The key of the input messages in the json argument.
retry_interval (int, defaults to 1) – The interval between retries when a request fails.

Note

When an object of PostApiModelWrapper is called, the arguments will of post requests will be used as follows:

request.post(
    url=api_url,
    headers=headers,
    json={
        messages_key: messages,
        **json_args
    },
    **post_args
)

config_name: str: The name of the model configuration.

model_name: str: The name of the model, which is used in model api calling.

class agentscope.models.post_model.PostAPIChatWrapper(config_name: str, api_url: str, headers: dict | None = None, max_length: int = 2048, timeout: int = 30, json_args: dict | None = None, post_args: dict | None = None, max_retries: int = 3, messages_key: str = 'messages', retry_interval: int = 1, **kwargs: Any)[source]

Bases: PostAPIModelWrapperBase

A post api model wrapper compatible with openai chat, e.g., vLLM, FastChat.

model_type: str = 'post_api_chat': The type of the model wrapper, which is to identify the model wrapper class in model configuration.

format(*args: Msg | Sequence[Msg]) → List[dict][source]

Format the input messages into a list of dict according to the model name. For example, if the model name is prefixed with “gpt-”, the input messages will be formatted for OpenAI models.

Parameters:: args (Union[Msg, Sequence[Msg]]) – The input arguments to be formatted, where each argument should be a Msg object, or a list of Msg objects. In distribution, placeholder is also allowed.
Returns:: The formatted messages.
Return type:: Union[List[dict]]

config_name: str: The name of the model configuration.

model_name: str: The name of the model, which is used in model api calling.

class agentscope.models.post_model.PostAPIDALLEWrapper(config_name: str, api_url: str, headers: dict | None = None, max_length: int = 2048, timeout: int = 30, json_args: dict | None = None, post_args: dict | None = None, max_retries: int = 3, messages_key: str = 'messages', retry_interval: int = 1, **kwargs: Any)[source]

Bases: PostAPIModelWrapperBase

A post api model wrapper compatible with openai dall_e

model_type: str = 'post_api_dall_e': The type of the model wrapper, which is to identify the model wrapper class in model configuration.

deprecated_model_type: str = 'post_api_dalle'

format(*args: Msg | Sequence[Msg]) → List[dict] | str[source]: Format the input messages into the format that the model API required.

class agentscope.models.post_model.PostAPIEmbeddingWrapper(config_name: str, api_url: str, headers: dict | None = None, max_length: int = 2048, timeout: int = 30, json_args: dict | None = None, post_args: dict | None = None, max_retries: int = 3, messages_key: str = 'messages', retry_interval: int = 1, **kwargs: Any)[source]

Bases: PostAPIModelWrapperBase

A post api model wrapper for embedding model

model_type: str = 'post_api_embedding': The type of the model wrapper, which is to identify the model wrapper class in model configuration.

format(*args: Msg | Sequence[Msg]) → List[dict] | str[source]: Format the input messages into the format that the model API required.