agentscope.parsers package

Submodules

Module contents

Model response parser module.

class agentscope.parsers.ParserBase[source]

Bases: ABC

The base class for model response parser.

abstract parse(response: ModelResponse) → ModelResponse[source]

Parse the response text to a specific object, and stored in the parsed field of the response object.

class agentscope.parsers.MarkdownJsonObjectParser(content_hint: Any | None = None)[source]

Bases: ParserBase

A parser to parse the response text to a json object.

name: str = 'json block'

The name of the parser.

tag_begin: str = '```json'

Opening tag for a code block.

tag_end: str = '```'

Closing end for a code block.

__init__(content_hint: Any | None = None) → None[source]

Initialize the parser with the content hint.

Parameters:

content_hint (Optional[Any], defaults to None) – The hint used to remind LLM what should be fill between the tags. If it is a string, it will be used as the content hint directly. If it is a dict, it will be converted to a json string and used as the content hint.

content_hint: str = '{your_json_object}'

The hint of the content.

parse(response: ModelResponse) → ModelResponse[source]

Parse the response text to a json object, and fill it in the parsed field in the response object.

property format_instruction: str

Get the format instruction for the json object, if the format_example is provided, it will be used as the example.

class agentscope.parsers.MarkdownJsonDictParser(content_hint: Any | None = None, required_keys: List[str] | None = None, keys_to_memory: str | bool | Sequence[str] = True, keys_to_content: str | bool | Sequence[str] = True, keys_to_metadata: str | bool | Sequence[str] = False)[source]

Bases: MarkdownJsonObjectParser, DictFilterMixin

A class used to parse a JSON dictionary object in a markdown fenced code

name: str = 'json block'

The name of the parser.

tag_begin: str = '```json'

Opening tag for a code block.

tag_end: str = '```'

Closing end for a code block.

__init__(content_hint: Any | None = None, required_keys: List[str] | None = None, keys_to_memory: str | bool | Sequence[str] = True, keys_to_content: str | bool | Sequence[str] = True, keys_to_metadata: str | bool | Sequence[str] = False) → None[source]

Initialize the parser with the content hint.

Parameters:

  • content_hint (Optional[Any], defaults to None) – The hint used to remind LLM what should be fill between the tags. If it is a string, it will be used as the content hint directly. If it is a dict, it will be converted to a json string and used as the content hint. If it’s a Pydantic model, the schema will be displayed in the instruction.

  • required_keys (List[str], defaults to []) – A list of required keys in the JSON dictionary object. If the response misses any of the required keys, it will raise a RequiredFieldNotFoundError.

  • (`Optional[Union[str (keys_to_memory) –

  • bool

  • Sequence[str]]]`

:param : :param defaults to True): The key or keys to be filtered in to_memory method. If

it’s - False, None will be returned in the to_memory method - str, the corresponding value will be returned - List[str], a filtered dictionary will be returned - True, the whole dictionary will be returned

Parameters:

  • (`Optional[Union[str (keys_to_content) –

  • bool

  • Sequence[str]]]`

:param : :param defaults to True): The key or keys to be filtered in to_content method. If

it’s - False, None will be returned in the to_content method - str, the corresponding value will be returned - List[str], a filtered dictionary will be returned - True, the whole dictionary will be returned

Parameters:

  • (`Optional[Union[str (keys_to_metadata) –

  • bool

  • Sequence[str]]`

:param : :param defaults to False): The key or keys to be filtered in to_metadata method. If

it’s - False, None will be returned in the to_metadata method - str, the corresponding value will be returned - List[str], a filtered dictionary will be returned - True, the whole dictionary will be returned

content_hint: str = '{your_json_dictionary}'

The hint of the content.

required_keys: List[str]

A list of required keys in the JSON dictionary object. If the response misses any of the required keys, it will raise a RequiredFieldNotFoundError.

property format_instruction: str

Get the format instruction for the json object, if the format_example is provided, it will be used as the example.

parse(response: ModelResponse) → ModelResponse[source]

Parse the text field of the response to a JSON dictionary object, store it in the parsed field of the response object, and check if the required keys exists.

class agentscope.parsers.MarkdownCodeBlockParser(language_name: str, content_hint: str | None = None)[source]

Bases: ParserBase

The base class for parsing the response text by fenced block.

tag_end: str = '```'

The ending tag.

__init__(language_name: str, content_hint: str | None = None) → None[source]

Initialize the parser with the language name and the optional content hint.

Parameters:

language_name – The name of the language, which will be used in ```{language_name}

name: str = '{language_name} block'

The name of the parser.

tag_begin: str = '```{language_name}'

The beginning tag.

content_hint: str = '${{your_{language_name}_code}}'

The hint of the content.

format_instruction: str = 'You should generate {language_name} code in a {language_name} fenced code block as follows: \n```{language_name}\n{content_hint}\n```'

The instruction for the format of the code block.

parse(response: ModelResponse) → ModelResponse[source]

Extract the content between the tag_begin and tag_end in the response and store it in the parsed field of the response object.

class agentscope.parsers.TaggedContent(name: str, tag_begin: str, content_hint: str, tag_end: str, parse_json: bool = False)[source]

Bases: object

A tagged content object to store the tag name, tag begin, content hint and tag end.

__init__(name: str, tag_begin: str, content_hint: str, tag_end: str, parse_json: bool = False) → None[source]

Initialize the tagged content object.

Parameters:

  • name (str) – The name of the tagged content.

  • tag_begin (str) – The beginning tag.

  • content_hint (str) – The hint of the content.

  • tag_end (str) – The ending tag.

  • parse_json (bool, defaults to False) – Whether to parse the content as a json object.

name: str

The name of the tagged content, which will be used as the key in extracted dictionary.

tag_begin: str

The beginning tag.

content_hint: str

The hint of the content.

tag_end: str

The ending tag.

parse_json: bool

Whether to parse the content as a json object.

class agentscope.parsers.MultiTaggedContentParser(*tagged_contents: TaggedContent, keys_to_memory: str | bool | Sequence[str] | None = True, keys_to_content: str | bool | Sequence[str] | None = True, keys_to_metadata: str | bool | Sequence[str] | None = False, keys_allow_missing: List[str] | None = None)[source]

Bases: ParserBase, DictFilterMixin

Parse response text by multiple tags, and return a dict of their content. Asking llm to generate JSON dictionary object directly maybe not a good idea due to involving escape characters and other issues. So we can ask llm to generate text with tags, and then parse the text to get the final JSON dictionary object.

json_required_hint = ', and the content between {} MUST be a JSON object:'

If a tagged content is required to be a JSON object by parse_json equals to True, this instruction will be used to remind the model to generate JSON object.

__init__(*tagged_contents: TaggedContent, keys_to_memory: str | bool | Sequence[str] | None = True, keys_to_content: str | bool | Sequence[str] | None = True, keys_to_metadata: str | bool | Sequence[str] | None = False, keys_allow_missing: List[str] | None = None) → None[source]

Initialize the parser with tags.

Parameters:

  • *tagged_contents (dict[str, Tuple[str, str]]) – Multiple TaggedContent objects, each object contains the tag name, tag begin, content hint and tag end. The name will be used as the key in the extracted dictionary.

  • required_keys (Optional[List[str]], defaults to None) – A list of required

  • (`Optional[Union[str (keys_to_memory) –

  • bool

  • Sequence[str]]]`

:param : :param defaults to True): The key or keys to be filtered in to_memory method. If

it’s - False, None will be returned in the to_memory method - str, the corresponding value will be returned - List[str], a filtered dictionary will be returned - True, the whole dictionary will be returned

Parameters:

  • (`Optional[Union[str (keys_to_content) –

  • bool

  • Sequence[str]]`

:param : :param defaults to True): The key or keys to be filtered in to_content method. If

it’s - False, None will be returned in the to_content method - str, the corresponding value will be returned - List[str], a filtered dictionary will be returned - True, the whole dictionary will be returned

Parameters:

  • (`Optional[Union[str (keys_to_metadata) –

  • bool

  • Sequence[str]]]`

:param : :param defaults to False): The key or keys to be filtered in to_metadata method. If

it’s - False, None will be returned in the to_metadata method - str, the corresponding value will be returned - List[str], a filtered dictionary will be returned - True, the whole dictionary will be returned

Parameters:

keys_allow_missing (Optional[List[str]], defaults to None) – A list of keys that are allowed to be missing in the response.

format_instruction = 'Respond with specific tags as outlined below{json_required_hint}\n{tag_lines_format}'

The instruction for the format of the tagged content.

parse(response: ModelResponse) → ModelResponse[source]

Parse the response text by tags, and return a dict of their content in the parsed field of the model response object. If the tagged content requires to parse as a JSON object by parse_json equals to True, it will be parsed as a JSON object by json.loads.

class agentscope.parsers.RegexTaggedContentParser(tagged_content_pattern: str = '<(?P<name>[^>]+)>(?P<content>.*?)</\\1?>', format_instruction: str | None = None, try_parse_json: bool = True, required_keys: List[str] | None = None, keys_to_memory: str | bool | Sequence[str] = True, keys_to_content: str | bool | Sequence[str] = True, keys_to_metadata: str | bool | Sequence[str] = False)[source]

Bases: ParserBase, DictFilterMixin

A regex tagged content parser, which extracts tagged content according to the provided regex pattern. Different from other parsers, this parser allows to extract multiple tagged content without knowing the keys in advance. The parsed result will be a dictionary within the parsed field of the model response.

Compared with other parsers, this parser is more flexible and can be used in dynamic scenarios where - the keys are not known in advance - the number of the tagged content is not fixed

Note: Without knowing the keys in advance, it’s hard to prepare a format instruction template for different scenarios. Therefore, we ask the user to provide the format instruction in the constructor. Of course, the user can construct and manage the prompt by themselves optionally.

Example

By default, the parser use a regex pattern to extract tagged content with the following format: ` <{name1}>{content1}</{name1}> <{name2}>{content2}</{name2}> ` The parser will extract the content as the following dictionary: ``` {

“name1”: content1, “name2”: content2,

}

__init__(tagged_content_pattern: str = '<(?P<name>[^>]+)>(?P<content>.*?)</\\1?>', format_instruction: str | None = None, try_parse_json: bool = True, required_keys: List[str] | None = None, keys_to_memory: str | bool | Sequence[str] = True, keys_to_content: str | bool | Sequence[str] = True, keys_to_metadata: str | bool | Sequence[str] = False) → None[source]

Initialize the regex tagged content parser.

Parameters:

  • (Optional[str] (tagged_content_pattern)

  • to (defaults)

  • `"< –

    The regex pattern to extract tagged content. The pattern should contain two named groups: name and content. The name group is used as the key of the tagged content, and the content group is used as the value.

  • format_instruction (Optional[str], defaults to None) – The instruction for the format of the tagged content, which will be attached to the end of the prompt messages to remind the LLM to follow the format.

  • try_parse_json (bool, defaults to True) – Whether to try to parse the tagged content as JSON. Note the parsing function won’t raise exceptions.

  • required_keys (Optional[List[str]], defaults to None) – The keys that are required in the tagged content.

  • (`Union[str (keys_to_memory) –

  • bool

  • Sequence[str]]`

:param : :param defaults to True): The keys to save to memory. :param keys_to_content (Union[str: :param bool: :param Sequence[str]]: :param : :param defaults to True): The keys to save to content. :param keys_to_metadata (Union[str: :param bool: :param Sequence[str]]: :param : :param defaults to False): The key or keys to be filtered in to_metadata method. If

it’s - False, None will be returned in the to_metadata method - str, the corresponding value will be returned - List[str], a filtered dictionary will be returned - True, the whole dictionary will be returned

property format_instruction: str

The format instruction for the tagged content.

parse(response: ModelResponse) → ModelResponse[source]

Parse the response text by the regex pattern, and return a dict of the content in the parsed field of the response.

Parameters:

response (ModelResponse) – The response to be parsed.

Returns:

The response with the parsed field as the parsed result.

Return type:

ModelResponse