agentscope.rag package

Submodules

• agentscope.rag.knowledge module
  • Knowledge
    • Knowledge.__init__()
    • Knowledge.retrieve()
    • Knowledge.post_processing()
• agentscope.rag.knowledge_bank module
  • KnowledgeBank
    • KnowledgeBank.__init__()
    • KnowledgeBank.add_data_as_knowledge()
    • KnowledgeBank.get_knowledge()
    • KnowledgeBank.equip()
• agentscope.rag.llama_index_knowledge module
  • LlamaIndexKnowledge
    • LlamaIndexKnowledge.__init__()
    • LlamaIndexKnowledge.retrieve()
    • LlamaIndexKnowledge.refresh_index()

Module contents

Import all pipeline related modules in the package.

class agentscope.rag.Knowledge(knowledge_id: str, emb_model: Any | None = None, knowledge_config: dict | None = None, model: ModelWrapperBase | None = None, **kwargs: Any)

Bases: ABC

Base class for RAG, CANNOT be instantiated directly

__init__(knowledge_id: str, emb_model: Any | None = None, knowledge_config: dict | None = None, model: ModelWrapperBase | None = None, **kwargs: Any) → None

initialize the knowledge component Args: knowledge_id (str):

The id of the knowledge unit.

emb_model (ModelWrapperBase):

The embedding model used for generate embeddings

knowledge_config (dict):

The configuration to generate or load the index.

abstract retrieve(query: Any, similarity_top_k: int | None = None, to_list_strs: bool = False, **kwargs: Any) → list[Any]

retrieve list of content from database (vector stored index) to memory :param query: query for retrieval :type query: Any :param similarity_top_k: the number of most similar data returned by the

retriever.

Parameters:

to_list_strs (bool) – whether return a list of str

Returns:

return a list with retrieved documents (in strings)

post_processing(retrieved_docs: list[str], prompt: str, **kwargs: Any) → Any

A default solution for post-processing function, generates answer based on the retrieved documents. :param retrieved_docs: list of retrieved documents :type retrieved_docs: list[str] :param prompt: prompt for LLM generating answer with the retrieved documents :type prompt: str

Returns:

a synthesized answer from LLM with retrieved documents

Return type:

Any

Example

self.postprocessing_model(prompt.format(retrieved_docs))

class agentscope.rag.KnowledgeBank(configs: dict | str)

Bases: object

KnowledgeBank enables 1) provide an easy and fast way to initialize the Knowledge object; 2) make Knowledge object reusable and sharable for multiple agents.

__init__(configs: dict | str) → None

initialize the knowledge bank

add_data_as_knowledge(knowledge_id: str, emb_model_name: str, data_dirs_and_types: dict[str, list[str]] | None = None, model_name: str | None = None, knowledge_config: dict | None = None) → None

Transform data in a directory to be ready to work with RAG. :param knowledge_id: user-defined unique id for the knowledge :type knowledge_id: str :param emb_model_name: name of the embedding model :type emb_model_name: str :param model_name: name of the LLM for potential post-processing or query rewrite :type model_name: Optional[str] :param data_dirs_and_types: dictionary of data paths (keys) to the data types

(file extensions) for knowledgebase (e.g., [“.md”, “.py”, “.html”])

Parameters:

• knowledge_config (optional[dict]) – complete indexing configuration, used for more advanced applications. Users can customize - loader, - transformations, - … Examples can refer to../examples/conversation_with_RAG_agents/

• object (a simple example of importing data to Knowledge)

• '' –

  knowledge_bank.add_data_as_knowledge(

  knowledge_id=”agentscope_tutorial_rag”, emb_model_name=”qwen_emb_config”, data_dirs_and_types={

  ”../../docs/sphinx_doc/en/source/tutorial”: [“.md”],

  }, persist_dir=”./rag_storage/tutorial_assist”,

  )

• ''

get_knowledge(knowledge_id: str, duplicate: bool = False) → Knowledge

Get a Knowledge object from the knowledge bank. :param knowledge_id: unique id for the Knowledge object :type knowledge_id: str :param duplicate: whether return a copy of the Knowledge object. :type duplicate: bool

Returns:

the Knowledge object defined with Llama-index

Return type:

Knowledge

equip(agent: AgentBase, knowledge_id_list: list[str] | None = None, duplicate: bool = False) → None

Equip the agent with the knowledge by knowledge ids.

Parameters:

• agent (AgentBase) – the agent to be equipped with knowledge

• knowledge_id_list – the list of knowledge ids to be equipped with the agent

• duplicate (bool) – whether to deepcopy the knowledge object

TODO: to accommodate with distributed setting