Skip to main content

如何合并多个检索器的结果

¥How to combine results from multiple retrievers

Prerequisites

本指南假设你熟悉以下概念：

¥This guide assumes familiarity with the following concepts:

EnsembleRetriever 支持集成来自多个检索器的结果。它使用 BaseRetriever 对象列表进行初始化。EnsembleRetrievers 根据 倒数排序融合 算法对组成检索器的结果进行重新排序。

¥The EnsembleRetriever supports ensembling of results from multiple retrievers. It is initialized with a list of BaseRetriever objects. EnsembleRetrievers rerank the results of the constituent retrievers based on the Reciprocal Rank Fusion algorithm.

通过利用不同算法的优势，EnsembleRetriever 可以获得比任何单一算法都更好的性能。

¥By leveraging the strengths of different algorithms, the EnsembleRetriever can achieve better performance than any single algorithm.

一种有用的模式是将关键字匹配检索器与密集检索器（如嵌入相似性）结合起来，因为它们的优势是互补的。这可以被视为 "混合搜索" 的一种形式。稀疏检索器擅长根据关键词查找相关文档，而密集检索器擅长根据语义相似性查找相关文档。

¥One useful pattern is to combine a keyword matching retriever with a dense retriever (like embedding similarity), because their strengths are complementary. This can be considered a form of "hybrid search". The sparse retriever is good at finding relevant documents based on keywords, while the dense retriever is good at finding relevant documents based on semantic similarity.

下面我们演示如何将 简单的自定义检索器 与源自 演示，内存，向量存储 的检索器进行集成，简单的自定义检索器 仅返回直接包含输入查询的文档。

¥Below we demonstrate ensembling of a simple custom retriever that simply returns documents that directly contain the input query with a retriever derived from a demo, in-memory, vector store.

import { EnsembleRetriever } from "langchain/retrievers/ensemble";
import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { OpenAIEmbeddings } from "@langchain/openai";
import { BaseRetriever, BaseRetrieverInput } from "@langchain/core/retrievers";
import { Document } from "@langchain/core/documents";

class SimpleCustomRetriever extends BaseRetriever {
  lc_namespace = [];

  documents: Document[];

  constructor(fields: { documents: Document[] } & BaseRetrieverInput) {
    super(fields);
    this.documents = fields.documents;
  }

  async _getRelevantDocuments(query: string): Promise<Document[]> {
    return this.documents.filter((document) =>
      document.pageContent.includes(query)
    );
  }
}

const docs1 = [
  new Document({ pageContent: "I like apples", metadata: { source: 1 } }),
  new Document({ pageContent: "I like oranges", metadata: { source: 1 } }),
  new Document({
    pageContent: "apples and oranges are fruits",
    metadata: { source: 1 },
  }),
];

const keywordRetriever = new SimpleCustomRetriever({ documents: docs1 });

const docs2 = [
  new Document({ pageContent: "You like apples", metadata: { source: 2 } }),
  new Document({ pageContent: "You like oranges", metadata: { source: 2 } }),
];

const vectorstore = await MemoryVectorStore.fromDocuments(
  docs2,
  new OpenAIEmbeddings()
);

const vectorstoreRetriever = vectorstore.asRetriever();

const retriever = new EnsembleRetriever({
  retrievers: [vectorstoreRetriever, keywordRetriever],
  weights: [0.5, 0.5],
});

const query = "apples";
const retrievedDocs = await retriever.invoke(query);

console.log(retrievedDocs);

/*
  [
    Document { pageContent: 'You like apples', metadata: { source: 2 } },
    Document { pageContent: 'I like apples', metadata: { source: 1 } },
    Document { pageContent: 'You like oranges', metadata: { source: 2 } },
    Document {
      pageContent: 'apples and oranges are fruits',
      metadata: { source: 1 }
    }
  ]
*/

API Reference:

后续步骤​

¥Next steps

现在你已经学习了如何合并来自多个检索器的结果。接下来，查看其他检索操作指南，例如如何执行 使用每个文档的多个嵌入来改进结果 或如何执行 创建你自己的自定义检索器。

¥You've now learned how to combine results from multiple retrievers. Next, check out some other retrieval how-to guides, such as how to improve results using multiple embeddings per document or how to create your own custom retriever.