Skip to main content

Azure Cosmos DB NoSQL 语义缓存

¥Azure Cosmos DB NoSQL Semantic Cache

Azure Cosmos DB 支持语义缓存功能，用于 NoSQL 集成，使用户能够根据用户输入与先前缓存结果之间的语义相似性检索缓存的响应。它利用了 AzureCosmosDBNoSQLVectorStore，后者存储了缓存提示的向量嵌入。这些嵌入支持基于相似度的搜索，允许系统检索相关的缓存结果。

¥The Semantic Cache feature is supported with Azure Cosmos DB for NoSQL integration, enabling users to retrieve cached responses based on semantic similarity between the user input and previously cached results. It leverages AzureCosmosDBNoSQLVectorStore, which stores vector embeddings of cached prompts. These embeddings enable similarity-based searches, allowing the system to retrieve relevant cached results.

如果你没有 Azure 账户，可以使用 创建免费账户 开始使用。

¥If you don't have an Azure account, you can create a free account to get started.

设置​

¥Setup

你首先需要安装 @langchain/azure-cosmosdb 软件包：

¥You'll first need to install the @langchain/azure-cosmosdb package:

tip

请参阅 此部分包含有关安装集成包的一般说明。

¥See this section for general instructions on installing integration packages.

npm install @langchain/azure-cosmosdb @langchain/core

你还需要运行一个 Azure Cosmos DB for NoSQL 实例。你可以按照 此指南 在 Azure 门户上免费部署一个版本。

¥You'll also need to have an Azure Cosmos DB for NoSQL instance running. You can deploy a free version on Azure Portal without any cost, following this guide.

实例运行后，请确保你拥有连接字符串。如果你使用托管身份，则需要拥有端点。你可以在 Azure 门户中实例的 "设置/密钥" 部分下找到它们。

¥Once you have your instance running, make sure you have the connection string. If you are using Managed Identity, you need to have the endpoint. You can find them in the Azure Portal, under the "Settings / Keys" section of your instance.

info

使用 Azure 托管标识和基于角色的访问控制时，必须确保已预先创建数据库和容器。RBAC 不提供创建数据库和容器的权限。你可以在 Azure Cosmos DB 文档 中获取有关权限模型的更多信息。

¥When using Azure Managed Identity and role-based access control, you must ensure that the database and container have been created beforehand. RBAC does not provide permissions to create databases and containers. You can get more information about the permission model in the Azure Cosmos DB documentation.

用法示例​

¥Usage example

import {
  AzureCosmosDBNoSQLConfig,
  AzureCosmosDBNoSQLSemanticCache,
} from "@langchain/azure-cosmosdb";
import { ChatOpenAI, OpenAIEmbeddings } from "@langchain/openai";

const embeddings = new OpenAIEmbeddings();
const config: AzureCosmosDBNoSQLConfig = {
  databaseName: "<DATABASE_NAME>",
  containerName: "<CONTAINER_NAME>",
  // use endpoint to initiate client with managed identity
  connectionString: "<CONNECTION_STRING>",
};

/**
 * Sets the threshold similarity score for returning cached results based on vector distance.
 * Cached output is returned only if the similarity score meets or exceeds this threshold;
 * otherwise, a new result is generated. Default is 0.6, adjustable via the constructor
 * to suit various distance functions and use cases.
 * (see: https://aka.ms/CosmosVectorSearch).
 */

const similarityScoreThreshold = 0.5;
const cache = new AzureCosmosDBNoSQLSemanticCache(
  embeddings,
  config,
  similarityScoreThreshold
);

const model = new ChatOpenAI({ cache });

// Invoke the model to perform an action
const response1 = await model.invoke("Do something random!");
console.log(response1);
/*
  AIMessage {
    content: "Sure! I'll generate a random number for you: 37",
    additional_kwargs: {}
  }
*/

const response2 = await model.invoke("Do something random!");
console.log(response2);
/*
  AIMessage {
    content: "Sure! I'll generate a random number for you: 37",
    additional_kwargs: {}
  }
*/

API Reference:

¥Related