Class CacheBackedEmbeddings

Interface for caching results from embedding models.

The interface allows works with any store that implements the abstract store interface accepting keys of type str and values of list of floats.

If need be, the interface can be extended to accept other implementations of the value serializer and deserializer, as well as the key encoder.

Example

const underlyingEmbeddings = new OpenAIEmbeddings();

const cacheBackedEmbeddings = CacheBackedEmbeddings.fromBytesStore(
  underlyingEmbeddings,
  new ConvexKVStore({ ctx }),
  {
    namespace: underlyingEmbeddings.modelName,
  },
);

const loader = new TextLoader("./state_of_the_union.txt");
const rawDocuments = await loader.load();
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 0,
});
const documents = await splitter.splitDocuments(rawDocuments);

let time = Date.now();
const vectorstore = await ConvexVectorStore.fromDocuments(
  documents,
  cacheBackedEmbeddings,
  { ctx },
);
console.log(`Initial creation time: ${Date.now() - time}ms`);

time = Date.now();
const vectorstore2 = await ConvexVectorStore.fromDocuments(
  documents,
  cacheBackedEmbeddings,
  { ctx },
);
console.log(`Cached creation time: ${Date.now() - time}ms`);

Copy