Skip to content

Add support for Vector Search #4882

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed

Add support for Vector Search #4882

wants to merge 6 commits into from

Conversation

mp911de
Copy link
Member

@mp911de mp911de commented Jan 20, 2025

We now support MongoDB's Vector Search through a Vector Search Aggregation stage. Also, we can create and remove Vector search indices and support Spring Data's Vector abstraction:

Model

class MyDocument {

  ObjectId id;
  Vector vector;
}


MyDocument d = new MyDocument();
d.vector = Vector.of(/* some float array values */);

d.vector = MongoVector.of(BinaryVector.floatVector(/* some float array values */));

d.vector = MongoVector.of(BinaryVector.int8Vector(/* some byte array values */));

d.vector = MongoVector.of(BinaryVector.int8Vector(/* some packed byte array values */, padding));

Index Creation

VectorIndex vectorIndex = new VectorIndex("movie_vector_index")
		.addVector("plot_embedding", field -> field.dimensions(1536).similarity(COSINE))
		.addFilter("language");
template.searchIndexOps("mflix").ensureIndex(vectorIndex);

Vector Search

VectorSearchOperation $vectorSearch = VectorSearchOperation.search("movie_vector_index")
    .path("plot_embedding")
    .vector(vectors)
    .limit(10)
    .numCandidates(150)
    .withSearchScore();

Aggregation agg = Aggregation.newAggregation($vectorSearch, Aggregation.project("plot", "title"));

AggregationResults<Document> aggregate = template.aggregate(agg, "mflix", Document.class);

Depends on

@mp911de mp911de added the type: enhancement A general enhancement label Jan 20, 2025
@mp911de mp911de linked an issue Jan 20, 2025 that may be closed by this pull request
christophstrobl and others added 4 commits February 3, 2025 15:08
so there are still test failures. let's check tomorrow

Still open: binaryvector failures in integration tests

split tests for native array and BinaryVector

introduce internal search field abstraction
remove methods on DefaultSearchIndexOperations

Introduce SearchIndexDefinition -> up next a litte documentation

revert some (no longer needed) changes

Update documentation

upgrade antora-ui-spring

tests seem to run fine after container update
Remove Field type. Refactor container to subclass MongoDBAtlasLocalContainer. Introduce wait/synchronization to avoid container crashes on create index + list search indexes.
mp911de pushed a commit that referenced this pull request Feb 4, 2025
mp911de pushed a commit that referenced this pull request Feb 4, 2025
mp911de added a commit that referenced this pull request Feb 4, 2025
mp911de pushed a commit that referenced this pull request Feb 4, 2025
See #4706
Original pull request: #4882
mp911de added a commit that referenced this pull request Feb 4, 2025
Remove Field type. Refactor container to subclass MongoDBAtlasLocalContainer. Introduce wait/synchronization to avoid container crashes on create index + list search indexes.

See #4706
Original pull request: #4882
@mp911de mp911de closed this Feb 4, 2025
@mp911de mp911de deleted the issue/4706 branch February 4, 2025 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for VectorSearch Aggregation Stage
3 participants