When we need semantic search over embeddings, vector similarity search is the natural tool. It finds items that are closest to a query vector using a distance metric such as cosine similarity or Euclidean distance.

I've written a dedicated blog on the basics, embeddings and VectorDistance() in EF Core 10 that you can read here if you want the full picture first, Vector Similarity Search in EF Core 10.

That approach is exact search. The database computes distance between the query vector and every row. That full scan gets slower as the table grows. Even with tuning, latency often stops matching what modern apps expect.

There is another path we will look at in this blog, approximate vector search, backed by vector indexes.

Approximate Vector Search

Approximate Nearest Neighbor (ANN) search looks for vectors that are close enough to the query without promising the single mathematically closest match. It relies on specialized structures and algorithms so the engine does not evaluate every row.

That introduces a deliberate speed vs accuracy trade-off. You may skip a marginally better match, but queries stay fast and scale far better. In most products, ranked results stay highly relevant while response times drop sharply.

Vector indexes are what make ANN practical. They organize embeddings so the server can jump to strong candidates instead of scanning the full table.

Getting Started

You'll need SQL Server 2025 (vector features and index types depend on that release) and the following package so the provider lines up with SQL Server's vector types.

Install-Package Microsoft.EntityFrameworkCore.SqlServer

Register DbContext as usual. Map the embedding column to a fixed-size VECTOR type so dimensions match your model

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Document>(entity =>
    {
        entity.Property(e => e.Embedding)
            .HasColumnType("VECTOR(1024)");
    });
}

A typical Document entity for search looks like this.

public sealed class Document
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string Content { get; set; }
    public SqlVector<float> Embedding { get; set; }
}

To create a vector index, your table must have a clustered primary key on a single column of type INT.

Creating a Vector Index

Official docs show the HasVectorIndex() method but at the time of writing this blog, this method as well as the VectorSearch() method are not exposed in the EF Core API yet, they are still in development, so we create the index with raw SQL.

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Document>(entity =>
    {
        entity.Property(e => e.Embedding)
            .HasColumnType("VECTOR(1024)");

        entity.HasVectorIndex(e => e.Embedding, "cosine")
    });
}

NOTE: Current vector index behavior treats the table as read-only while the index exists. Plan to bulk-load or finish writes first, then create the index. If you must change data later, you may need to drop the index, update rows, and recreate it.

Here is a DiskANN-style index with cosine distance.

CREATE VECTOR INDEX vec_idx ON Documents(Embedding)
WITH (METRIC = 'cosine', TYPE = 'diskann');
GO

Choosing a distance metric matches what we already covered for exact search, vector distance functions in the earlier post. Align METRIC with how you embed and rank (for example cosine, euclidean or manhattan) so index semantics and query metrics stay consistent.

Vector Search with Indexes

EF Core will expose something like VectorSearch() on DbSet in the future, until that ships, we call SQL Server's VECTOR_SEARCH table-valued function from EF Core using SqlQueryRaw and parameters.

Results include a distance column for ranking, smaller values mean closer matches under the chosen metric.

The commented block shows the intended VectorSearch shape, the active code uses raw SQL.

app.MapPost("/api/search", async (
    string query,
    ApplicationDbContext db,
    IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator) =>
{
    var embeddingResult = await embeddingGenerator.GenerateAsync([query]);
    var queryVector = new SqlVector<float>(embeddingResult.Single().Vector);

    // This is what the code would look like with the new VectorSearch method,
    // but since it's not implemented yet, we will use raw SQL

    //var results = db.Documents
    //    .VectorSearch(b => b.Embedding, "cosine", queryVector, topN: 5)
    //    .ToListAsync();

    var sql = """
        SELECT TOP(3)
            t.id,
            t.title,
            t.content,
            s.distance
        FROM VECTOR_SEARCH(
            TABLE = dbo.Documents AS t,
            COLUMN = Embedding,
            SIMILAR_TO = @qv,
            METRIC = 'cosine',
            TOP_N = 5
        ) AS s
        ORDER BY s.distance;
        """;

    var param = new Microsoft.Data.SqlClient.SqlParameter("@qv", queryVector);

    var results = await db.Database
        .SqlQueryRaw<DocumentSearchResult>(sql, param)
        .ToListAsync();

    return Results.Ok(results);
});

Conclusion

Exact vector search is a solid starting point. Once volume grows, full scans become the bottleneck.

ANN search with vector indexes targets a different goal, fast and relevant results instead of exhaustive comparison on every row.

Use exact search when datasets are small or correctness demands optimal results. Move to approximate indexed search when latency and scale matter and slightly softer guarantees are acceptable.

If you want to check out examples I created, you can find the source code here:

I hope you enjoyed it, subscribe and get a notification when new blog is up!