Qdrant in Ypipe: Local Semantic Memory for AI Agents, What We Found Testing It

Most search works by matching text. Type a word, get results containing that word. That breaks the moment you phrase something differently than how you originally saved it. Qdrant solves that specific problem, and this piece covers what it actually is, how it got installed inside Ypipe as a fully local memory layer, exactly which tools it exposes and why nothing needed disabling, the real problems hit along the way, and genuine test results across every model size from 9B down to 90M.

Table of Contents

What Qdrant Actually Is

Qdrant (pronounced “quadrant”) is an open source vector database built specifically for storing and searching high-dimensional embeddings, the numerical representations of meaning that machine learning models produce from text, images, or other data. It’s written entirely in Rust, which is a deliberate choice for this kind of workload, Rust gives close to C++-level performance while eliminating a whole category of memory-safety bugs that plague lower-level systems languages.

The company behind it is based in Berlin, co-founded by Andre Zayarni as CEO and Andrey Vasnetsov as CTO, who wrote the original search engine himself, according to Qdrant’s own company history. Under the hood, Qdrant uses HNSW (Hierarchical Navigable Small World graphs) for its approximate nearest-neighbor indexing, the same general family of algorithm most serious vector search engines rely on for combining speed with high recall on large datasets.

Why Semantic Search Matters Here

The distinction that actually matters in practice: semantic search finds results based on meaning, not exact word matches. Store a note using one set of words, search for it later using completely different words, and it still gets found, because both pieces of text get converted into vectors that sit close together in the embedding space if their meaning is similar, regardless of the literal characters used.

This is the foundation of most local memory and RAG (retrieval-augmented generation) systems being built around local AI agents right now, an agent that can genuinely remember something and recall it later, without a human manually tagging or indexing every note.

How It Got Installed in Ypipe

The actual server used is mcp-server-qdrant, the official MCP server maintained directly by Qdrant’s own team, not a third-party wrapper. Like every other MCP integration in this series, it’s built on FastMCP, the framework by Jeremiah Lowin, and it launches through uvx, the tool-running command from Astral‘s uv project, no manual Python environment setup needed.

Getting it into Ypipe followed the same blueprint pattern as the earlier email integrations: a McpIntegrationBlueprint YAML file defining the launch command, the configuration schema for the install form, and overrides for each tool’s description and discovery hint so smaller models have a real shot at picking the right tool.

Running Fully Local: No Cloud, No API Key

This is the part that made Qdrant worth building into Ypipe specifically. The server supports a fully embedded local mode through the QDRANT_LOCAL_PATH environment variable, pointing it at a folder on disk instead of a remote server URL. In that mode, there’s no Qdrant Cloud account, no API key, no network call of any kind, everything gets written as local files, similar in spirit to how SQLite works compared to running a full separate database server.

The embeddings themselves are generated locally too, using FastEmbed, Qdrant’s own lightweight, ONNX-based embedding library, configured here with sentence-transformers/all-MiniLM-L6-v2, a small, well-established embedding model from Hugging Face. No embedding API call leaves the machine either. The entire pipeline, model inference, embedding generation, and vector storage, runs on one device.

The Tools It Exposes

Only two tools, deliberately minimal:

ToolWhat it does
qdrant-storeSaves a piece of text as a memory, optionally with metadata attached
qdrant-findSearches stored memories using natural language, matched by meaning

Both get a discoveryHint written specifically to distinguish them from each other, since testing showed models can blend “store” and “find” together when their descriptions read too similarly, more on that below.

Why Nothing Got Disabled

Every other MCP blueprint covered in this series (Gmail, the generalized custom-email integration) needed specific tools disabled by default, delete, move, and archive operations, since those are genuinely destructive and shouldn’t run without explicit intent. Qdrant’s tool set doesn’t have an equivalent, qdrant-store only adds data, qdrant-find only reads it. There’s no delete, update, or drop-collection tool exposed at all in this configuration, so there was nothing that needed locking down. Worth stating plainly rather than assuming, this wasn’t an oversight, it’s just a smaller, inherently safer tool surface.

Real Problems Hit During Setup

Three genuine issues came up, worth documenting precisely since none of them were obvious from the outside.

A Windows-specific dependency gap. The server failed on first install with No time zone found with key UTC, a known issue on Windows because Python’s zoneinfo module relies on an IANA timezone database that Windows doesn’t ship by default. Fixed by injecting the tzdata package directly into the launch command: uvx --with tzdata mcp-server-qdrant.

A cold-cache installation timeout. The first install attempt inside Ypipe failed after 120 seconds, timing out before uv finished downloading Python and building dependencies into Ypipe’s own isolated cache directory. Manually pre-warming that exact cache outside the 120-second window, using the same environment variables Ypipe sets internally, resolved it on the next attempt.

A tool-discovery gap on plain language. Asking a model in plain English to “store this as a memory” sometimes failed to surface qdrant-store at all, even though it was installed and enabled, apparently because its description and qdrant-find‘s description both centered heavily on the word “memory.” Naming the tool explicitly in the prompt worked around it reliably. This looked like a real weakness in Ypipe’s semantic tool-discovery step specifically, worth flagging separately from anything about Qdrant itself.

How It Actually Performed Across Model Sizes

The same two-part test ran across every model size available: store a memory, then in a completely fresh chat, search for it using deliberately different wording than what was stored, the real test of whether retrieval works by meaning rather than exact keyword overlap.

ModelResult
9BClean pass, both steps worked on the first try with plain language
7BStored correctly once, verified on disk, but crashed the underlying engine twice attempting the find step, a reproducible failure, not a fluke
4BClean pass with explicit tool naming, no crash
2BFailed twice first (a hallucinated tool name, then an unnecessary clarifying question), succeeded on the third attempt once the exact parameter name was spelled out
800MEventually stored correctly, but hallucinated an entirely unrelated tool from a different MCP integration mid-task, only self-corrected because Ypipe’s own recovery layer caught it

The 9B retrieval result is worth sitting with for a moment: the stored note read “Ypipe supports local semantic search using Qdrant, running entirely on-device with no cloud dependency.” The search query that found it was “anything about local search”, no mention of Qdrant, no mention of the word “semantic.” That’s genuine proof the retrieval works by meaning, not text overlap.

The Honest Summary

Reliability tracked closely with the pattern seen in earlier testing across this whole series: 9B and 4B worked cleanly, 2B needed real hand-holding to succeed at all, and 800M’s failure was structurally different from a simple wrong answer, it lost track of which MCP integration it was even supposed to be using, reaching for an email tool while being asked to store a note in a vector database.

What’s Next

The current work extends past testing into fine-tuning, training a small model specifically on the real tool-call format used across all of Ypipe’s MCP integrations, including these exact Qdrant tools, to see whether a purpose-trained small model can close the gap that plain prompting couldn’t.

FAQ

What is Qdrant used for? Qdrant is a vector database used to store and search embeddings, the numerical representations of meaning that machine learning models generate from text or other data. It’s commonly used for semantic search, recommendation systems, and giving AI agents persistent memory.

Does Qdrant require an internet connection or cloud account? No, not when configured in local mode using QDRANT_LOCAL_PATH. In that mode, everything runs and stores data entirely on the local machine, with no network calls, no API key, and no cloud account required.

What embedding model does this setup use? sentence-transformers/all-MiniLM-L6-v2, run locally through Qdrant’s own FastEmbed library, which uses the ONNX runtime rather than a heavier framework like PyTorch, keeping the footprint small.

Why weren’t any Qdrant tools disabled by default? Because neither exposed tool is destructive. qdrant-store only adds new data and qdrant-find only reads existing data, unlike the email blueprints in this series, there’s no delete, move, or overwrite operation in this tool set to lock down.

What size model is reliable for using Qdrant tools? Based on this testing, 9B and 4B models worked reliably. 2B required explicit, spelled-out prompting to succeed consistently, and 800M showed a deeper structural confusion, mistaking which installed tool integration it was even supposed to be using.

What does HNSW mean and why does Qdrant use it? HNSW stands for Hierarchical Navigable Small World, a graph-based algorithm for approximate nearest-neighbor search. It’s widely used in vector databases because it offers a strong balance of search speed and accuracy on large, high-dimensional datasets.

Is Qdrant open source? Yes, Qdrant is open source and written in Rust, with official client libraries for Python, JavaScript/TypeScript, Rust, Go, .NET, and Java.

What’s the difference between qdrant-store and qdrant-find? qdrant-store saves a piece of text as a new memory, optionally with structured metadata attached. qdrant-find searches existing memories using natural language, returning results ranked by semantic similarity rather than exact keyword matches.

Tags: