YPipe KnowledgePipes: Local Vector & Zero-ETL SQL RAG

Build sovereign local and on-premise Retrieval-Augmented Generation (RAG) architectures with YPipe KnowledgePipes using three composable building blocks: Stores, Scoped Contexts, and Assets.

KnowledgePipes supports zero-configuration embedded H2 vector stores and enterprise pgvector/MariaDB tables alongside zero-ETL read-only relational SQL databases and live HTTP REST APIs as direct RAG sources. With KnowledgePipe Contexts enforcing strict parameter isolation and Kubernetes-style .ypipe YAML manifests for GitOps, foreign records never leak into your AI prompt context.

Table of Contents

1. Key Takeaways
2. Composable RAG Architecture: Separating Stores, Contexts, and Assets
3. Embedded H2 & pgvector Alongside Zero-ETL Relational SQL and REST API RAG Sources
4. KnowledgePipe Context: Preventing Foreign Data Leakage by Design
5. Multi-Format Assets, Live SmartPipe RAG Sinks & GitOps YAML Manifests
6. The Three Declarative Building Blocks of YPipe KnowledgePipes
7. Step-by-Step Walkthrough Inside YPipe
8. Frequently Asked Questions
9. Related Articles & Video Series

Key Takeaways

• Composable RAG Architecture (Stores, Contexts & Assets): Cleanly separate storage engines (KnowledgePipe Store), scoped retrieval policies (KnowledgePipe Context), and ingestion bundles (KnowledgePipe Asset).
• Vector Stores + Zero-ETL Relational SQL & REST API RAG: Run embedded H2, PostgreSQL pgvector, or MariaDB vector stores—or query read-only SQL tables and HTTP REST APIs directly without an embedding pipeline.
• KnowledgePipe Context & Foreign Data Isolation: Lock retrieval scope with SpEL filter templates (e.g., patient_id or department) so YPipe Chat, SmartPipes, and YAgents never leak foreign records.
• Multi-Format KnowledgePipe Assets & SmartPipe RAG Sinks: Ingest folders of PDF, Word, PowerPoint, Markdown, AsciiDoc, HTML, JSONL, and CSV files, or stream live Whisper transcripts into a Store via a SmartPipe RAG Sink.
• GitOps-Ready .ypipe YAML Manifests: Persist every Store, Context, and Asset as a declarative Kubernetes-style YAML manifest for version control and headless on-premise deployment.

Composable RAG Architecture: Separating Stores, Contexts, and Assets

Traditional Retrieval-Augmented Generation (RAG) pipelines often tangle database credentials, document chunking scripts, and tenant access filters inside brittle Python code. Complementing our vector memory deep dive in Qdrant in Ypipe: Local Semantic Memory for AI Agents, YPipe KnowledgePipes structures local and on-premise enterprise knowledge into three clean, declarative building blocks:

1) KnowledgePipe Store (the storage backend and metadata filter schema); 2) KnowledgePipe Context (a scoped retrieval proxy that enforces hard access boundaries); and 3) KnowledgePipe Asset (reproducible multi-format document ingestion packages).

Embedded H2 & pgvector Alongside Zero-ETL Relational SQL and REST API RAG Sources

A KnowledgePipe Store lets you start immediately with a zero-configuration embedded H2 vector database on disk or scale to enterprise PostgreSQL pgvector and MariaDB vector stores.

Uniquely, KnowledgePipes also provides Zero-ETL Structured Grounding: you can connect existing relational SQL databases (PostgreSQL, MariaDB, MySQL, H2) via parameterized read-only SQL queries—or connect live HTTP REST APIs directly as RAG sources—without duplicating structured transactional tables into vector embeddings.

KnowledgePipe Context: Preventing Foreign Data Leakage by Design

In multi-patient clinical systems or multi-department enterprises, pointing an LLM at a shared vector store risks retrieving chunks belonging to another patient or restricted department. Instead of spinning up hundreds of separate databases, a KnowledgePipe Context sits on top of a Store and binds mandatory SpEL filter expressions (for example, locking patient_id or department).

When YPipe Chat, a SmartPipe workflow, or an autonomous YAgent queries that Context, the scope filter is enforced at the storage adapter layer—guaranteeing foreign records can never leak into the prompt context.

Multi-Format Assets, Live SmartPipe RAG Sinks & GitOps YAML Manifests

KnowledgePipe Assets ingest folders of Markdown, PDF, Word (`.docx`), PowerPoint (`.pptx`), AsciiDoc, HTML, pre-chunked JSON/JSONL arrays, and CSV tables with configurable token chunk sizes and overlap rules. You can also wire a SmartPipe RAG Sink to stream live workflow outputs—such as Whisper voice transcripts—directly into a Store, with every Store, Context, and Asset persisted as a Kubernetes-style .ypipe YAML manifest.

The Three Declarative Building Blocks of YPipe KnowledgePipes

Step-by-Step Walkthrough Inside YPipe

Step 1: KnowledgePipes — Vector Stores, Read-Only SQL DBs & REST API RAG Sources

Welcome to Knowledge-Pipes, the declarative Retrieval-Augmented Generation engine inside Y-Pipe, The Sovereign Local-First AI Platform. Knowledge-Pipes structures your local and on-premise knowledge layer into three composable building blocks: Stores, Contexts, and Assets. First, a Knowledge-Pipe Store defines the underlying storage engine and metadata schema. You can start immediately with the zero-setup embedded H-2 vector store on disk, or connect production on-premise PostgreSQL P-G-Vector and Maria-D-B vector tables. Even better, a unique specialty of Knowledge-Pipes is that you can connect traditional relational databases—like PostgreSQL and Maria-D-B—directly as read-only RAG sources using parameterized SQL queries, so for existing structured data you don’t have to worry about setting up a vector embedding or chunk ingestion pipeline at all. And in the exact same way, you can connect HTTP REST APIs directly as live RAG sources—turning any internal on-premise microservice or search endpoint into a governed retrieval source.

Step 2: KnowledgePipe Context — Scoped RAG & Foreign Data Isolation

Second, a Knowledge-Pipe Context strictly limits the retrieval scope on top of a Store, so foreign data never leaks into your AI context. Instead of spinning up separate databases for every patient, department, or project, the Context binds fixed filter templates and scope parameters—such as locking retrieval to a single patient ID or document category. When Y-Pipe, a Smart-Pipe, or a Y-Agent queries the Context, it is guaranteed to retrieve only the matching records—completely preventing foreign data, like another patient’s medical history, from ever entering the prompt.

Step 3: KnowledgePipe Asset — Multi-Format RAG Content & Data Types

Third, Knowledge-Pipe Assets let you package and ingest diverse RAG data types reproducibly. Opening the Asset Type selector, an Asset can ingest entire directory folders of Markdown, PDF, Word, PowerPoint, and HTML files, single standalone documents, pre-chunked JSON and JSON-Lines arrays, or structured CSV tables. Each Asset also defines chunk sizes, overlap windows, and default metadata tags like category and document type—so with a single click, you can ingest any Asset into a Store, automatically splitting, embedding, and indexing the content locally.

Step 4: SmartPipes-to-RAG Ingestion & Kubernetes-Style .ypipe DevOps Manifests

Finally, you can also stream live workflow outputs directly into a Store using a Smart-Pipe RAG Sink—so results like Whisper voice transcripts are indexed automatically and become immediately searchable through your Context. And for Dev-Ops teams, every Knowledge-Pipe Store, Context, and Asset is saved under the hood as a clean, Kubernetes-style dot-ypipe YAML manifest—making your entire local and on-premise RAG architecture version-controllable in Git and ready for automated deployment.

Frequently Asked Questions

What is YPipe KnowledgePipes? KnowledgePipes is YPipe’s declarative local and on-premise Retrieval-Augmented Generation (RAG) engine, built around composable Stores, Scoped Contexts, and Assets.

Do I need to set up an external vector database to use KnowledgePipes? No. KnowledgePipes includes a zero-setup embedded H2 vector database out of the box, and also supports enterprise PostgreSQL (pgvector) and MariaDB.

What is Zero-ETL Read-Only Relational SQL RAG? It allows KnowledgePipes to query existing PostgreSQL or MariaDB tables directly using pre-approved, parameterized read-only SQL SELECT statements without copying or embedding the rows into a vector database first.

How does a KnowledgePipe Context prevent data leakage? A KnowledgePipe Context binds mandatory metadata or SQL filter parameters (such as a specific patient ID or department name) so queries routed through that Context can never retrieve records outside their authorized scope.

What document formats can a KnowledgePipe Asset ingest? PDF, Microsoft Word (DOCX), PowerPoint (PPTX), Markdown, AsciiDoc, HTML, CSV spreadsheets, and pre-chunked JSON/JSONL files.

Can live SmartPipe workflows write new knowledge into KnowledgePipes? Yes. Any SmartPipe workflow can attach a KnowledgePipe RAG Sink to automatically chunk, embed, and store its output in a KnowledgePipe Store.

Can I test retrieval quality before connecting an AI agent? Yes. Every KnowledgePipe Store and Context includes an interactive Query Sandbox showing exact matched chunks, similarity scores, and metadata attributes.

How are KnowledgePipes configurations deployed to production? Every Store, Context, and Asset is saved as a declarative .ypipe YAML manifest that DevOps teams can version in Git and deploy headlessly on-premise.

Related Articles & YPipe Video Series

YPipe by iunera is The Sovereign Local-First AI Platform—running 100% locally on workstations or on-premise on your enterprise and clinic servers with full GDPR, HIPAA, and EU AI Act sovereignty. Explore our related engineering studies on Kraken and the full video series below:

Related Kraken Articles on YPipe & Local AI

• Qdrant in Ypipe: Local Semantic Memory for AI Agents, What We Found Testing It
• PostgreSQL MCP Server in Ypipe: Engineering Secure Transactional AI Memory
• MariaDB MCP Server in Ypipe: Engineering Reliable Relational AI
• chDB (Embedded ClickHouse) MCP Server in Ypipe: Serverless OLAP for AI Agents

Explore All Episodes in the YPipe Video Series

• Episode 01: YPipe — The Sovereign Local-First AI Platform | Manifold Engine, MCP, SmartPipes & On-Premise RAG
• Episode 02: Re:Think Local AI — 800M Qwen 3.5 & YPipe Manifold Engine: Tool Calling & Reasoning on Laptops
• Episode 03: YPipe SmartPipes — Visual Multi-Model AI Workflows (Whisper, Vision OCR, REST Webhooks & MCP Tools)
• Episode 04 (This Article): YPipe KnowledgePipes — Declarative Local & On-Premise RAG | Vector DBs, SQL & Scoped Contexts

Official YPipe Resources

• Official YPipe Website: https://ypipe.com
• iunera Company Website: https://www.iunera.com
• GitHub Repository: https://github.com/iunera/ypipe
• Full YouTube Video Series Playlist: https://www.youtube.com/playlist?list=PLIB7VtTuXX3Y