Open-Weights GLiNER Neuro-Symbolic Symbol Extractor (INT8 ONNX): 22 ms CPU/NPU Schema.org JSON-LD & Clinical Relation Extraction in Python and Pure Java 17+

In Neuro-Symbolic AI architecture (Kautz, AAAI 2020; Kambhampati et al., ICML 2024), fast System 1 neural perception must translate messy human prose into typed entities, exact character offsets, and semantic relations before System 2 symbolic engines evaluate domain ontologies and compliance rules. Today, iunera—creators of YPipe — The Sovereign Local-First AI Platform—is releasing ypipe/gliner-neuro-symbolic-extractor under the Apache-2.0 license alongside a free live interactive CPU demo Space (ypipe/gliner-neuro-symbolic-space). Built gratefully upon Knowledgator's GLiNER-RelEx and GLiNER (DeBERTa-v3), our 283 MB (base) and 655 MB (large) INT8 ONNX research prototypes extract Schema.org W3C JSON-LD graphs and structured clinical values in a single 22.0 ms–54.7 ms laptop CPU forward pass—delivering 0% substring mutation by construction and 148,085 docs/hr in-process inside pure Java 17+ Big-Data streams.

Open-Weights Resources & Research Prototype Notice

  • Model Weights, ONNX Graphs & Java/Python SDKs (Apache-2.0): huggingface.co/ypipe/gliner-neuro-symbolic-extractor
  • Live Free Interactive CPU Demo Space (Zero Install): huggingface.co/spaces/ypipe/gliner-neuro-symbolic-space
  • Research Prototype Framing (No Warranties): Published strictly as an open-weights Research Prototype & Proof of Concept for neuro-symbolic AI research, architectural evaluation, and prototyping (not a certified medical device). Validated primarily on English (en) text; behavior on German and other languages is experimental.
Neuro-Symbolic Symbol Extraction: System-1 GLiNER INT8 + System-2 Logic
Neuro-Symbolic Symbol Extraction: System-1 GLiNER INT8 + System-2 Logic

Table of Contents

  1. Stop Burning Autoregressive LLM Compute to Parse Structured Entities & Relations
  2. Inside ypipe/gliner-neuro-symbolic-extractor: Training Scope, Domain Transferability & INT8 ONNX Engineering
  3. Four Reproducible Empirical Studies (Schema.org F1, Dosage Fidelity, Edge NPU Scaling & Pure Java 17+)
  4. Global Healthcare Vision: Private, Low-Compute Patient-to-Study Matching Worldwide
  5. In Plain English for Business Leaders: The "Optical Highlighter + Certified Rule Auditor" Analogy
  6. Try It Live (Hugging Face Space), ~50-Line DIY Python Loop & Enterprise Scale With YPipe
  7. Frequently Asked Questions (FAQ)
  8. Acknowledgments, Scientific References & Further Reading

1. Stop Burning Autoregressive LLM Compute to Parse Structured Entities & Relations

Across enterprise data engineering, healthcare informatics, and RAG pipelines, one architectural anti-pattern dominates modern AI budgets: routing every incoming paragraph, supplier quote, Kafka event, or electronic health record (EHR) note through an autoregressive Large Language Model in "JSON mode" just to extract named entities and relationships.

Peer-reviewed computer science and empirical benchmarks expose three structural flaws in using generative next-token predictors as high-volume schema extractors:

  1. Sequential O(N_out) Decode Overhead vs. Single-Pass O(1) Perception: To emit a 130-token JSON object describing six Schema.org entities and three properties, an autoregressive decoder must execute 130 sequential forward passes, re-streaming hundreds of megabytes or gigabytes of model weights and growing KV-cache tensors out of RAM on every single token. By contrast, a bidirectional span-and-relation encoder (Zaratiana et al., NAACL 2024, arXiv:2311.08526) scores all entity boundaries and pairwise relations simultaneously in one parallel forward pass.
  2. Generative Token Re-Synthesis vs. Verbatim Pointer Slicing (Kalai & Vempala, ACM STOC 2024, arXiv:2311.14648): When a generative LLM writes {"dosage": "0.125 mg"}, it has no structural character-index pointer back to the input string—it synthesizes every digit and unit token from probability distributions. Compact generative models can silently re-case strings, invent label meta-tokens, or drop units, and even large models return detached strings without guaranteed [startChar..endChar] character provenance for UI highlighting or regulatory audit trails.
  3. Long-Context Dilution & The Cloud Data Trap (Liu et al., TACL 2024, arXiv:2307.03172): Stuffing multi-page documents into external cloud APIs degrades middle-context recall while exposing confidential patient files, HR records, and pricing contracts to third-party servers and opaque safety-classifier retention policies.

Inside YPipe — The Sovereign Local-First AI Platform, the neuro-symbolic YPipe Manifold Engine solves this by separating System 1 neural perception (compact local encoders that convert unstructured text into offset-locked symbols on CPU/NPU) from System 2 deterministic reasoning (ontologies, preconditions, and formal verification). Today we are releasing an open-weights standalone research prototype of this fast neural perception pattern so every Python and Java engineer can test it firsthand.


2. Inside ypipe/gliner-neuro-symbolic-extractor: Training Scope, Domain Transferability & INT8 ONNX Engineering

We gratefully built ypipe/gliner-neuro-symbolic-extractor upon Knowledgator’s pioneering open-weights GLiNER-RelEx models (knowledgator/gliner-relex-base-v1.0 and knowledgator/gliner-relex-large-v1.0), which extend Urchade Zaratiana et al.’s generalist bidirectional NER architecture (GLiNER) and Microsoft Research’s disentangled-attention DeBERTa-v3 backbones (He, Gao, & Chen, ICLR 2023, arXiv:2111.09543) with a joint pairwise relation-scoring head.

What This Research Prototype Was Fine-Tuned On—And Why It Transfers to Other Verticals

To evaluate whether a compact 225M–459M encoder (283 MB–655 MB INT8) can serve as a universal schematic front-end for neuro-symbolic pipelines, we fine-tuned both checkpoints across four complementary structural distributions:

  1. Official Schema.org Entity Classes & Relational Properties: Broad coverage across people (Person), organizations (Organization), products (Product), software (SoftwareApplication), locations (Place), events (Event), monetary values (MonetaryAmount), and relational predicates (founder, worksFor, location, manufacturer, price, author, memberOf).
  2. Quantitative Units, Dosages & Clinical Patient-File Relations: Medications, numeric strengths/dosages, administration frequencies, clinical conditions/indications, laboratory biomarkers (eGFR, INR, HbA1c, potassium), physical/clinical units (mg, mcg, mL/min, mmol/L, EUR, USD), and directional relations (treats, has_dosage, has_frequency, has_value, contraindicated_with).
  3. Software, Technical & Operational Parameters: System components, operational states, and attribute-value pairs common in DevOps and industrial logs.
  4. Hard-Negative & Interrogative Calibration: Uninstantiated questions (e.g., “Which medication treats hypertension?”) so the extractor distinguishes asserted factual instances from generic query nouns.

Because GLiNER-RelEx embeds the caller’s target schema dynamically inside the input prompt (<<ENT>> … <<SEP>> <<REL>> … <<SEP>>) alongside the text in one bidirectional pass, this approach transfers cleanly to virtually any structured industry vertical—including banking KYC & UBO graphs, maritime bills of lading (ICC UCP 600), customs HS tariff screening, industrial IoT telemetry, and legal contract auditing—without deploying 70B+ GPU clusters.

Architect’s Field Note: Three First-Hand Engineering Scars From Exporting & Quantizing GLiNER-RelEx to INT8 ONNX

Engineering Deep-Dive: Selective INT8 Quantization & Dynamic RelEx ONNX
Engineering Deep-Dive: Selective INT8 Quantization & Dynamic RelEx ONNX

First-Hand Systems Engineering E-E-A-T (What Broke During ONNX Export & How We Fixed It)

Getting joint entity-and-relation extraction running deterministically in pure C++/Java ONNX Runtime at INT8 precision required solving three subtle low-level bugs that standard export tutorials miss:

  1. Mixed-Precision Quantization (Keeping the Bi-LSTM in FP32): Naive full-graph dynamic INT8 quantization (quantize_dynamic across all operator types) quantized the recurrent gates inside GLiNER's 1-layer bidirectional LSTM word-projection layer—causing subtle trailing-word boundary drift on multi-word entities ("Munich," vs. "Munich, Germany"). Restricting INT8 quantization strictly to MatMul, Gemm, and Gather while preserving the LSTM operator in exact FP32 slashed base size by 67% (861 MB → 283 MB) and accelerated CPU inference by 3.2x with zero boundary degradation.
  2. Eliminating Frozen Python .item() Trace Shapes: During torch.onnx.export, upstream Python .item() calls inside target-embedding selection and entity-pair construction froze trace-time tensor shapes—causing ScatterND runtime crashes whenever a sentence contained >= 3 entities in large or returning 0 relations in base. Replacing scalar .item() branches with dynamic nonzero-gather ops and encoding rel_idx onto a deterministic rectangular span grid (start_word * 12 + width) unlocked arbitrary entity/relation counts in one static ONNX graph.
  3. Zero-Pickle model.safetensors & Clean models/<variant>/onnx/ Subdirectory Packaging: Both models/base/ (~225M, token_level BIO) and models/large/ (~459M, markerV0 span-level) ship with zero-pickle model.safetensors plus isolated onnx/model.onnx and onnx/model_quantized.onnx files—allowing lightweight Java and Python containers to pull only the 283 MB INT8 ONNX artifact and tokenizer JSONs without downloading PyTorch checkpoints.

3. Four Reproducible Empirical Studies (With Preliminary Scope & Larger-LLM Nuance)

All benchmark harnesses and raw JSON outputs are published openly in benchmarks/ and were executed on a single standard business laptop CPU (AMD Ryzen 5 PRO 7540U Zen 4, 6 Cores / 12 Threads, Linux x86_64, zero GPU) against native C++ llama.cpp (llama-server build b11541 with Zen 4 AVX-512, structured JSON mode, warm session, –reasoning-budget 0).

Study 1 (Preliminary Assessment, N = 8): Schema.org JSON-LD Extraction vs. Upstream & llama.cpp

In our first preliminary micro-benchmark (N = 8 multi-clause enterprise, software, industrial, and clinical passages in benchmarks/schema_org_benchmark.py), we measured single-pass extraction of official Schema.org classes and properties serialized directly into valid W3C JSON-LD 1.1:

Preliminary Assessment: Schema.org JSON-LD — INT8 vs. llama.cpp
Preliminary Assessment: Schema.org JSON-LD — INT8 vs. llama.cpp
Model / Engine (N=8)RuntimeLatency10K DocsSpeedupEnt F1Rel F1Mutations
ypipe/gliner-neuro-symbolic (base INT8)ONNX Runtime CPU22.0 ms3.7 min ($0)398x faster98.3%74.4%0 (substring)
ypipe/gliner-neuro-symbolic (large INT8)ONNX Runtime CPU54.7 ms9.1 min ($0)160x faster98.2%82.1%0 (substring)
knowledgator/gliner-relex-base-v1.0PyTorch FP32 CPU118.9 ms19.8 min74x faster96.7%74.5%0 (substring)
knowledgator/gliner-relex-multi-v1.0PyTorch FP32 CPU117.7 ms19.6 min74x faster96.7%75.0%0 (substring)
Qwen3.5-0.8B (IQ4_XS GGUF, JSON mode)Native C++ llama.cpp3,209.7 ms8.9 hrs2.7x77.8%41.0%2 invented
Qwen3.5-2B (IQ4_XS GGUF, JSON mode)Native C++ llama.cpp8,745.7 ms24.3 hrs1.0x (baseline)84.6%77.8%0
Larger Local / Frontier Cloud LLMs (7B–70B+ JSON Mode)GPU / External Cloud~1.5–15+ s~4–40+ hrs + GPU/API costGPU / Cloud-bound~96–99% (Clearly higher)~85–95% (Clearly higher)Rare (no char offsets)

Study 2 (Preliminary Assessment, N = 20): Clinical Dosage & Numeric Fidelity Stress Test—And Fair Larger-LLM Comparison

To test substring fidelity on safety-critical numbers, we ran a preliminary 20-sentence stress assessment (benchmarks/dosage_and_numeric_mutation_benchmark.py) across clinical decimals (0.125 mg, 0.08 mcg/kg/min), renal biomarkers (27.5 mL/min, 4.2 mmol/L), and financial prices (2490.50 EUR):

Preliminary Assessment: Substring Slicing vs. LLM Synthesis
Preliminary Assessment: Substring Slicing vs. LLM Synthesis
Model / Engine (N=20)RuntimeMean LatencyExtracted SpansNon-Verbatim SpansGold Numeric Recall
ypipe/gliner-neuro-symbolic (base INT8)ONNX Runtime CPU21.8 ms840 (100% Verbatim Slice)91.7%
ypipe/gliner-neuro-symbolic (large INT8)ONNX Runtime CPU69.2 ms790 (100% Verbatim Slice)86.1%
Qwen3.5-0.8B (IQ4_XS GGUF, JSON mode)Native C++ llama.cpp1,856.0 ms907 non-verbatim ('Dosage', 'Orally', re-cased strings)63.9%
Qwen3.5-2B (IQ4_XS GGUF, JSON mode)Native C++ llama.cpp6,134.0 ms1070 (282x slower than base INT8)97.2%
  • Scientific Honesty — Larger LLMs Are Clearly Better on Recall: Look closely at the progression from Qwen3.5-0.8B (63.9% numeric recall, 7 non-verbatim strings including the invented meta-word 'Dosage') up to Qwen3.5-2B (97.2% numeric recall and 0 non-verbatim strings)—and larger 7B–70B+ local GPU models and frontier cloud LLMs are clearly stronger still on zero-shot coverage, implicit relations, and complex sentence structures.
  • The Exact Pareto Trade-Off: We do not claim a 283 MB encoder replaces 7B–70B+ LLMs on open-ended comprehension. Rather, reaching 97.2% recall with 2B already required 6.13 seconds per sentence (282x slower than base INT8’s 21.8 ms) and still emitted detached strings without [startChar..endChar] offset pointers. For high-throughput schematic perception, a 22 ms extractive encoder provides instant, offset-locked local symbols at 1/300th the compute cost.

Study 3: Pure Computational Scaling (O(1) vs. O(N_out)) & Why Edge NPUs Multiply the Advantage

Scaling from a single test sentence to 100,000 or 1,000,000 enterprise documents reveals why computational complexity (O(1) encoder pass vs. O(N_out) serial decode steps) governs infrastructure architecture:

Computational Scaling: Single-Pass O(1) Encoder vs. O(N) LLM Decode
Computational Scaling: Single-Pass O(1) Encoder vs. O(N) LLM Decode
  • At 1,000,000 Documents on One CPU: ypipe/gliner-neuro-symbolic (base INT8) executes exactly 1,000,000 parallel matrix passes (6.1 CPU-hours), whereas llama.cpp Qwen3.5-2B executes ~135,000,000 sequential token-decode passes (101.2 CPU-days — 398x more compute).
  • Why Single-Pass INT8 Graphs Excel Even Further on Edge NPUs (AMD Ryzen AI XDNA, Intel Core Ultra AI Boost, Apple ANE, Qualcomm Hexagon): Autoregressive batch=1 LLM decoding is starved by LPDDR5 memory bandwidth because weights and growing KV-caches must stream from RAM on every generated token. By contrast, our 283 MB models/base/onnx/model_quantized.onnx graph processes prompt labels and document tokens together in one dense static [seq_len, hidden_dim] INT8 MatMul pass with zero KV-cache, saturating NPU systolic SRAM tiles via ONNX Runtime execution providers (VitisAI, OpenVINO, CoreML, QNN) while leaving 100% of host CPU cores free.

Study 4: Enterprise Java & Big-Data Streaming—In-Process Pure JVM ONNX vs. Python Sidecar Sprawl

For data engineering teams running Apache Kafka, Apache Flink, Apache Spark, Apache Druid, ClickHouse, or HAPI HL7 FHIR pipelines on the JVM, deploying Python FastAPI/PyTorch NLP sidecars introduces network serialization latency, Python GIL contention, and multi-gigabyte container sprawl.

Because ypipe/gliner-neuro-symbolic-extractor ships with a standalone pure Java 17+ implementation (GlinerOnnxExtractor.java) requiring only 3 Maven dependencies (com.microsoft.onnxruntime:onnxruntime, ai.djl.huggingface:tokenizers, Jackson—zero Spring, zero Python), stream workers execute INT8 extraction directly in-memory:

Big-Data JVM Architecture: Python Sidecars vs. In-Process ONNX
Big-Data JVM Architecture: Python Sidecars vs. In-Process ONNX
JVM / Sidecar ArchitectureIPC / Network OverheadWorker RSSLatency (Med/p95)Docs/SecDocs/Hr
In-Process Java 17+ (GlinerOnnxExtractor, 3-Thread Shared OrtSession)Zero IPC (In-Memory)935 MB (entire JVM)— (parallel stream)41.1 docs/s148,085 docs/hr (5.8x)
In-Process Java 17+ (GlinerOnnxExtractor, Single-Thread)Zero IPC (In-Memory)935 MB (entire JVM)30.9 ms / 32.1 ms32.5 docs/s116,917 docs/hr (4.6x)
Python PyTorch + HTTP JSON Microservice Sidecar (FP32)JSON → TCP/HTTP → GIL → JSON2,638 MB (+ JVM client)141.1 ms / 152.5 ms7.0 docs/s25,335 docs/hr (1.0x)

4. Global Healthcare Vision: Private, Low-Compute Patient-to-Study Matching Worldwide

One of our primary motivations for fine-tuning and releasing this open-weights research prototype is advancing equitable, privacy-preserving clinical decision support—especially for hospitals and clinics in lesser-developed and resource-constrained regions:

Vision: Private, Low-Compute Patient-to-Study Matching Worldwide
Vision: Private, Low-Compute Patient-to-Study Matching Worldwide
  1. Strict Non-Sharing of Confidential Medical Data (Stage 1A): Patient files never leave the clinic’s local PC. In ~22–30 ms on a $300–$500 offline laptop CPU, gliner-neuro-symbolic-extractor extracts verbatim medications, dosages, conditions, and renal/metabolic biomarkers (eGFR, HbA1c) on-premise.
  2. Offline Distribution of Pre-Extracted Medical Studies (Stage 1B): Millions of global clinical trial protocols and dosing guidelines are extracted once into compact symbolic eligibility schemas—shippable offline on a USB drive or local disk without requiring datacenter GPUs or broadband cloud budgets.
  3. Deterministic Symbolic Matching & Safety Validation (Stages 2 & 3): A local symbolic solver evaluates exact biomarker thresholds (e.g., matching eGFR 28 mL/min into [20..45] renal-safe trials while deterministically blocking Metformin at eGFR < 30 mL/min) in < 10 ms, presenting physicians with exact character offsets and auditable rule proofs.

5. In Plain English for Business Leaders: The “Optical Highlighter + Certified Rule Auditor” Analogy

The 30-Second Executive Analogy: Using a generative LLM in JSON mode to process 1,000,000 invoices or patient notes is like hiring a novelist to read each page and re-type every SKU, dosage, and price digit-by-digit from memory—it takes 130+ steps per page and risks silent transcription drift. Neuro-Symbolic AI pairs an Optical Highlighter (GLiNER INT8, marking the exact characters directly on the original page in 22 ms with 0% re-typing) with a Certified Rulebook Auditor (a < 5 ms CPU symbolic engine that checks every contract, ontology, and compliance law with 100% mathematical certainty).

In Plain English: Manual Re-Typing vs. Optical Highlighter + Rule Auditor
In Plain English: Manual Re-Typing vs. Optical Highlighter + Rule Auditor

6. Try It Live, Build a ~50-Line Neuro-Symbolic Loop Yourself, or Scale On-Premise With YPipe

A. Test It Right Now in Your Browser — Live Hugging Face Space Walkthrough (Zero Install)

Paste any English clinical summary or business paragraph into our free Hugging Face Space—huggingface.co/spaces/ypipe/gliner-neuro-symbolic-space—and toggle between models/base (283 MB INT8 ONNX) and models/large (655 MB INT8 ONNX) directly in your browser:

  1. Tab 1 — Live Single-Pass Patient File Clinical Extraction (models/base/onnx/model_quantized.onnx): Highlights verbatim substring spans (type 2 diabetes, Metformin, 500, twice daily, severe renal failure, eGFR, 28 mL/min) alongside directional clinical relations (Metformin –[treats]–> type 2 diabetes, Metformin –[has_dosage]–> 500, Metformin –[contraindicated_with]–> severe renal failure) with exact [start..end] character offsets and 0% substring mutation:
Live Hugging Face Space Demo (Tab 1): Single-Pass Patient File Clinical Entity & Relation Extraction with Exact Substring Offsets
Live Hugging Face Space Demo (Tab 1): Single-Pass Patient File Clinical Entity & Relation Extraction with Exact Substring Offsets
  1. Tab 2 — Live Single-Pass Schema.org → W3C JSON-LD Extraction (models/large/onnx/model_quantized.onnx): Extracts Person (Dr. Lena Hoffmann), Organization (Helios MedTech GmbH), Product (CardioSense Pro), MonetaryAmount (2490 EUR), and Event (European Cardiology Summit) simultaneously with founder, manufacturer, and price relations, serializing them deterministically into valid https://schema.org JSON-LD:
Live Hugging Face Space Demo (Tab 2): Single-Pass Schema.org Entity & Property Extraction Serialized into Valid W3C JSON-LD
Live Hugging Face Space Demo (Tab 2): Single-Pass Schema.org Entity & Property Extraction Serialized into Valid W3C JSON-LD

B. Vendor-Neutral DIY Blueprint (~50 Lines of Python + Symbolic Safety Verifier)

You do not need a full platform to build a single narrow neuro-symbolic workflow! Using our Apache-2.0 examples/python/extract_onnx.py (onnxruntime + tokenizers + numpy, zero PyTorch), here is a complete System 1 Perception + System 2 Clinical Contraindication Gate in ~45 lines of Python:

import re
from pathlib import Path
from huggingface_hub import snapshot_download
from extract_onnx import GlinerOnnxExtractor  # from ypipe/gliner-neuro-symbolic-extractor

# 1. Pull ONLY the 283 MB INT8 ONNX graph + tokenizer JSONs from models/base/
repo_root = Path(snapshot_download(
    repo_id="ypipe/gliner-neuro-symbolic-extractor",
    allow_patterns=[
        "models/base/onnx/model_quantized.onnx",
        "models/base/*.json",
        "models/base/spm.model",
    ],
))
extractor = GlinerOnnxExtractor(repo_root / "models" / "base", onnx_filename="onnx/model_quantized.onnx")

# 2. System 1 Neural Perception: Single-pass offset-locked extraction (~22 ms CPU)
note = (
    "Patient File Summary: 64-year-old patient with type 2 diabetes prescribed Metformin 500 mg "
    "twice daily; contraindicated in severe renal failure after eGFR dropped to 28 mL/min."
)
entities, relations = extractor.extract(
    text=note,
    entity_labels=["medication", "dosage", "frequency", "condition", "biomarker", "measurement"],
    relation_labels=["has_dosage", "has_frequency", "treats", "contraindicated_with", "has_value"],
)

# 3. System 2 Deterministic Symbolic Safety Gate (< 1 ms CPU, 0% guesswork)
RENAL_EGFR_MIN_LIMIT_ML_MIN = {"metformin": 30.0}
prescribed = {e.text.lower(): e for e in entities if e.label == "medication"}
egfr_vals = [
    float(m.group(1))
    for e in entities if e.label == "measurement"
    for m in [re.search(r"(\d+(?:\.\d+)?)\s*mL/min", e.text, re.I)] if m
]

for drug_key, drug_span in prescribed.items():
    if drug_key in RENAL_EGFR_MIN_LIMIT_ML_MIN and egfr_vals:
        limit = RENAL_EGFR_MIN_LIMIT_ML_MIN[drug_key]
        if min(egfr_vals) < limit:
            print(
                f"[SYMBOLIC SAFETY BLOCK] '{drug_span.text}' (chars {drug_span.start}..{drug_span.end}) "
                f"BLOCKED: measured eGFR {min(egfr_vals)} mL/min < {limit} mL/min threshold!"
            )

C. Scaling Across the Enterprise With YPipe — The Sovereign Local-First AI Platform

When IT-affine business analysts and DevOps teams move from a single script to dozens of governed workflows, multi-step database agents, and enterprise-wide RAG, YPipe — The Sovereign Local-First AI Platform by iunera provides the complete neuro-symbolic stack out of the box on vanilla Windows, macOS, Linux, and on-premise servers:

  • ypipe Manifold Engine: Takes the lead over local models (800M to 32B, sized automatically to your hardware via Auto-Gear in Engine Foundry) to enforce deterministic reasoning and fulfill YPipe’s mission to beat 70B+ parameter models with local engines in deterministic scopes—as proven in our 3,829-test UC Berkeley BFCL V4 study.
  • KnowledgePipes: Hydrates unstructured PDFs, clinical guidelines, and SQL/OLAP tables (PostgreSQL, MariaDB, ClickHouse, Apache Druid, chDB) into sovereign local KnowledgePipe Stores, KnowledgePipe Contexts, and KnowledgePipe Assets.
  • SmartPipes & Couplings: Visually chains multi-model pipelines into REST webhooks, web forms, and reusable MCP tools—while bundling Node.js, Python, and Java runtimes out of the box and tuning tool parameters in Couplings so tool-calling loops on compact models become history.

👉 Want to evaluate YPipe on your own local or on-premise data? Contact us via ypipe.com or iunera.com/contact to request an evaluation build or schedule a tailored live demo.


7. Frequently Asked Questions (FAQ)

Q1: Where can I download ypipe/gliner-neuro-symbolic-extractor and test it online? All open-weights checkpoints (models/base/ and models/large/ in zero-pickle model.safetensors, FP32 ONNX, and INT8 quantized ONNX), benchmarks, and standalone Python and Java 17+ examples are available under the Apache-2.0 license at huggingface.co/ypipe/gliner-neuro-symbolic-extractor. You can also test it immediately in your browser with zero installation on our free CPU Space at huggingface.co/spaces/ypipe/gliner-neuro-symbolic-space.

Q2: What is the difference between models/base/ and models/large/? models/base/ (~225M parameters, token_level BIO, 283 MB INT8 ONNX) is optimized for ultra-low-latency CPU/NPU perception (~22–31 ms/doc) and high-throughput JVM Big-Data streams (148,085 docs/hr). models/large/ (~459M parameters, markerV0 span-level, 655 MB INT8 ONNX) delivers higher relational recall (82.1% Schema.org Relation F1 vs. 74.4% in our preliminary micro-benchmark) at ~55–100 ms/doc on CPU.

Q3: Does this 283 MB model beat 7B–70B+ Large Language Models on every extraction task? No—and stating that clearly is essential for good engineering! Larger LLMs (7B–70B+ local GPU models and frontier cloud APIs) are clearly stronger on open-ended zero-shot recall, nuanced/implicit phrasing, and complex cross-sentence reasoning. Even in our 20-case dosage micro-benchmark, scaling llama.cpp from 0.8B (63.9% recall) to 2B (97.2% recall) outperformed our prototype’s raw recall (91.7%). Our prototype’s advantage is the pareto compute and safety trade-off: 22 ms deterministic O(1) CPU speed (280x–398x less compute), 935 MB in-process Java execution, 100% on-premise privacy, and structural text[startChar:endChar] substring slicing (0% digit re-typing risk).

Q4: How does gliner-neuro-symbolic-extractor guarantee 0% substring mutation on dosages and numbers? Unlike generative LLMs that synthesize JSON text token-by-token from scratch, GLiNER-RelEx predicts (startWord, endWord) boundary indices mapped directly to character offsets (startChar, endChar) in the input string and returns source_text[startChar:endChar]. Because the model never re-types characters, it cannot invent digits or mutate units inside extracted spans.

Q5: Can I run this inside Apache Kafka, Flink, Spark, or Spring/Quarkus Java services without Python? Yes! examples/java/ provides a standalone Java 17+ GlinerOnnxExtractor requiring only 3 Maven dependencies (com.microsoft.onnxruntime:onnxruntime, ai.djl.huggingface:tokenizers, and Jackson). Sharing one thread-safe OrtSession across 3 stream threads achieved 148,085 docs/hr in 935 MB total JVM RSS—5.8x higher throughput and 2.8x lower memory than a Python PyTorch HTTP sidecar.

Q6: Why do single-pass INT8 encoder graphs run so efficiently on Edge NPUs? Autoregressive LLM decoding (batch=1) is bottlenecked by system RAM bandwidth because weights and growing KV-caches must be fetched for every output token. A single-pass encoder processes all prompt labels and document tokens simultaneously in one dense [seq_len, hidden_dim] INT8 MatMul pass with zero KV-cache, keeping NPU systolic matrix engines (AMD Ryzen AI XDNA, Intel Core Ultra AI Boost, Apple ANE, Qualcomm Hexagon) compute-saturated.

Q7: Which languages are supported, and is this a certified medical device? This release is an Apache-2.0 Research Prototype & Proof of Concept trained and benchmarked primarily on English (en) text; cross-lingual behavior on German or other languages is experimental and unvalidated. It comes with no warranties and is not a certified medical device or standalone clinical diagnostic tool.

Q8: Can this single-pass approach be fine-tuned for other enterprise domains like finance or customs? Yes. Because GLiNER-RelEx encodes entity and relation labels dynamically in the prompt (<<ENT>> … <<REL>> …), fine-tuning on domain-specific schemas—such as financial covenants, maritime UCP 600 trade documents, customs HS codes, or industrial maintenance logs—follows the exact same compact INT8 CPU/NPU deployment blueprint.


Acknowledgments, Scientific References & Further Reading

Open-Weights Model Repository & Live Interactive Demo Space

Upstream Open-Weights & Standards Acknowledgments

We extend our sincere gratitude to Knowledgator (@knowledgator on Hugging Face) for creating and releasing the outstanding open-weights GLiNER-RelEx joint entity-and-relation extraction checkpoints, Urchade Zaratiana et al. (urchade/GLiNER) for the foundational GLiNER architecture, Microsoft Research for DeBERTa-v3, and the Schema.org & W3C JSON-LD communities.

Peer-Reviewed Literature