In Neuro-Symbolic AI architecture (Kautz, AAAI 2020; Kambhampati et al., ICML 2024), fast System 1 neural perception must translate messy human prose into typed entities, exact character offsets, and semantic relations before System 2 symbolic engines evaluate domain ontologies and compliance rules. Today, iunera—creators of YPipe — The Sovereign Local-First AI Platform—is releasing ypipe/gliner-neuro-symbolic-extractor under the Apache-2.0 license alongside a free live interactive CPU demo Space (ypipe/gliner-neuro-symbolic-space). Built gratefully upon Knowledgator's GLiNER-RelEx and GLiNER (DeBERTa-v3), our 283 MB (base) and 655 MB (large) INT8 ONNX research prototypes extract Schema.org W3C JSON-LD graphs and structured clinical values in a single 22.0 ms–54.7 ms laptop CPU forward pass—delivering 0% substring mutation by construction and 148,085 docs/hr in-process inside pure Java 17+ Big-Data streams.
Open-Weights Resources & Research Prototype Notice
- Model Weights, ONNX Graphs & Java/Python SDKs (Apache-2.0): huggingface.co/ypipe/gliner-neuro-symbolic-extractor
- Live Free Interactive CPU Demo Space (Zero Install): huggingface.co/spaces/ypipe/gliner-neuro-symbolic-space
- Research Prototype Framing (No Warranties): Published strictly as an open-weights Research Prototype & Proof of Concept for neuro-symbolic AI research, architectural evaluation, and prototyping (not a certified medical device). Validated primarily on English (en) text; behavior on German and other languages is experimental.

Table of Contents
- Stop Burning Autoregressive LLM Compute to Parse Structured Entities & Relations
- Inside ypipe/gliner-neuro-symbolic-extractor: Training Scope, Domain Transferability & INT8 ONNX Engineering
- Four Reproducible Empirical Studies (Schema.org F1, Dosage Fidelity, Edge NPU Scaling & Pure Java 17+)
- Global Healthcare Vision: Private, Low-Compute Patient-to-Study Matching Worldwide
- In Plain English for Business Leaders: The "Optical Highlighter + Certified Rule Auditor" Analogy
- Try It Live (Hugging Face Space), ~50-Line DIY Python Loop & Enterprise Scale With YPipe
- Frequently Asked Questions (FAQ)
- Acknowledgments, Scientific References & Further Reading
1. Stop Burning Autoregressive LLM Compute to Parse Structured Entities & Relations
Across enterprise data engineering, healthcare informatics, and RAG pipelines, one architectural anti-pattern dominates modern AI budgets: routing every incoming paragraph, supplier quote, Kafka event, or electronic health record (EHR) note through an autoregressive Large Language Model in "JSON mode" just to extract named entities and relationships.
Peer-reviewed computer science and empirical benchmarks expose three structural flaws in using generative next-token predictors as high-volume schema extractors:
- Sequential O(N_out) Decode Overhead vs. Single-Pass O(1) Perception: To emit a 130-token JSON object describing six Schema.org entities and three properties, an autoregressive decoder must execute 130 sequential forward passes, re-streaming hundreds of megabytes or gigabytes of model weights and growing KV-cache tensors out of RAM on every single token. By contrast, a bidirectional span-and-relation encoder (Zaratiana et al., NAACL 2024, arXiv:2311.08526) scores all entity boundaries and pairwise relations simultaneously in one parallel forward pass.
- Generative Token Re-Synthesis vs. Verbatim Pointer Slicing (Kalai & Vempala, ACM STOC 2024, arXiv:2311.14648): When a generative LLM writes {"dosage": "0.125 mg"}, it has no structural character-index pointer back to the input string—it synthesizes every digit and unit token from probability distributions. Compact generative models can silently re-case strings, invent label meta-tokens, or drop units, and even large models return detached strings without guaranteed [startChar..endChar] character provenance for UI highlighting or regulatory audit trails.
- Long-Context Dilution & The Cloud Data Trap (Liu et al., TACL 2024, arXiv:2307.03172): Stuffing multi-page documents into external cloud APIs degrades middle-context recall while exposing confidential patient files, HR records, and pricing contracts to third-party servers and opaque safety-classifier retention policies.
Inside YPipe — The Sovereign Local-First AI Platform, the neuro-symbolic YPipe Manifold Engine solves this by separating System 1 neural perception (compact local encoders that convert unstructured text into offset-locked symbols on CPU/NPU) from System 2 deterministic reasoning (ontologies, preconditions, and formal verification). Today we are releasing an open-weights standalone research prototype of this fast neural perception pattern so every Python and Java engineer can test it firsthand.
2. Inside ypipe/gliner-neuro-symbolic-extractor: Training Scope, Domain Transferability & INT8 ONNX Engineering
We gratefully built ypipe/gliner-neuro-symbolic-extractor upon Knowledgator’s pioneering open-weights GLiNER-RelEx models (knowledgator/gliner-relex-base-v1.0 and knowledgator/gliner-relex-large-v1.0), which extend Urchade Zaratiana et al.’s generalist bidirectional NER architecture (GLiNER) and Microsoft Research’s disentangled-attention DeBERTa-v3 backbones (He, Gao, & Chen, ICLR 2023, arXiv:2111.09543) with a joint pairwise relation-scoring head.
What This Research Prototype Was Fine-Tuned On—And Why It Transfers to Other Verticals
To evaluate whether a compact 225M–459M encoder (283 MB–655 MB INT8) can serve as a universal schematic front-end for neuro-symbolic pipelines, we fine-tuned both checkpoints across four complementary structural distributions:
- Official Schema.org Entity Classes & Relational Properties: Broad coverage across people (Person), organizations (Organization), products (Product), software (SoftwareApplication), locations (Place), events (Event), monetary values (MonetaryAmount), and relational predicates (founder, worksFor, location, manufacturer, price, author, memberOf).
- Quantitative Units, Dosages & Clinical Patient-File Relations: Medications, numeric strengths/dosages, administration frequencies, clinical conditions/indications, laboratory biomarkers (eGFR, INR, HbA1c, potassium), physical/clinical units (mg, mcg, mL/min, mmol/L, EUR, USD), and directional relations (treats, has_dosage, has_frequency, has_value, contraindicated_with).
- Software, Technical & Operational Parameters: System components, operational states, and attribute-value pairs common in DevOps and industrial logs.
- Hard-Negative & Interrogative Calibration: Uninstantiated questions (e.g., “Which medication treats hypertension?”) so the extractor distinguishes asserted factual instances from generic query nouns.
Because GLiNER-RelEx embeds the caller’s target schema dynamically inside the input prompt (<<ENT>> … <<SEP>> <<REL>> … <<SEP>>) alongside the text in one bidirectional pass, this approach transfers cleanly to virtually any structured industry vertical—including banking KYC & UBO graphs, maritime bills of lading (ICC UCP 600), customs HS tariff screening, industrial IoT telemetry, and legal contract auditing—without deploying 70B+ GPU clusters.
Architect’s Field Note: Three First-Hand Engineering Scars From Exporting & Quantizing GLiNER-RelEx to INT8 ONNX

First-Hand Systems Engineering E-E-A-T (What Broke During ONNX Export & How We Fixed It)
Getting joint entity-and-relation extraction running deterministically in pure C++/Java ONNX Runtime at INT8 precision required solving three subtle low-level bugs that standard export tutorials miss:
- Mixed-Precision Quantization (Keeping the Bi-LSTM in FP32): Naive full-graph dynamic INT8 quantization (quantize_dynamic across all operator types) quantized the recurrent gates inside GLiNER's 1-layer bidirectional LSTM word-projection layer—causing subtle trailing-word boundary drift on multi-word entities ("Munich," vs. "Munich, Germany"). Restricting INT8 quantization strictly to MatMul, Gemm, and Gather while preserving the LSTM operator in exact FP32 slashed base size by 67% (861 MB → 283 MB) and accelerated CPU inference by 3.2x with zero boundary degradation.
- Eliminating Frozen Python .item() Trace Shapes: During torch.onnx.export, upstream Python .item() calls inside target-embedding selection and entity-pair construction froze trace-time tensor shapes—causing ScatterND runtime crashes whenever a sentence contained >= 3 entities in large or returning 0 relations in base. Replacing scalar .item() branches with dynamic nonzero-gather ops and encoding rel_idx onto a deterministic rectangular span grid (start_word * 12 + width) unlocked arbitrary entity/relation counts in one static ONNX graph.
- Zero-Pickle model.safetensors & Clean models/<variant>/onnx/ Subdirectory Packaging: Both models/base/ (~225M, token_level BIO) and models/large/ (~459M, markerV0 span-level) ship with zero-pickle model.safetensors plus isolated onnx/model.onnx and onnx/model_quantized.onnx files—allowing lightweight Java and Python containers to pull only the 283 MB INT8 ONNX artifact and tokenizer JSONs without downloading PyTorch checkpoints.
3. Four Reproducible Empirical Studies (With Preliminary Scope & Larger-LLM Nuance)
All benchmark harnesses and raw JSON outputs are published openly in benchmarks/ and were executed on a single standard business laptop CPU (AMD Ryzen 5 PRO 7540U Zen 4, 6 Cores / 12 Threads, Linux x86_64, zero GPU) against native C++ llama.cpp (llama-server build b11541 with Zen 4 AVX-512, structured JSON mode, warm session, –reasoning-budget 0).
Study 1 (Preliminary Assessment, N = 8): Schema.org JSON-LD Extraction vs. Upstream & llama.cpp
In our first preliminary micro-benchmark (N = 8 multi-clause enterprise, software, industrial, and clinical passages in benchmarks/schema_org_benchmark.py), we measured single-pass extraction of official Schema.org classes and properties serialized directly into valid W3C JSON-LD 1.1:

| Model / Engine (N=8) | Runtime | Latency | 10K Docs | Speedup | Ent F1 | Rel F1 | Mutations |
|---|---|---|---|---|---|---|---|
| ypipe/gliner-neuro-symbolic (base INT8) | ONNX Runtime CPU | 22.0 ms | 3.7 min ($0) | 398x faster | 98.3% | 74.4% | 0 (substring) |
| ypipe/gliner-neuro-symbolic (large INT8) | ONNX Runtime CPU | 54.7 ms | 9.1 min ($0) | 160x faster | 98.2% | 82.1% | 0 (substring) |
| knowledgator/gliner-relex-base-v1.0 | PyTorch FP32 CPU | 118.9 ms | 19.8 min | 74x faster | 96.7% | 74.5% | 0 (substring) |
| knowledgator/gliner-relex-multi-v1.0 | PyTorch FP32 CPU | 117.7 ms | 19.6 min | 74x faster | 96.7% | 75.0% | 0 (substring) |
| Qwen3.5-0.8B (IQ4_XS GGUF, JSON mode) | Native C++ llama.cpp | 3,209.7 ms | 8.9 hrs | 2.7x | 77.8% | 41.0% | 2 invented |
| Qwen3.5-2B (IQ4_XS GGUF, JSON mode) | Native C++ llama.cpp | 8,745.7 ms | 24.3 hrs | 1.0x (baseline) | 84.6% | 77.8% | 0 |
| Larger Local / Frontier Cloud LLMs (7B–70B+ JSON Mode) | GPU / External Cloud | ~1.5–15+ s | ~4–40+ hrs + GPU/API cost | GPU / Cloud-bound | ~96–99% (Clearly higher) | ~85–95% (Clearly higher) | Rare (no char offsets) |
Study 2 (Preliminary Assessment, N = 20): Clinical Dosage & Numeric Fidelity Stress Test—And Fair Larger-LLM Comparison
To test substring fidelity on safety-critical numbers, we ran a preliminary 20-sentence stress assessment (benchmarks/dosage_and_numeric_mutation_benchmark.py) across clinical decimals (0.125 mg, 0.08 mcg/kg/min), renal biomarkers (27.5 mL/min, 4.2 mmol/L), and financial prices (2490.50 EUR):

| Model / Engine (N=20) | Runtime | Mean Latency | Extracted Spans | Non-Verbatim Spans | Gold Numeric Recall |
|---|---|---|---|---|---|
| ypipe/gliner-neuro-symbolic (base INT8) | ONNX Runtime CPU | 21.8 ms | 84 | 0 (100% Verbatim Slice) | 91.7% |
| ypipe/gliner-neuro-symbolic (large INT8) | ONNX Runtime CPU | 69.2 ms | 79 | 0 (100% Verbatim Slice) | 86.1% |
| Qwen3.5-0.8B (IQ4_XS GGUF, JSON mode) | Native C++ llama.cpp | 1,856.0 ms | 90 | 7 non-verbatim ('Dosage', 'Orally', re-cased strings) | 63.9% |
| Qwen3.5-2B (IQ4_XS GGUF, JSON mode) | Native C++ llama.cpp | 6,134.0 ms | 107 | 0 (282x slower than base INT8) | 97.2% |
- Scientific Honesty — Larger LLMs Are Clearly Better on Recall: Look closely at the progression from Qwen3.5-0.8B (63.9% numeric recall, 7 non-verbatim strings including the invented meta-word 'Dosage') up to Qwen3.5-2B (97.2% numeric recall and 0 non-verbatim strings)—and larger 7B–70B+ local GPU models and frontier cloud LLMs are clearly stronger still on zero-shot coverage, implicit relations, and complex sentence structures.
- The Exact Pareto Trade-Off: We do not claim a 283 MB encoder replaces 7B–70B+ LLMs on open-ended comprehension. Rather, reaching 97.2% recall with 2B already required 6.13 seconds per sentence (282x slower than base INT8’s 21.8 ms) and still emitted detached strings without [startChar..endChar] offset pointers. For high-throughput schematic perception, a 22 ms extractive encoder provides instant, offset-locked local symbols at 1/300th the compute cost.
Study 3: Pure Computational Scaling (O(1) vs. O(N_out)) & Why Edge NPUs Multiply the Advantage
Scaling from a single test sentence to 100,000 or 1,000,000 enterprise documents reveals why computational complexity (O(1) encoder pass vs. O(N_out) serial decode steps) governs infrastructure architecture:

- At 1,000,000 Documents on One CPU: ypipe/gliner-neuro-symbolic (base INT8) executes exactly 1,000,000 parallel matrix passes (6.1 CPU-hours), whereas llama.cpp Qwen3.5-2B executes ~135,000,000 sequential token-decode passes (101.2 CPU-days — 398x more compute).
- Why Single-Pass INT8 Graphs Excel Even Further on Edge NPUs (AMD Ryzen AI XDNA, Intel Core Ultra AI Boost, Apple ANE, Qualcomm Hexagon): Autoregressive batch=1 LLM decoding is starved by LPDDR5 memory bandwidth because weights and growing KV-caches must stream from RAM on every generated token. By contrast, our 283 MB models/base/onnx/model_quantized.onnx graph processes prompt labels and document tokens together in one dense static [seq_len, hidden_dim] INT8 MatMul pass with zero KV-cache, saturating NPU systolic SRAM tiles via ONNX Runtime execution providers (VitisAI, OpenVINO, CoreML, QNN) while leaving 100% of host CPU cores free.
Study 4: Enterprise Java & Big-Data Streaming—In-Process Pure JVM ONNX vs. Python Sidecar Sprawl
For data engineering teams running Apache Kafka, Apache Flink, Apache Spark, Apache Druid, ClickHouse, or HAPI HL7 FHIR pipelines on the JVM, deploying Python FastAPI/PyTorch NLP sidecars introduces network serialization latency, Python GIL contention, and multi-gigabyte container sprawl.
Because ypipe/gliner-neuro-symbolic-extractor ships with a standalone pure Java 17+ implementation (GlinerOnnxExtractor.java) requiring only 3 Maven dependencies (com.microsoft.onnxruntime:onnxruntime, ai.djl.huggingface:tokenizers, Jackson—zero Spring, zero Python), stream workers execute INT8 extraction directly in-memory:

| JVM / Sidecar Architecture | IPC / Network Overhead | Worker RSS | Latency (Med/p95) | Docs/Sec | Docs/Hr |
|---|---|---|---|---|---|
| In-Process Java 17+ (GlinerOnnxExtractor, 3-Thread Shared OrtSession) | Zero IPC (In-Memory) | 935 MB (entire JVM) | — (parallel stream) | 41.1 docs/s | 148,085 docs/hr (5.8x) |
| In-Process Java 17+ (GlinerOnnxExtractor, Single-Thread) | Zero IPC (In-Memory) | 935 MB (entire JVM) | 30.9 ms / 32.1 ms | 32.5 docs/s | 116,917 docs/hr (4.6x) |
| Python PyTorch + HTTP JSON Microservice Sidecar (FP32) | JSON → TCP/HTTP → GIL → JSON | 2,638 MB (+ JVM client) | 141.1 ms / 152.5 ms | 7.0 docs/s | 25,335 docs/hr (1.0x) |
4. Global Healthcare Vision: Private, Low-Compute Patient-to-Study Matching Worldwide
One of our primary motivations for fine-tuning and releasing this open-weights research prototype is advancing equitable, privacy-preserving clinical decision support—especially for hospitals and clinics in lesser-developed and resource-constrained regions:

- Strict Non-Sharing of Confidential Medical Data (Stage 1A): Patient files never leave the clinic’s local PC. In ~22–30 ms on a $300–$500 offline laptop CPU, gliner-neuro-symbolic-extractor extracts verbatim medications, dosages, conditions, and renal/metabolic biomarkers (eGFR, HbA1c) on-premise.
- Offline Distribution of Pre-Extracted Medical Studies (Stage 1B): Millions of global clinical trial protocols and dosing guidelines are extracted once into compact symbolic eligibility schemas—shippable offline on a USB drive or local disk without requiring datacenter GPUs or broadband cloud budgets.
- Deterministic Symbolic Matching & Safety Validation (Stages 2 & 3): A local symbolic solver evaluates exact biomarker thresholds (e.g., matching eGFR 28 mL/min into [20..45] renal-safe trials while deterministically blocking Metformin at eGFR < 30 mL/min) in < 10 ms, presenting physicians with exact character offsets and auditable rule proofs.
5. In Plain English for Business Leaders: The “Optical Highlighter + Certified Rule Auditor” Analogy
The 30-Second Executive Analogy: Using a generative LLM in JSON mode to process 1,000,000 invoices or patient notes is like hiring a novelist to read each page and re-type every SKU, dosage, and price digit-by-digit from memory—it takes 130+ steps per page and risks silent transcription drift. Neuro-Symbolic AI pairs an Optical Highlighter (GLiNER INT8, marking the exact characters directly on the original page in 22 ms with 0% re-typing) with a Certified Rulebook Auditor (a < 5 ms CPU symbolic engine that checks every contract, ontology, and compliance law with 100% mathematical certainty).

6. Try It Live, Build a ~50-Line Neuro-Symbolic Loop Yourself, or Scale On-Premise With YPipe
A. Test It Right Now in Your Browser — Live Hugging Face Space Walkthrough (Zero Install)
Paste any English clinical summary or business paragraph into our free Hugging Face Space—huggingface.co/spaces/ypipe/gliner-neuro-symbolic-space—and toggle between models/base (283 MB INT8 ONNX) and models/large (655 MB INT8 ONNX) directly in your browser:
- Tab 1 — Live Single-Pass Patient File Clinical Extraction (models/base/onnx/model_quantized.onnx): Highlights verbatim substring spans (type 2 diabetes, Metformin, 500, twice daily, severe renal failure, eGFR, 28 mL/min) alongside directional clinical relations (Metformin –[treats]–> type 2 diabetes, Metformin –[has_dosage]–> 500, Metformin –[contraindicated_with]–> severe renal failure) with exact [start..end] character offsets and 0% substring mutation:

- Tab 2 — Live Single-Pass Schema.org → W3C JSON-LD Extraction (models/large/onnx/model_quantized.onnx): Extracts Person (Dr. Lena Hoffmann), Organization (Helios MedTech GmbH), Product (CardioSense Pro), MonetaryAmount (2490 EUR), and Event (European Cardiology Summit) simultaneously with founder, manufacturer, and price relations, serializing them deterministically into valid https://schema.org JSON-LD:

B. Vendor-Neutral DIY Blueprint (~50 Lines of Python + Symbolic Safety Verifier)
You do not need a full platform to build a single narrow neuro-symbolic workflow! Using our Apache-2.0 examples/python/extract_onnx.py (onnxruntime + tokenizers + numpy, zero PyTorch), here is a complete System 1 Perception + System 2 Clinical Contraindication Gate in ~45 lines of Python:
import re
from pathlib import Path
from huggingface_hub import snapshot_download
from extract_onnx import GlinerOnnxExtractor # from ypipe/gliner-neuro-symbolic-extractor
# 1. Pull ONLY the 283 MB INT8 ONNX graph + tokenizer JSONs from models/base/
repo_root = Path(snapshot_download(
repo_id="ypipe/gliner-neuro-symbolic-extractor",
allow_patterns=[
"models/base/onnx/model_quantized.onnx",
"models/base/*.json",
"models/base/spm.model",
],
))
extractor = GlinerOnnxExtractor(repo_root / "models" / "base", onnx_filename="onnx/model_quantized.onnx")
# 2. System 1 Neural Perception: Single-pass offset-locked extraction (~22 ms CPU)
note = (
"Patient File Summary: 64-year-old patient with type 2 diabetes prescribed Metformin 500 mg "
"twice daily; contraindicated in severe renal failure after eGFR dropped to 28 mL/min."
)
entities, relations = extractor.extract(
text=note,
entity_labels=["medication", "dosage", "frequency", "condition", "biomarker", "measurement"],
relation_labels=["has_dosage", "has_frequency", "treats", "contraindicated_with", "has_value"],
)
# 3. System 2 Deterministic Symbolic Safety Gate (< 1 ms CPU, 0% guesswork)
RENAL_EGFR_MIN_LIMIT_ML_MIN = {"metformin": 30.0}
prescribed = {e.text.lower(): e for e in entities if e.label == "medication"}
egfr_vals = [
float(m.group(1))
for e in entities if e.label == "measurement"
for m in [re.search(r"(\d+(?:\.\d+)?)\s*mL/min", e.text, re.I)] if m
]
for drug_key, drug_span in prescribed.items():
if drug_key in RENAL_EGFR_MIN_LIMIT_ML_MIN and egfr_vals:
limit = RENAL_EGFR_MIN_LIMIT_ML_MIN[drug_key]
if min(egfr_vals) < limit:
print(
f"[SYMBOLIC SAFETY BLOCK] '{drug_span.text}' (chars {drug_span.start}..{drug_span.end}) "
f"BLOCKED: measured eGFR {min(egfr_vals)} mL/min < {limit} mL/min threshold!"
)C. Scaling Across the Enterprise With YPipe — The Sovereign Local-First AI Platform
When IT-affine business analysts and DevOps teams move from a single script to dozens of governed workflows, multi-step database agents, and enterprise-wide RAG, YPipe — The Sovereign Local-First AI Platform by iunera provides the complete neuro-symbolic stack out of the box on vanilla Windows, macOS, Linux, and on-premise servers:
- ypipe Manifold Engine: Takes the lead over local models (800M to 32B, sized automatically to your hardware via Auto-Gear in Engine Foundry) to enforce deterministic reasoning and fulfill YPipe’s mission to beat 70B+ parameter models with local engines in deterministic scopes—as proven in our 3,829-test UC Berkeley BFCL V4 study.
- KnowledgePipes: Hydrates unstructured PDFs, clinical guidelines, and SQL/OLAP tables (PostgreSQL, MariaDB, ClickHouse, Apache Druid, chDB) into sovereign local KnowledgePipe Stores, KnowledgePipe Contexts, and KnowledgePipe Assets.
- SmartPipes & Couplings: Visually chains multi-model pipelines into REST webhooks, web forms, and reusable MCP tools—while bundling Node.js, Python, and Java runtimes out of the box and tuning tool parameters in Couplings so tool-calling loops on compact models become history.
👉 Want to evaluate YPipe on your own local or on-premise data? Contact us via ypipe.com or iunera.com/contact to request an evaluation build or schedule a tailored live demo.
7. Frequently Asked Questions (FAQ)
Q1: Where can I download ypipe/gliner-neuro-symbolic-extractor and test it online? All open-weights checkpoints (models/base/ and models/large/ in zero-pickle model.safetensors, FP32 ONNX, and INT8 quantized ONNX), benchmarks, and standalone Python and Java 17+ examples are available under the Apache-2.0 license at huggingface.co/ypipe/gliner-neuro-symbolic-extractor. You can also test it immediately in your browser with zero installation on our free CPU Space at huggingface.co/spaces/ypipe/gliner-neuro-symbolic-space.
Q2: What is the difference between models/base/ and models/large/? models/base/ (~225M parameters, token_level BIO, 283 MB INT8 ONNX) is optimized for ultra-low-latency CPU/NPU perception (~22–31 ms/doc) and high-throughput JVM Big-Data streams (148,085 docs/hr). models/large/ (~459M parameters, markerV0 span-level, 655 MB INT8 ONNX) delivers higher relational recall (82.1% Schema.org Relation F1 vs. 74.4% in our preliminary micro-benchmark) at ~55–100 ms/doc on CPU.
Q3: Does this 283 MB model beat 7B–70B+ Large Language Models on every extraction task? No—and stating that clearly is essential for good engineering! Larger LLMs (7B–70B+ local GPU models and frontier cloud APIs) are clearly stronger on open-ended zero-shot recall, nuanced/implicit phrasing, and complex cross-sentence reasoning. Even in our 20-case dosage micro-benchmark, scaling llama.cpp from 0.8B (63.9% recall) to 2B (97.2% recall) outperformed our prototype’s raw recall (91.7%). Our prototype’s advantage is the pareto compute and safety trade-off: 22 ms deterministic O(1) CPU speed (280x–398x less compute), 935 MB in-process Java execution, 100% on-premise privacy, and structural text[startChar:endChar] substring slicing (0% digit re-typing risk).
Q4: How does gliner-neuro-symbolic-extractor guarantee 0% substring mutation on dosages and numbers? Unlike generative LLMs that synthesize JSON text token-by-token from scratch, GLiNER-RelEx predicts (startWord, endWord) boundary indices mapped directly to character offsets (startChar, endChar) in the input string and returns source_text[startChar:endChar]. Because the model never re-types characters, it cannot invent digits or mutate units inside extracted spans.
Q5: Can I run this inside Apache Kafka, Flink, Spark, or Spring/Quarkus Java services without Python? Yes! examples/java/ provides a standalone Java 17+ GlinerOnnxExtractor requiring only 3 Maven dependencies (com.microsoft.onnxruntime:onnxruntime, ai.djl.huggingface:tokenizers, and Jackson). Sharing one thread-safe OrtSession across 3 stream threads achieved 148,085 docs/hr in 935 MB total JVM RSS—5.8x higher throughput and 2.8x lower memory than a Python PyTorch HTTP sidecar.
Q6: Why do single-pass INT8 encoder graphs run so efficiently on Edge NPUs? Autoregressive LLM decoding (batch=1) is bottlenecked by system RAM bandwidth because weights and growing KV-caches must be fetched for every output token. A single-pass encoder processes all prompt labels and document tokens simultaneously in one dense [seq_len, hidden_dim] INT8 MatMul pass with zero KV-cache, keeping NPU systolic matrix engines (AMD Ryzen AI XDNA, Intel Core Ultra AI Boost, Apple ANE, Qualcomm Hexagon) compute-saturated.
Q7: Which languages are supported, and is this a certified medical device? This release is an Apache-2.0 Research Prototype & Proof of Concept trained and benchmarked primarily on English (en) text; cross-lingual behavior on German or other languages is experimental and unvalidated. It comes with no warranties and is not a certified medical device or standalone clinical diagnostic tool.
Q8: Can this single-pass approach be fine-tuned for other enterprise domains like finance or customs? Yes. Because GLiNER-RelEx encodes entity and relation labels dynamically in the prompt (<<ENT>> … <<REL>> …), fine-tuning on domain-specific schemas—such as financial covenants, maritime UCP 600 trade documents, customs HS codes, or industrial maintenance logs—follows the exact same compact INT8 CPU/NPU deployment blueprint.
Acknowledgments, Scientific References & Further Reading
Open-Weights Model Repository & Live Interactive Demo Space
- Hugging Face Open-Weights Model & Python/Java 17+ SDKs (Apache-2.0): https://huggingface.co/ypipe/gliner-neuro-symbolic-extractor
- Live Interactive Hugging Face Demo Space (Zero Install, CPU INT8 ONNX): https://huggingface.co/spaces/ypipe/gliner-neuro-symbolic-space
Upstream Open-Weights & Standards Acknowledgments
We extend our sincere gratitude to Knowledgator (@knowledgator on Hugging Face) for creating and releasing the outstanding open-weights GLiNER-RelEx joint entity-and-relation extraction checkpoints, Urchade Zaratiana et al. (urchade/GLiNER) for the foundational GLiNER architecture, Microsoft Research for DeBERTa-v3, and the Schema.org & W3C JSON-LD communities.
Peer-Reviewed Literature
- Zaratiana, Tomeh, Holat, & Charnois (NAACL 2024): GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer (arXiv:2311.08526)
- He, Gao, & Chen, Microsoft Research (ICLR 2023): DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing (arXiv:2111.09543)
- Kautz (AAAI 2020): The Third AI Summer: AAAI Robert S. Engelmore Memorial Lecture (DOI:10.1002/aaai.12036)
- Kambhampati et al. (ICML 2024): LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks (arXiv:2402.01817)
- Kalai & Vempala (ACM STOC 2024): Calibrated Language Models Must Hallucinate (arXiv:2311.14648)
- Liu et al. (Transactions of the ACL 2024): Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172)
Related Engineering Studies & Videos From iunera & YPipe
- Neuro-Symbolic Benchmarks: What a Neuro-Symbolic BFCL AI Test Revealed About the Anthropic IPO (800M Local Model + Manifold Engine vs. Frontier LLMs) • Watch the BFCL Benchmark Video on YouTube (@iunera)
- Healthcare & Clinical Privacy: Private On-Premise AI for Hospitals & Clinics: Bedside Voice Ward Rounds, Patient Privacy & Drug Safety in YPipe • Unleashing RAGs from Vector Search Shackles in Healthcare • Watch the Hospital AI Video
- Big Data, OLAP & Everyday CPU AI: Apache Druid MCP Server in YPipe: Engineering Real-Time AI Analytics • chDB (Embedded ClickHouse) MCP Server in YPipe • Alibaba Qwen, MariaDB & AMD Ryzen: Sovereign Enterprise AI on Everyday CPUs • Watch on YouTube
- Platform Architecture: YPipe: Sovereign Local-First AI Platform, Neuro-Symbolic Manifold Engine & MCP Overview • Subscribe to the @iunera Neuro-Symbolic AI YouTube Channel