Skip to content

Document Indexing: From Gold Layer to Knowledge Graph

What actually happens when a document is indexed into IndentiaDB? This interactive visualization follows one real-world example — a Dutch parliamentary document (https://id.indentia.ai/parl/document/00000000-0000-4000-a000-000000000000/tweedekamer/kv-tk-2019z23964) — from the moment its curated gold record arrives, to the triples, relations, embeddings and projections it becomes.

Document indexing — from gold record to graph
Click the image to open the full interactive visualization (2D + 3D, with clickable triples and a complete glossary).

Open the interactive visualization :octicons-link-external-16:

What the visualization covers

  • Three explanation levels — a high-level conceptual summary, a translation for readers who know Elasticsearch, and a detailed walkthrough of every phase with component references.
  • The pipeline — gold topic → gold-bridge worker (one anchor slice + one chunk slice per chunk, chained via relatedTo) → /context/ingest → batch embedding → N-Triples → one atomic SPARQL INSERT DATA into the named graph → derived Elasticsearch projection.
  • The graph in 2D and 3D — the document anchor slice, the chunk-slice chain (ctx:relatedTo), the provenance run node (prov:wasInformedBy), the RDF-star ACL annotation, the vector space (HNSW) and the ES projection — all enclosed by the named graph as the tenant/ACL boundary.
  • The exact triples per slice, as generated by the ingest handler — click any node in the 2D graph.
  • Elasticsearch comparison — indices-with-fields vs. triples-around-a-subject, and how the Elasticsearch-compatible API translates between them (codec, optimistic concurrency, projection).
  • A full glossary — every term on the page (medallion, triple, IRI, named graph, RDF-star, bitemporality, BM25, HNSW, RRF, OCC, …) explained inline.

The core idea

Storage is triples in a named graph; cohesion is edges plus provenance; everything else — search indexes, vectors, projections — is a rebuildable derivative. If you drop the search index, you lose nothing: you rebuild it from the graph.

Every slice carries its own evidence and its own lock: provenance (prov:wasInformedBy → the pipeline run → all the way back to the raw bronze file), confidence and authority scores, bitemporal validity, and an RDF-star ACL that sits on the triple itself — see Triple-Level ACL.