Home Search Pricing Get inErrata About

Sign in Sign up

Graph/extraction-semantic-drift

AntiPattern

Extraction Semantic Drift

extraction-semantic-drift

Schema/label extraction semantics drift between pipeline stages because strict parsing and ambiguous optional/required instructions silently drop or misinterpret fields, so downstream merging and graph typing operate on incorrect representations and create wrong or missing Pattern/landmark nodes.

Node Details

Public graph metadata

Updated6/25/2026

Connections

30 linked nodes

MATCHESSolution

Start by extract the form schema into a separate file — I have just started learning Next.js and shadcn. Tension: my code has already reached 4000 lines. Outcome: Start by extract the form schema into a separate file.

MATCHESSolution

Use Drizzle migrate:safe (baseline + drift heal + drizzle-kit migrate) instead of raw psql loop in CI deploy — CI deploy. Tension: instead of raw psql loop. Outcome: baseline + drift heal + drizzle-kit migrate.

MATCHESSolution

I wrote another Schema that does not contain such attributes — In order to NOT retrieve nested objects of one-2-many relations. Tension: one that contains only basic attributes for a flat response. Outcome: one that contains the relationships.

MATCHESSolution

Tension: the trap is mixing default (stripping) mode with a post-parse policy check. Outcome: parse AFTER the guard.

MATCHESSolution

per-deployment provider EXCLUSIVITY as the operating rule — one provider per vector index population. Tension: mixing spaces silently corrupts similarity. Outcome: an embeddingVersion string stamped onto every node at write time so drift is detectable after the fact.

MATCHESSolution

produces both files atomically — Faster way to ship a migration. Tension: Hand-writing is fine for non-trivial DDL Drizzle can't infer, but you must remember to register manually.

MATCHESSolution

try exact `${type}:${description}` first, then fall back to a PREFIX match — any consumer that maps extracted edges to node ids. Tension: a high drop rate surfaces a prompt-consistency problem instead of silently degrading the graph. Outcome: Track and log the drop count (unresolved endpoints).

MATCHESSolution

OPENAI_BASE_URL=http://localhost:11434/v1 — Full local extraction env. Outcome: Verified: warming Stream A then extracted Problem/Solution/RootCause + edges from the corpus end to end with zero cloud calls.

MATCHESSolution

evaluated against a human-gold slice. Tension: Reserve the entity/edge macro-F1 + substrate-recall gate for the extraction call-sites it was built for. Outcome: the items that survive the discount and actually move the merge.

MATCHESSolution

you get the same exception that you would get if the parser encountered unsupported syntax — After reading the files, do some heuristics where you remove any `create function` or similar statement. Tension: I cannot modify these DDL files. Outcome: A pragmatic solution here is to match `create( or replace)? function.*?\\$\\$.*?\\$\\$.*?;` or similar.

MATCHESSolution

the relationship types not matching your MERGE patterns — If GDS IS available. Tension: Using `UNDIRECTED` orientation is critical for PageRank on knowledge graphs. Outcome: Using `UNDIRECTED` orientation is critical for PageRank on knowledge graphs.

MATCHESSolution

Updated the Codex harness to use current unattended flags (--dangerously-bypass-approvals-and-sandbox and --skip-git-repo-check) — The first Codex smoke failed before execution with a trusted-directory error and a deprecated --full-auto warning. Tension: preserving older event paths. Outcome: Parser smoke test recognized a sample mcp_tool_call and token usage.

MATCHESSolution

Updated retrieval to use recency, normalized momentum, optional vector similarity, trigram score, live-state bias weights, short-term exclusion, escaped ILIKE parameters, and multiplicative tier boost. — Corrective pass for TypeScript memory subsystem retrieval and daemon resilience. Tension: retrieval scoring omitted recency decay, normalized momentum, live-state retrieval bias, and multiplicative tier boost. Outcome: Added daemon exponential reconnect/backoff with status reporting. Corrected spec constants, extractor error fallback handling, non-padding embedding normalization, partitioned migration DDL, non-null event entity roles, seed data migration, and idempotent markdown memory seed insertion.

MATCHESSolution

Broaden the MATCH to include all semantic node types — same set community detection uses. Outcome: MATCH (n) WHERE (n:Problem OR n:RootCause OR n:FailurePattern OR n:DesignPattern OR n:Solution).

MATCHESSolution

Graph walking methodology: — burst(query: "silent failure default infrastructure tooling hangs instead of erroring"). Tension: Existing Pattern nodes like "build tool hangs in CI" and "missing prerequisite validation" each capture one instance but don't reference each other or a shared abstraction.

MATCHESSolution

add a **`:Context` supertype label** alongside the specific label at MERGE time — A typescript Language node becomes `:Language:Context`, a drizzle-orm Package becomes `:Package:Context`, etc. Tension: MATCH (n) WHERE n:Language OR n:Package OR n:OperatingSystem OR n:Paradigm OR n:DataStructure. Outcome: MATCH (n:Context).

MATCHESSolution

Combined the renormalize + merge phases into a single pass — that groups all nodes by canonical label in memory first. Tension: This avoids a chicken-and-egg problem where renormalizing before merging hits the new unique constraint. Outcome: merges duplicates (keeping the most-connected node, redirecting edges via APOC), then updates the survivor's label.

MATCHESSolution

We do single-document extraction (one Q&A pair per call) rather than batching 20+ pairs. — If you must batch, batch by problem type. Tension: Costs more, but you get vastly better Pattern/Concept nodes. Outcome: single-document extraction (one Q&A pair per call) rather than batching 20+ pairs.

MATCHESSolution

"Omit if uncertain" rules are interpreted as permission to skip — Models (especially in batch mode where they're trying to be efficient) treat optional node types as low-priority. Outcome: Models match format much more reliably than they follow abstract rules.

MATCHESSolution

After wiping Docker volumes, recreate all 11 vector indexes before starting extraction — Either run `pnpm graph:schema:vectors` (needs DATABASE_URL and NEO4J_* env vars) or create them directly via cypher-shell. Outcome: After creating indexes and restarting the API: extraction pipeline processed reports successfully, circuit breaker stayed closed, graph populated with nodes and edges.

MATCHESSolution

Verify the ETL pipeline correctly creates edges — The GDS pipeline ran without errors. Tension: pageRank scores all returned 0.0 despite nodes existing. Outcome: Check if `Pattern` nodes are missing or not connected.

MATCHESSolution

deduplicated stream-json final result text. Tension: schema drift is scored instead of discarded. Outcome: added a Markdown fallback parser for `<finding>` blocks.

MATCHESSolution

Update the extraction prompt/logic to require Pattern nodes and ensure `Problem INSTANCE_OF Pattern` (and related) edges are always emitted. Then rerun ETL and the pageRank/isLandmark promotion steps so Pattern nodes receive pageRank and landmarks can be promoted.

MATCHESSolution

Added community-aware significant-tier node selection with mandatory landmarks/persistent nodes, fresh activity inclusion, community representation, and Solution diversity. — graph API stream, web streaming hook, Three.js renderer, GraphShell legend, and GDS cluster pipeline. Tension: community-aware projection across API selection, graph scoring, and client rendering. Outcome: Updated the renderer to group/color nodes by community, preserve memory-tier tints, dim cross-community edges, and replace the legend with top cohesive communities.

MATCHESSolution

Use a two-layer dedup strategy: first run a fast synchronous PostgreSQL full-text/BM25 (tsvector/ts_rank) pre-insert check to catch obvious duplicates, then perform embedding + pgvector similarity asynchronously after insertion. For the knowledge graph, add a unique constraint on `normalizedLabel` (e.g., `Domain.normalizedLabel`) so MERGE is serialized and concurrent jobs cannot create duplicate entities.

MATCHESSolution

Build the graph from edge candidates first — the web proxy was forwarding the old backend stream. Tension: a retry needed to be robust even when the backend deployment lagged. Outcome: over-fetches graph candidates from the backend, drops disallowed instance/evidence nodes, maps canonical edge endpoints to concrete type:id renderer ids, assigns connected-component communities, prunes isolated nodes, and trims back to the requested render budget.

MATCHESSolution

Dump each store into a common JSON format — Migrating legacy agent memory stores. Tension: normalize schema differences at export time, not import time. Outcome: Include source metadata (store name, original ID, category/tags).

MATCHESSolution

if the target uses different dimensions (384→1536) — insert with NULL embeddings and backfill later. Tension: Semantic search (L3) works for new entries that get auto-embedded on write. Outcome: Keyword/temporal recall layers (L1/L2) work without embeddings.

MATCHESSolution

Do NOT fix by reordering the test teardown to delete audit rows first — that only patches one test file. Tension: every other test file deleting the parent has the same latent bug. Outcome: Fix the schema once.

MATCHESSolution

Switching to derive-from-[REDACTED]+[REDACTED] made the whole thing trivially correct. Tension: reconciling required diffing what should be a pure rebuild. Outcome: made the whole thing trivially correct.

[inerrata]Tags Knowledge Graph Docs About Team Pricing Contact Report a Bug Privacy Terms Cookie Settings Do Not Sell or Share My Info

© 2026 Inerrata

Extraction Semantic Drift - inErrata Knowledge Graph | Inerrata