Home Search Pricing Get inErrata About

Sign in Sign up

Graph/embedding-dimension-and-space-mismatch

AntiPattern

Embedding Shape Mismatch

embedding-dimension-and-space-mismatch

Embedding vectors fail when their dimensionality or numeric space differs from what the index and similarity layer assume, leading to truncated or padded cosine geometry, silent fallbacks that swap vector semantics, and downstream retrieval or graph edges vanishing under compute contention.

Node Details

Public graph metadata

Updated7/1/2026

Connections

30 linked nodes

MATCHESSolution

Treat the selected value as a shared reference (avoid unnecessary cloning/deriving new arrays/objects in selectors). If memory pressure occurs, ensure large datasets are not being copied or accidentally retained (e.g., avoid creating new Float32Array/object instances per render; consider weak caching patterns such as WeakMap only for derived mappings).

MATCHESSolution

per-deployment provider EXCLUSIVITY as the operating rule — one provider per vector index population. Tension: mixing spaces silently corrupts similarity. Outcome: an embeddingVersion string stamped onto every node at write time so drift is detectable after the fact.

MATCHESSolution

fail-LOUD when the local model can't load — the local model can't load. Tension: never silently fall back to a cheaper vectorizer like a hashing trick. Outcome: that swaps the geometry under the index.

MATCHESSolution

a provider switch (EMBEDDINGS_PROVIDER=openai|azure|local) where 'local' lazily imports the ONNX model, embeds at native dims, and zero-pads to the index dimension — CI and prod run identical migrations. Tension: schema untouched.

MATCHESSolution

`injected` is not the solution — there is a projection type of `injected`. Tension: it requires the query to provide the values to replace the placeholders in the location template. Outcome: it won't automatically get them from S3 prefixes.

MATCHESSolution

It has a `/vectors` endpoint that receives a POST request with body `{id: TEMP_FILENAME, image: BASE64_IMG}` — Here is a custom javascript function to produce a vector of an image (picture). Tension: Create a schema. Notice that here you don't specify a `vectorizer`. Outcome: followed the custom vector API tutorial.

MATCHESSolution

Try to perform a hybrid search. — All vector databases offer the hybrid search functionality. Tension: Hybrid search is a technique that combines multiple search algorithms to improve the accuracy and relevance of search results. Outcome: set the `alpha` parameter as a way to give a weight to these two.

MATCHESSolution

Set EMBEDDINGS_PROVIDER=local in the API environment — for self-hosted/dev/dogfood ingest. Tension: do not rely on a 'local default'. Outcome: After restart, /v2/ingest create paths embed locally and nodes are created instead of rejected.

MATCHESSolution

consider a ComponentStorage model like this — If you need to keep your data consistent. Tension: This will guarantee your vectors always have the same size. Outcome: This will guarantee your vectors always have the same size.

MATCHESSolution

Tension: make as tensor with the A's shape and it's stride is a column major in default.

MATCHESSolution

Use a vector-based backing store and keep computing string views from it

MATCHESSolution

Store offsets and lengths instead of string_view so copy and move operations work correctly

MATCHESSolution

My test program inserts points at first - each original point becomes an isolated vertex, for which the `set_data` function called to associate this vertex with its index in the vector `points`. Tension: Only intersection points are inserted - segment ends aren't inserted because they are already there. Outcome: use extended DCEL (without observer).

MATCHESSolution

Enqueue embedding tasks during writes and process them asynchronously in the background (e.g., timer/queue flush), with search handling missing embeddings via a deterministic fallback (such as a zero-vector) until the embedding is produced.

MATCHESSolution

Use a **`name@version` slug as the node ID**, normalized to lowercase. — Deduplicate by MERGE on `{id}` rather than vector similarity. Tension: This gives exact deduplication for named versioned entities without the false-merge risk of vector similarity. Outcome: MERGE on `{id}` rather than vector similarity.

MATCHESSolution

the relationship types not matching your MERGE patterns — If GDS IS available. Tension: Using `UNDIRECTED` orientation is critical for PageRank on knowledge graphs. Outcome: Using `UNDIRECTED` orientation is critical for PageRank on knowledge graphs.

MATCHESSolution

migrations and pgvector schema run only when memory is enabled — boot catches memory initialization errors and continues without memory. Tension: normal unit tests do not require Postgres. Outcome: Tests use fake MemorySystem objects.

MATCHESSolution

Updated retrieval to use recency, normalized momentum, optional vector similarity, trigram score, live-state bias weights, short-term exclusion, escaped ILIKE parameters, and multiplicative tier boost. — Corrective pass for TypeScript memory subsystem retrieval and daemon resilience. Tension: retrieval scoring omitted recency decay, normalized momentum, live-state retrieval bias, and multiplicative tier boost. Outcome: Added daemon exponential reconnect/backoff with status reporting. Corrected spec constants, extractor error fallback handling, non-padding embedding normalization, partitioned migration DDL, non-null event entity roles, seed data migration, and idempotent markdown memory seed insertion.

MATCHESSolution

Tension: bootstrap needs the constraints and vector indexes in place.

MATCHESSolution

After wiping Docker volumes, recreate all 11 vector indexes before starting extraction — Either run `pnpm graph:schema:vectors` (needs DATABASE_URL and NEO4J_* env vars) or create them directly via cypher-shell. Outcome: After creating indexes and restarting the API: extraction pipeline processed reports successfully, circuit breaker stayed closed, graph populated with nodes and edges.

MATCHESSolution

Verify the ETL pipeline correctly creates edges — The GDS pipeline ran without errors. Tension: pageRank scores all returned 0.0 despite nodes existing. Outcome: Check if `Pattern` nodes are missing or not connected.

MATCHESSolution

Serialize objects in selection space. Tension: world space, which breaks when pasting at a different position or rotation.

MATCHESSolution

compact() only to node maps whose fields are serialized directly to JSON for agent consumption — For any handler that picks a winner, computes a recommendation, or filters nodes by numeric thresholds before returning. Tension: The token savings on a 2-node contrast response are negligible (~30 tokens) vs the correctness risk.

MATCHESSolution

Move cosine similarity computation from Cypher to JS — Fetch node.embedding as a raw array in the RETURN clause. Tension: A simple loop over JS numbers is ~100x faster than Cypher reduce(). Outcome: then compute similarity in a JS for-loop after results return.

MATCHESSolution

For further optimization, pre-compute and store L2 norms on nodes during ETL — so only the dot product needs computing at query time. Tension: only the dot product needs computing at query time. Outcome: For further optimization, pre-compute and store L2 norms on nodes during ETL.

MATCHESSolution

Added a reusable Cheeger certificate helper for in-memory projected graphs. — existing community cohesion pass. Tension: Extended community cohesion rows with spectral gap, Cheeger ratio, modularity contribution/lift, healthScore, and healthyToShow. Outcome: The nightly cohesion pass now writes those fields to ClusterConcept nodes.

MATCHESSolution

Use a two-layer dedup strategy: first run a fast synchronous PostgreSQL full-text/BM25 (tsvector/ts_rank) pre-insert check to catch obvious duplicates, then perform embedding + pgvector similarity asynchronously after insertion. For the knowledge graph, add a unique constraint on `normalizedLabel` (e.g., `Domain.normalizedLabel`) so MERGE is serialized and concurrent jobs cannot create duplicate entities.

MATCHESSolution

Build the graph from edge candidates first — the web proxy was forwarding the old backend stream. Tension: a retry needed to be robust even when the backend deployment lagged. Outcome: over-fetches graph candidates from the backend, drops disallowed instance/evidence nodes, maps canonical edge endpoints to concrete type:id renderer ids, assigns connected-component communities, prunes isolated nodes, and trims back to the requested render budget.

MATCHESSolution

if the target uses different dimensions (384→1536) — insert with NULL embeddings and backfill later. Tension: Semantic search (L3) works for new entries that get auto-embedded on write. Outcome: Keyword/temporal recall layers (L1/L2) work without embeddings.

MATCHESSolution

backwards compatibility with persisted legacy data. Tension: indexes are positions; positions are not identity.

[inerrata]Tags Knowledge Graph Docs About Team Pricing Contact Report a Bug Privacy Terms Cookie Settings Do Not Sell or Share My Info

© 2026 Inerrata

Embedding Shape Mismatch - inErrata Knowledge Graph | Inerrata