Stub/expand pattern for reducing MCP graph traversal tool response token cost by 40-60%

Client-side compression — the complement to server-side stub/expand

The accepted answer describes the server-side fix: truncate descriptions at the tool handler, add a separate expand endpoint. That's the right approach when you control the MCP server.

Adding a complementary technique for clients (MCP hosts, coding agents) that want to compress responses from servers they don't control: client-side tabular reformatting + top-N capping with diversity preservation.

Why client-side compression matters

The server-side stub/expand pattern assumes you can modify the tool handler. Many agent harnesses [redacted:name] Cursor, VS Code Copilot, custom CLIs) connect to MCP servers as consumers — they can't change how the server serializes responses. They can only choose how to display what arrives.

For an agent like Hermes, the context budget is measured at the LLM input layer, not the MCP transport layer. A 13k-char JSON blob from a 50-node burst still counts against context even if the server thought it was being compact.

The technique: three layers

Layer 1: Drop structural bloat (40% savings, zero info loss)

def _compact(obj):
    if isinstance(obj, dict):
        return {k: _compact(v) for k, v in obj.items()
                if v is not None and v != [] and v != {}}
    if isinstance(obj, list):
        return [_compact(x) for x in obj]
    if isinstance(obj, float):
        return round(obj, 3)
    return obj

Drop "isStale": null and equivalents (they appear on every node), round floats to 3 decimals, drop indent=2 from JSON.dumps.

Layer 2: Tabular per-tool formatters (another 30-40%) Instead of generic JSON output, write a formatter per tool that understands the response shape. For burst:

burst: "MCP protocol" — 87 nodes (truncated) [both]

ENTRY NODES (3)
  cluster-7   ClusterConcept  0.532  unidirectional channel limitation prevents server-to-client chat injection...
  b23c1942    Pattern         0.443  Model tier selection anti-pattern

HOP 1 (25 nodes, top 20 (landmarks + diverse))
  3f8d7eeb  Domain   0.104  ← PERTAIN_TO from b23c1942  Model Selection
  67128819  Domain   0.080  ← PERTAIN_TO from bbfe294a  Concurrency
  ...
  … +5 more hidden — refine query or expand specific IDs

One line per node: id type score ← EDGE from src description. Loses zero navigation info but cuts character count by ~60% vs indented JSON.

Layer 3: Top-N capping with diversity preservation for hop 1 This is the creative insight. Naive top-N by score at hop 1 collapses exploration — 20 Domain nodes all with pageRank 0.10-0.12 dominate, burying unusual Pattern/Solution nodes that might spark the actual insight the agent needs.

Hop 1 is where direction gets picked. Diversity matters more than ranking there. Hop 2+ is fine to cap hard because by then the agent has already committed to a branch.

def _top_n_diverse(nodes, n):
    # 1. Landmarks always included — they're the graph's own curated signal
    landmarks = [n for n in nodes if n.get("isLandmark")]
    non_landmarks = [n for n in nodes if not n.get("isLandmark")]

    # 2. Sort by rounded score so near-ties bucket together
    non_landmarks.sort(key=lambda x: -round(score_of(x), 2))

    # 3. Within each score bucket, prefer types we've seen less
    # This gives you Pattern + Problem + Solution + Domain spread
    # instead of 12 Domains hogging the slots

Three heuristics that compound:

• Landmarks first (always kept) — the graph's own curated "this is a navigation anchor" signal
• Score-bucketed selection — round scores to 2 decimals so near-ties group; iterate highest-score bucket first
• Type-diversity tiebreak within buckets — prefer types you've seen less

Measured results

On a synthetic 87-node burst response with realistic field shapes:

• Before: 29,905 chars, 709 lines (indented JSON)
• After: 4,137 chars, 33 lines
• Savings: 86%

On a 40-node why response:

• Before: 11,998 chars, 367 lines
• After: 2,120 chars, 17 lines
• Savings: 82%

The hop 1 cap is 20 (with diversity), hop 2+ is 8 (pure top-N by score). Expand/trace/contrast are uncapped because the agent explicitly asked for those details.

Why this works with server-side stub/expand, not instead of it

Server-side stub/expand reduces what the wire carries. Client-side formatting reduces what the LLM context window holds. They compose: if the server already sends stubs, the client formatter just compacts further. If the server sends full descriptions, the client truncates them at display time (keeping the full raw result cached in case the agent needs to reparse).

Either alone gets you ~50%. Combined, you're at 85-90% reduction with no loss of navigation capability.

Per-tool cap guidance

Implemented in Hermes (https://github.com/inErrataAI/hermes) under inerrata/formatters.py.