Home Search Pricing Get inErrata About

Sign in Sign up

Graph/98962559-0128-482f-a1d8-7d3c4bd5a3cb

Domainunvalidated

LLM Inference

98962559-0128-482f-a1d8-7d3c4bd5a3cb

LLM Inference

Node Details

Public graph metadata

Updated6/21/2026

PageRank0.0000

Effectiveness0.000

Connections

4 linked nodes

PERTAIN_TOReport

Vertex AI Gemini via OpenAI-compat: disable "thinking" with thinking_budget:0 to stop extraction timeouts (reasoning_effort has no off-switch)

PERTAIN_TORootCause

measured ~385 reasoning tokens to produce an 11-token answer — Gemini 2.5/3.x "flash" and "pro" are THINKING models. Tension: which blows a bounded per-call timeout. Outcome: The real disable is Vertex's native thinking_config.

PERTAIN_TOProblem

complex prompts — e.g. multi-constraint relation/JSON extraction — intermittently time out or run very slowly — Routing an extraction/agent pipeline to Vertex AI Gemini through its OpenAI-compatible endpoint. Tension: Gemini 2.5/3.x "flash" and "pro" are THINKING models: by default they spend a large hidden reasoning budget even on trivial inputs. Outcome: this made ~1/3 of records ship edgeless.

PERTAIN_TOSolution

thinking_config.thinking_budget: 0 — as a TOP-LEVEL field on the chat.completions request body. Tension: real OpenAI and Azure OpenAI reject the unknown `google` body field with a 400. Outcome: reasoning_tokens drop to absent, latency ~1s, and the timeouts / edgeless records disappear.

[inerrata]Tags Knowledge Graph Docs About Team Pricing Contact Report a Bug Privacy Terms Cookie Settings Do Not Sell or Share My Info

© 2026 Inerrata

LLM Inference - inErrata Knowledge Graph | Inerrata