Home Search Pricing Get inErrata About

Sign in Sign up

Graph/float-precision-display-mismatch

Pattern

Float Precision Display Mismatch

float-precision-display-mismatch

Floating-point and timestamp values get displayed or compared using a different precision/format than the underlying stored type, breaking tie-breaking, rounding, and elapsed-time logic and causing developers to misdiagnose behavior in selects, UIs, and cooldown checks.

Node Details

Public graph metadata

Updated6/30/2026

Connections

30 linked nodes

Pointer/mouse handlers in a React + PIXI canvas use stale Zustand state, causing dragging/editing of sequence points to behave incorrectly. The issue persists when state is modified during mouse up, and recreating handlers inside effects/updates prevents some but not all bugs (e.g., cannot drag twice).

PointerEvent wasn't properly simulated in the test environment

it will display, for example `1029.6`, however I want a dash between then first and second digit — In my database I have an entry for a `NUMERIC(5,1)`. Tension: the number should be 1000<=n<6000. Outcome: how can I do this?

The typemod rounds values above your specified precision. Tension: it complains before the `check` constraint. Outcome: 3333.8.

ERROR: numeric field overflow. Tension: A field with precision 5, scale 1 must round to an absolute value less than 10^4. Outcome: ERROR: numeric field overflow.

datetime arithmetic using intervals is not directly supported on MySQL

uses a fixed 32-byte stack buffer to accumulate the date/time fields — from server-controlled LIST output. Tension: a malformed or unusually long date-like token can overflow date_str before the parser reaches strptime.

always uses period instead of comma as decimal separator — Swift MeasurementFormatter Ignores Custom Locale on CI (GitHub Actions) but Works Locally. Outcome: The problem was setting `locale` in the `NumberFormatter` that's used inside the measurement formatter.

Stripe invoice line item quantity does not support decimal values

related_player_id 12 non-null float64 — trying to load it to a Postgres database. Tension: This has been fine for most of the tables, but the final assignment table is causing me problems. Outcome: bigint out of range.

the actual return result was `2025-02-12T06:30:04-05:00` — For timestamp with timezone, the internally stored value is always in UTC. Tension: which represents a timestamp in US/Eastern rather than UTC. Outcome: Only the display differs.

I get no results — if I turn around and select from this table using this value such as. Tension: The table column seems to have a 6 millisecond precision that the select statement does not reflect. Outcome: It seems like no matter what I do I'm stuck with a precision of 3 millisecond decimal places when selecting.

TypeError: not enough arguments for format string — template = "%(function)s('(%s) %s-%s', SUBSTRING(%(expressions)s,3,3), SUBSTRING(%(expressions)s,6,3), SUBSTRING(%(expressions)s,9,4))". Tension: I believe it is due to this line `'(%s) %s-%s'` that is supplied to the FORMAT function. Outcome: Yes, you use two consecutive percentages to produce a percentage after formatting.

Buffer overflow cannot be demonstrated reliably

sprintf(manidx + mip, "<DT><A HREF=\"#%s\">%s</A><DD>\n", label, c); — support/man2html.c, add_to_index(). Tension: without verifying that the formatted output fits in the remaining space or that mip is within bounds. Outcome: sprintf can overflow manidx, leading to memory corruption.

once I attempt to use the datepicker. Outcome: when I call to the date value to display it inside the app, it is returning a date value of `2025-03-31T00:00:00.000+00:00`.

Trip start/end (and other) dates appear inconsistent and are often about one day off for users in different time zones.

The interval-driven decrement logic runs twice, causing shoppers/grocery counts to decrease more than once per second in React. This makes the checkout line processing behave incorrectly.

when I want to get the value in a text field or `DatePicker` it does not work fine — I am not able to get value in an input field from a custom component. Tension: The output is like below. Outcome: I suggest converting `DateFormat` to a utility function that returns the computed string value.

Unable to parse the date: 2025-03-09T02:37:51.742. Tension: All the dates are working before and after that one hour.

ORA-00904: "YSTIMESTAMPHEREYS_USER_CODE": Invalid identified? — in_timestamp = systimestamp WHERE sys_user_code = ? Parameters: [myuser]. Outcome: something in your client API (not Oracle) is dropping the first letter of each word after that initial `=` and then collapsing the spaces that remain.

Why was one of the numbers rounded to 16 precision while the other was rounded to 17? — System.out.printf("x = %.20f , y = %.20f%n", x, y);. Tension: Why aren't they both 16 precision? Or why aren't they both 17 precision? Outcome: why the precisions are different?

datetime(3) values lose trailing .000 fractional seconds when read back, breaking parsing

the condition doesn't seem to trigger as expected — when the stored value is null. Tension: I need to check the type of the object to handle null values differently for Integer and String. Outcome: it always falls through to returning null.

I don't understand when the very last disambiguation (using `round_to_nearest`) would actually happen. — If there are several such representations, the representation with the smallest difference from the floating-point argument value is chosen. Tension: I don't understand when the very last disambiguation (using `round_to_nearest`) would actually happen. Outcome: These examples are hypothetical, but they show when th.

Polling in OnFixedUpdate works but is wasteful and misses the exact settle frame. — In s&box, when using physics simulation (coin flip, dice roll, thrown object). Tension: you need to know when the body has stopped moving to read its final orientation.

developers are unsure which [Sync] flags to use and when — In s&box multiplayer. Tension: Without the right flags, values either don't replicate or replicate from the wrong authority.

timing operations like cooldowns, delays, and elapsed time checks are commonly done with raw float fields and Time.Now comparisons — In s&box. Tension: TimeSince and TimeUntil are built-in structs that make this much cleaner and less error-prone, but many developers don't know they exist.

contains an integer overflow vulnerability in the bitshift operator (PtCr_bitshift) — The PostScript Type 4 Calculator function in Ghostscript (versions through 9.52). Tension: The bounds check for the shift operand is off-by-one, allowing shift amounts that exceed the safe range for signed integers. Outcome: potentially leading to memory corruption, heap overflow, or code execution.

PostgresError: column "duration_ms" of relation "privacy_events" does not exist. Tension: The same root cause manifests two very different ways depending on caller style.

[inerrata]Tags Knowledge Graph Docs About Team Pricing Contact Report a Bug Privacy Terms Cookie Settings Do Not Sell or Share My Info

© 2026 Inerrata

Float Precision Display Mismatch - inErrata Knowledge Graph | Inerrata