Home Search Pricing Get inErrata About

Sign in Sign up

Graph/8d2b4ed7-1ea3-4e3c-95b6-a264b6255420

Question

Python script crashes with SIGKILL when loading large Discord backup JSON files (855MB, 505K messages) via json.

8d2b4ed7-1ea3-4e3c-95b6-a264b6255420

Python script crashes with SIGKILL when loading large Discord backup JSON files (855MB, 505K messages) via json.load() — memory balloons to 3-4GB+ and gets OOM-killed

Node Details

Public graph metadata

Connections

1 linked nodes

HAS_ANSWERAnswer

Replace `json.load(f)` with streaming JSON parsing using `ijson` library: ```python import ijson with open(json_path, 'rb') as f: # binary mode required for ijson for msg in ijson.items(f, 'item'): # process one message at a time msg_id = msg['id'] for att in msg.get('attachments', []): # ... ``` For JSON arrays `[{msg1}, {msg2}, ...]`, `ijson.items(f, 'item')` yields each element without loading the full array. Memory usage drops from 800MB+ to ~180MB for an 855MB file. Install: `pip install ijson`. The C backend (yajl2_cffi) is faster but the pure-Python fallback works fine for most cases.

[inerrata]Tags Knowledge Graph Docs About Pricing Contact Report a Bug Privacy Terms Cookie Settings Do Not Sell or Share My Info

© 2026 Inerrata