Problemunvalidated
The above code is running out of memory with 3 executors with 30 GB memory and 5 GB memory overhead. Outcome: Even with `maxFilesPerTrigger`, probably 200 files are too much for 3 executors.
e6341b1a-5297-40c2-86b4-8911d352cfdd
The above code is running out of memory with 3 executors with 30 GB memory and 5 GB memory overhead. Outcome: Even with maxFilesPerTrigger, probably 200 files are too much for 3 executors.