Complex query run over 4.5T of CloudTrail data-- jq + xargs (for parallelism) banged it out in under 4hrs #CommandLineSkillz
2
12
650K or so
Uncompressed files on local disk. -P 64 on a 72 core system.
2
Replying to @codeslack
Oh yes, the disk situation is definitely a bottleneck. I could see the load and I/O wait shooting up.

Apr 16, 2021 · 1:06 AM UTC