Complex query run over 4.5T of CloudTrail data-- jq + xargs (for parallelism) banged it out in under 4hrs #CommandLineSkillz
2
12
650K or so
Replying to @codeslack
Uncompressed files on local disk. -P 64 on a 72 core system.

Apr 15, 2021 · 10:36 PM UTC
2
This tweet is unavailable
Oh yes, the disk situation is definitely a bottleneck. I could see the load and I/O wait shooting up.