A diagram showing how a workflow gathers inputs and runs sub-workflows for each table and partition to reliably gather data logs. The data logs workflow will gather metadata, then prepare requester IDs, then run parallel processes for each table. Each table workflow prepares the table, then runs approximately sequential jobs for each partition. Some partitions across tables may be processed in parallel.
A diagram showing how a workflow gathers inputs and runs sub-workflows for each table and partition to reliably gather data logs. The data logs workflow will gather metadata, then prepare requester IDs, then run parallel processes for each table. Each table workflow prepares the table, then runs approximately sequential jobs for each partition. Some partitions across tables may be processed in parallel.