DataHubScheduler - Details for Query 4441

Submitted Time: 2026/10/18 07:30:05
Duration: 1 s
Succeeded Jobs: 20572 20573 20574 20575

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 0 ms"; 2 [labelType="html" label="HashAggregate time in aggregation build: 0 ms number of output rows: 1"]; } 3 [labelType="html" label="Exchange shuffle records written: 1 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 30042.0: task 64608)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1 local bytes read: 59.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 16.0 B (0.0 B, 16.0 B, 16.0 B (stage 30042.0: task 64608)) local merged bytes read: 0.0 B number of partitions: 1 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 59.0 B (0.0 B, 59.0 B, 59.0 B (stage 30042.0: task 64608))"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: 7 ms"; 5 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 7 ms peak memory: 0.0 B number of output rows: 1 number of sort fallback tasks: 0 avg hash probes per key: 0"]; 6 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 0 ms peak memory: 256.0 KiB number of output rows: 64 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 7 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 3.5 KiB number of coalesced partitions: 1"]; 8 [labelType="html" label="Exchange shuffle records written: 64 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 1 ms (0 ms, 1 ms, 1 ms (stage 30040.0: task 64607)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 64 local bytes read: 3.3 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 6.5 KiB (0.0 B, 6.5 KiB, 6.5 KiB (stage 30040.0: task 64607)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 3.3 KiB (0.0 B, 3.3 KiB, 3.3 KiB (stage 30040.0: task 64607))"]; subgraph cluster9 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n223 ms (0 ms, 223 ms, 223 ms (stage 30040.0: task 64607))"; 10 [labelType="html" label="HashAggregate spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 30042.0: task 64608)) time in aggregation build total (min, med, max (stageId: taskId)) 5 ms (0 ms, 5 ms, 5 ms (stage 30040.0: task 64607)) peak memory total (min, med, max (stageId: taskId)) 256.0 KiB (0.0 B, 256.0 KiB, 256.0 KiB (stage 30040.0: task 64607)) number of output rows: 64 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 11 [labelType="html" label="InMemoryTableScan number of output rows: 64"]; subgraph cluster12 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n147 ms (0 ms, 0 ms, 147 ms (stage 30039.0: task 64606))"; 13 [labelType="html" label=" Project "]; 14 [labelType="html" label="Scan JDBCRelation(( SELECT rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version, rs.networkId AS NetworkID, rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest, rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts, rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks, rs.latest_source, rs.launch_request_name, prof.profile_supplier_name AS direct_supplier_name, prof.profile_supplierUID AS supplier_uid, ou.ou_division, ou.ou_location FROM dbo.Datahub_SCM_Network_Load_Queue inc INNER JOIN dbo.dimscm_response_summary rs ON rs.networkId = inc.NetworkID OUTER APPLY ( SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID FROM dbo.dimsupplier_profilemerge p WHERE p.brandid = inc.brand_context_id AND p.supplier_id = inc.company_id ORDER BY p.profile_supplierUID ) prof LEFT JOIN ( SELECT d.networkId, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location FROM ( SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value FROM dbo.Datahub_SCM_Network_Load_Queue q INNER JOIN dbo.dimscm_response_summary_ou_attribute oa ON oa.networkId = q.NetworkID WHERE q.status = 'PENDING' AND oa.ou_attribute_id IN ('ou_division', 'ou_location') ) d GROUP BY d.networkId ) ou ON ou.networkId = rs.networkId WHERE inc.status = 'PENDING' AND (rs.source IS NULL OR rs.source <> 'PO_TRACING') AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED')) ) AS _t) [numPartitions=1] number of output rows: 64"]; } 2->0; 3->2; 5->3; 6->5; 7->6; 8->7; 10->8; 11->10; 13->11; 14->13; }

AdaptiveSparkPlan isFinalPlan=true

HashAggregate(keys=[], functions=[count(1)])

WholeStageCodegen (3)

Exchange SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4044191]

HashAggregate(keys=[], functions=[partial_count(1)])

HashAggregate(keys=[brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490], functions=[])

WholeStageCodegen (2)

AQEShuffleRead coalesced

Exchange hashpartitioning(brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, 10), ENSURE_REQUIREMENTS, [plan_id=4044159]

HashAggregate(keys=[brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490], functions=[])

WholeStageCodegen (1)

InMemoryTableScan [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]

Project [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, NetworkID#12896491, config_name#12896492, config_version#12896493, entity_id#12896494, submission_status#12896495, is_latest#12896496, is_editable#12896497, launch_date#12896498, expiry_date#12896499, submitted_date#12896500, create_ts#12896501, update_ts#12896502, create_date#12896503, update_date#12896504, source#12896505, recurrence_count#12896506, priority#12896507, remarks#12896508, latest_source#12896509, launch_request_name#12896510, ... 4 more fields]

Scan JDBCRelation(( SELECT rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version, rs.networkId AS NetworkID, rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest, rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts, rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks, rs.latest_source, rs.launch_request_name, prof.profile_supplier_name AS direct_supplier_name, prof.profile_supplierUID AS supplier_uid, ou.ou_division, ou.ou_location FROM dbo.Datahub_SCM_Network_Load_Queue inc INNER JOIN dbo.dimscm_response_summary rs ON rs.networkId = inc.NetworkID OUTER APPLY ( SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID FROM dbo.dimsupplier_profilemerge p WHERE p.brandid = inc.brand_context_id AND p.supplier_id = inc.company_id ORDER BY p.profile_supplierUID ) prof LEFT JOIN ( SELECT d.networkId, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location FROM ( SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value FROM dbo.Datahub_SCM_Network_Load_Queue q INNER JOIN dbo.dimscm_response_summary_ou_attribute oa ON oa.networkId = q.NetworkID WHERE q.status = 'PENDING' AND oa.ou_attribute_id IN ('ou_division', 'ou_location') ) d GROUP BY d.networkId ) ou ON ou.networkId = rs.networkId WHERE inc.status = 'PENDING' AND (rs.source IS NULL OR rs.source <> 'PO_TRACING') AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED')) ) AS _t) [numPartitions=1] [brand_context_id#12896487,company_id#12896488,launch_config_ref#12896489,revision_version#12896490,NetworkID#12896491,config_name#12896492,config_version#12896493,entity_id#12896494,submission_status#12896495,is_latest#12896496,is_editable#12896497,launch_date#12896498,expiry_date#12896499,submitted_date#12896500,create_ts#12896501,update_ts#12896502,create_date#12896503,update_date#12896504,source#12896505,recurrence_count#12896506,priority#12896507,remarks#12896508,latest_source#12896509,launch_request_name#12896510,... 4 more fields] PushedFilters: [], ReadSchema: struct<brand_context_id:string,company_id:string,launch_config_ref:string,revision_version:smalli...

WholeStageCodegen (1)

Details

== Physical Plan ==
AdaptiveSparkPlan (21)
+- == Final Plan ==
   * HashAggregate (14)
   +- ShuffleQueryStage (13), Statistics(sizeInBytes=16.0 B, rowCount=1)
      +- Exchange (12)
         +- * HashAggregate (11)
            +- * HashAggregate (10)
               +- AQEShuffleRead (9)
                  +- ShuffleQueryStage (8), Statistics(sizeInBytes=6.5 KiB, rowCount=64)
                     +- Exchange (7)
                        +- * HashAggregate (6)
                           +- TableCacheQueryStage (5), Statistics(sizeInBytes=25.1 KiB, rowCount=64)
                              +- InMemoryTableScan (1)
                                    +- InMemoryRelation (2)
                                          +- * Project (4)
                                             +- * Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1]  (3)
+- == Initial Plan ==
   HashAggregate (20)
   +- Exchange (19)
      +- HashAggregate (18)
         +- HashAggregate (17)
            +- Exchange (16)
               +- HashAggregate (15)
                  +- InMemoryTableScan (1)
                        +- InMemoryRelation (2)
                              +- * Project (4)
                                 +- * Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1]  (3)


(1) InMemoryTableScan
Output [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Arguments: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]

(2) InMemoryRelation
Arguments: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, NetworkID#12896491, config_name#12896492, config_version#12896493, entity_id#12896494, submission_status#12896495, is_latest#12896496, is_editable#12896497, launch_date#12896498, expiry_date#12896499, submitted_date#12896500, create_ts#12896501, update_ts#12896502, create_date#12896503, update_date#12896504, source#12896505, recurrence_count#12896506, priority#12896507, remarks#12896508, latest_source#12896509, launch_request_name#12896510, ... 4 more fields], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@730ae8bf,StorageLevel(disk, memory, deserialized, 1 replicas),*(1) Project [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, NetworkID#12896491, config_name#12896492, config_version#12896493, entity_id#12896494, submission_status#12896495, is_latest#12896496, is_editable#12896497, launch_date#12896498, expiry_date#12896499, submitted_date#12896500, create_ts#12896501, update_ts#12896502, create_date#12896503, update_date#12896504, source#12896505, recurrence_count#12896506, priority#12896507, remarks#12896508, latest_source#12896509, launch_request_name#12896510, ... 4 more fields]
+- *(1) Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1] [brand_context_id#12896487,company_id#12896488,launch_config_ref#12896489,revision_version#12896490,NetworkID#12896491,config_name#12896492,config_version#12896493,entity_id#12896494,submission_status#12896495,is_latest#12896496,is_editable#12896497,launch_date#12896498,expiry_date#12896499,submitted_date#12896500,create_ts#12896501,update_ts#12896502,create_date#12896503,update_date#12896504,source#12896505,recurrence_count#12896506,priority#12896507,remarks#12896508,latest_source#12896509,launch_request_name#12896510,... 4 more fields] PushedFilters: [], ReadSchema: struct<brand_context_id:string,company_id:string,launch_config_ref:string,revision_version:smalli...
,None)

(3) Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1]  [codegen id : 1]
Output [28]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, NetworkID#12896491, config_name#12896492, config_version#12896493, entity_id#12896494, submission_status#12896495, is_latest#12896496, is_editable#12896497, launch_date#12896498, expiry_date#12896499, submitted_date#12896500, create_ts#12896501, update_ts#12896502, create_date#12896503, update_date#12896504, source#12896505, recurrence_count#12896506, priority#12896507, remarks#12896508, latest_source#12896509, launch_request_name#12896510, direct_supplier_name#12896511, supplier_uid#12896512, ou_division#12896513, ou_location#12896514]
ReadSchema: struct<brand_context_id:string,company_id:string,launch_config_ref:string,revision_version:smallint,NetworkID:string,config_name:string,config_version:boolean,entity_id:string,submission_status:string,is_latest:boolean,is_editable:boolean,launch_date:string,expiry_date:string,submitted_date:string,create_ts:timestamp,update_ts:timestamp,create_date:date,update_date:date,source:string,recurrence_count:int,priority:string,remarks:string,latest_source:string,launch_request_name:string,direct_supplier_name:string,supplier_uid:string,ou_division:string,ou_location:string>

(4) Project [codegen id : 1]
Output [28]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, NetworkID#12896491, config_name#12896492, config_version#12896493, entity_id#12896494, submission_status#12896495, is_latest#12896496, is_editable#12896497, launch_date#12896498, expiry_date#12896499, submitted_date#12896500, create_ts#12896501, update_ts#12896502, create_date#12896503, update_date#12896504, source#12896505, recurrence_count#12896506, priority#12896507, remarks#12896508, latest_source#12896509, launch_request_name#12896510, CASE WHEN (trim(regexp_replace(regexp_replace(direct_supplier_name#12896511,  ,  , 1),  {2,},  , 1), None) = ) THEN null ELSE trim(regexp_replace(regexp_replace(direct_supplier_name#12896511,  ,  , 1),  {2,},  , 1), None) END AS direct_supplier_name#12896543, supplier_uid#12896512, ou_division#12896513, ou_location#12896514]
Input [28]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, NetworkID#12896491, config_name#12896492, config_version#12896493, entity_id#12896494, submission_status#12896495, is_latest#12896496, is_editable#12896497, launch_date#12896498, expiry_date#12896499, submitted_date#12896500, create_ts#12896501, update_ts#12896502, create_date#12896503, update_date#12896504, source#12896505, recurrence_count#12896506, priority#12896507, remarks#12896508, latest_source#12896509, launch_request_name#12896510, direct_supplier_name#12896511, supplier_uid#12896512, ou_division#12896513, ou_location#12896514]

(5) TableCacheQueryStage
Output [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Arguments: 0

(6) HashAggregate [codegen id : 1]
Input [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Keys [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Functions: []
Aggregate Attributes: []
Results [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]

(7) Exchange
Input [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Arguments: hashpartitioning(brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, 10), ENSURE_REQUIREMENTS, [plan_id=4044159]

(8) ShuffleQueryStage
Output [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Arguments: 1

(9) AQEShuffleRead
Input [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Arguments: coalesced

(10) HashAggregate [codegen id : 2]
Input [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Keys [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Functions: []
Aggregate Attributes: []
Results: []

(11) HashAggregate [codegen id : 2]
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#12897143L]
Results [1]: [count#12897144L]

(12) Exchange
Input [1]: [count#12897144L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4044191]

(13) ShuffleQueryStage
Output [1]: [count#12897144L]
Arguments: 2

(14) HashAggregate [codegen id : 3]
Input [1]: [count#12897144L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#12896720L]
Results [1]: [count(1)#12896720L AS count#12896721L]

(15) HashAggregate
Input [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Keys [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Functions: []
Aggregate Attributes: []
Results [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]

(16) Exchange
Input [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Arguments: hashpartitioning(brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490, 10), ENSURE_REQUIREMENTS, [plan_id=4044104]

(17) HashAggregate
Input [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Keys [4]: [brand_context_id#12896487, company_id#12896488, launch_config_ref#12896489, revision_version#12896490]
Functions: []
Aggregate Attributes: []
Results: []

(18) HashAggregate
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#12897143L]
Results [1]: [count#12897144L]

(19) Exchange
Input [1]: [count#12897144L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4044108]

(20) HashAggregate
Input [1]: [count#12897144L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#12896720L]
Results [1]: [count(1)#12896720L AS count#12896721L]

(21) AdaptiveSparkPlan
Output [1]: [count#12896721L]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.shuffle.partitions	10