DataHubScheduler - Details for Query 7263

Submitted Time: 2026/10/19 07:30:05
Duration: 0.6 s
Succeeded Jobs: 27022 27023 27024 27025

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 0 ms"; 2 [labelType="html" label="HashAggregate time in aggregation build: 0 ms number of output rows: 1"]; } 3 [labelType="html" label="Exchange shuffle records written: 1 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 37512.0: task 71881)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1 local bytes read: 59.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 16.0 B (0.0 B, 16.0 B, 16.0 B (stage 37512.0: task 71881)) local merged bytes read: 0.0 B number of partitions: 1 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 59.0 B (0.0 B, 59.0 B, 59.0 B (stage 37512.0: task 71881))"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: 2 ms"; 5 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 2 ms peak memory: 0.0 B number of output rows: 1 number of sort fallback tasks: 0 avg hash probes per key: 0"]; 6 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 0 ms peak memory: 256.0 KiB number of output rows: 20 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 7 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 1665.0 B number of coalesced partitions: 1"]; 8 [labelType="html" label="Exchange shuffle records written: 20 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 37510.0: task 71880)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 20 local bytes read: 1586.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 2.0 KiB (0.0 B, 2.0 KiB, 2.0 KiB (stage 37510.0: task 71880)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 1586.0 B (0.0 B, 1586.0 B, 1586.0 B (stage 37510.0: task 71880))"]; subgraph cluster9 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n78 ms (0 ms, 78 ms, 78 ms (stage 37510.0: task 71880))"; 10 [labelType="html" label="HashAggregate spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 37510.0: task 71880)) time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 37510.0: task 71880)) peak memory total (min, med, max (stageId: taskId)) 256.0 KiB (0.0 B, 256.0 KiB, 256.0 KiB (stage 37510.0: task 71880)) number of output rows: 20 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 11 [labelType="html" label="InMemoryTableScan number of output rows: 20"]; subgraph cluster12 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n4 ms (0 ms, 0 ms, 4 ms (stage 37509.0: task 71879))"; 13 [labelType="html" label=" Project "]; 14 [labelType="html" label="Scan JDBCRelation(( SELECT rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version, rs.networkId AS NetworkID, rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest, rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts, rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks, rs.latest_source, rs.launch_request_name, prof.profile_supplier_name AS direct_supplier_name, prof.profile_supplierUID AS supplier_uid, ou.ou_division, ou.ou_location FROM dbo.Datahub_SCM_Network_Load_Queue inc INNER JOIN dbo.dimscm_response_summary rs ON rs.networkId = inc.NetworkID OUTER APPLY ( SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID FROM dbo.dimsupplier_profilemerge p WHERE p.brandid = inc.brand_context_id AND p.supplier_id = inc.company_id ORDER BY p.profile_supplierUID ) prof LEFT JOIN ( SELECT d.networkId, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location FROM ( SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value FROM dbo.Datahub_SCM_Network_Load_Queue q INNER JOIN dbo.dimscm_response_summary_ou_attribute oa ON oa.networkId = q.NetworkID WHERE q.status = 'PENDING' AND oa.ou_attribute_id IN ('ou_division', 'ou_location') ) d GROUP BY d.networkId ) ou ON ou.networkId = rs.networkId WHERE inc.status = 'PENDING' AND (rs.source IS NULL OR rs.source <> 'PO_TRACING') AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED')) ) AS _t) [numPartitions=1] number of output rows: 20"]; } 2->0; 3->2; 5->3; 6->5; 7->6; 8->7; 10->8; 11->10; 13->11; 14->13; }

AdaptiveSparkPlan isFinalPlan=true

HashAggregate(keys=[], functions=[count(1)])

WholeStageCodegen (3)

Exchange SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4308930]

HashAggregate(keys=[], functions=[partial_count(1)])

HashAggregate(keys=[brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025], functions=[])

WholeStageCodegen (2)

AQEShuffleRead coalesced

Exchange hashpartitioning(brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, 10), ENSURE_REQUIREMENTS, [plan_id=4308898]

HashAggregate(keys=[brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025], functions=[])

WholeStageCodegen (1)

InMemoryTableScan [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]

Project [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, NetworkID#15205026, config_name#15205027, config_version#15205028, entity_id#15205029, submission_status#15205030, is_latest#15205031, is_editable#15205032, launch_date#15205033, expiry_date#15205034, submitted_date#15205035, create_ts#15205036, update_ts#15205037, create_date#15205038, update_date#15205039, source#15205040, recurrence_count#15205041, priority#15205042, remarks#15205043, latest_source#15205044, launch_request_name#15205045, ... 4 more fields]

Scan JDBCRelation(( SELECT rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version, rs.networkId AS NetworkID, rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest, rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts, rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks, rs.latest_source, rs.launch_request_name, prof.profile_supplier_name AS direct_supplier_name, prof.profile_supplierUID AS supplier_uid, ou.ou_division, ou.ou_location FROM dbo.Datahub_SCM_Network_Load_Queue inc INNER JOIN dbo.dimscm_response_summary rs ON rs.networkId = inc.NetworkID OUTER APPLY ( SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID FROM dbo.dimsupplier_profilemerge p WHERE p.brandid = inc.brand_context_id AND p.supplier_id = inc.company_id ORDER BY p.profile_supplierUID ) prof LEFT JOIN ( SELECT d.networkId, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division, STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location FROM ( SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value FROM dbo.Datahub_SCM_Network_Load_Queue q INNER JOIN dbo.dimscm_response_summary_ou_attribute oa ON oa.networkId = q.NetworkID WHERE q.status = 'PENDING' AND oa.ou_attribute_id IN ('ou_division', 'ou_location') ) d GROUP BY d.networkId ) ou ON ou.networkId = rs.networkId WHERE inc.status = 'PENDING' AND (rs.source IS NULL OR rs.source <> 'PO_TRACING') AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED')) ) AS _t) [numPartitions=1] [brand_context_id#15205022,company_id#15205023,launch_config_ref#15205024,revision_version#15205025,NetworkID#15205026,config_name#15205027,config_version#15205028,entity_id#15205029,submission_status#15205030,is_latest#15205031,is_editable#15205032,launch_date#15205033,expiry_date#15205034,submitted_date#15205035,create_ts#15205036,update_ts#15205037,create_date#15205038,update_date#15205039,source#15205040,recurrence_count#15205041,priority#15205042,remarks#15205043,latest_source#15205044,launch_request_name#15205045,... 4 more fields] PushedFilters: [], ReadSchema: struct<brand_context_id:string,company_id:string,launch_config_ref:string,revision_version:smalli...

WholeStageCodegen (1)

Details

== Physical Plan ==
AdaptiveSparkPlan (21)
+- == Final Plan ==
   * HashAggregate (14)
   +- ShuffleQueryStage (13), Statistics(sizeInBytes=16.0 B, rowCount=1)
      +- Exchange (12)
         +- * HashAggregate (11)
            +- * HashAggregate (10)
               +- AQEShuffleRead (9)
                  +- ShuffleQueryStage (8), Statistics(sizeInBytes=2.0 KiB, rowCount=20)
                     +- Exchange (7)
                        +- * HashAggregate (6)
                           +- TableCacheQueryStage (5), Statistics(sizeInBytes=8.3 KiB, rowCount=20)
                              +- InMemoryTableScan (1)
                                    +- InMemoryRelation (2)
                                          +- * Project (4)
                                             +- * Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1]  (3)
+- == Initial Plan ==
   HashAggregate (20)
   +- Exchange (19)
      +- HashAggregate (18)
         +- HashAggregate (17)
            +- Exchange (16)
               +- HashAggregate (15)
                  +- InMemoryTableScan (1)
                        +- InMemoryRelation (2)
                              +- * Project (4)
                                 +- * Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1]  (3)


(1) InMemoryTableScan
Output [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Arguments: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]

(2) InMemoryRelation
Arguments: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, NetworkID#15205026, config_name#15205027, config_version#15205028, entity_id#15205029, submission_status#15205030, is_latest#15205031, is_editable#15205032, launch_date#15205033, expiry_date#15205034, submitted_date#15205035, create_ts#15205036, update_ts#15205037, create_date#15205038, update_date#15205039, source#15205040, recurrence_count#15205041, priority#15205042, remarks#15205043, latest_source#15205044, launch_request_name#15205045, ... 4 more fields], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@730ae8bf,StorageLevel(disk, memory, deserialized, 1 replicas),*(1) Project [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, NetworkID#15205026, config_name#15205027, config_version#15205028, entity_id#15205029, submission_status#15205030, is_latest#15205031, is_editable#15205032, launch_date#15205033, expiry_date#15205034, submitted_date#15205035, create_ts#15205036, update_ts#15205037, create_date#15205038, update_date#15205039, source#15205040, recurrence_count#15205041, priority#15205042, remarks#15205043, latest_source#15205044, launch_request_name#15205045, ... 4 more fields]
+- *(1) Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1] [brand_context_id#15205022,company_id#15205023,launch_config_ref#15205024,revision_version#15205025,NetworkID#15205026,config_name#15205027,config_version#15205028,entity_id#15205029,submission_status#15205030,is_latest#15205031,is_editable#15205032,launch_date#15205033,expiry_date#15205034,submitted_date#15205035,create_ts#15205036,update_ts#15205037,create_date#15205038,update_date#15205039,source#15205040,recurrence_count#15205041,priority#15205042,remarks#15205043,latest_source#15205044,launch_request_name#15205045,... 4 more fields] PushedFilters: [], ReadSchema: struct<brand_context_id:string,company_id:string,launch_config_ref:string,revision_version:smalli...
,None)

(3) Scan JDBCRelation((
            SELECT
                rs.brand_context_id, rs.company_id, rs.launch_config_ref, rs.revision_version,
                rs.networkId AS NetworkID,
                rs.config_name, rs.config_version, rs.entity_id, rs.submission_status, rs.is_latest,
                rs.is_editable, rs.launch_date, rs.expiry_date, rs.submitted_date, rs.create_ts, rs.update_ts,
                rs.create_date, rs.update_date, rs.source, rs.recurrence_count, rs.priority, rs.remarks,
                rs.latest_source, rs.launch_request_name,
                prof.profile_supplier_name AS direct_supplier_name,
                prof.profile_supplierUID AS supplier_uid,
                ou.ou_division, ou.ou_location
            FROM dbo.Datahub_SCM_Network_Load_Queue inc
            INNER JOIN dbo.dimscm_response_summary rs
              ON rs.networkId = inc.NetworkID
            OUTER APPLY (
                SELECT TOP 1 p.profile_supplier_name, p.profile_supplierUID
                FROM dbo.dimsupplier_profilemerge p
                WHERE p.brandid = inc.brand_context_id
                  AND p.supplier_id = inc.company_id
                ORDER BY p.profile_supplierUID
            ) prof
            LEFT JOIN (
                SELECT
                    d.networkId,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_division' THEN d.ou_attribute_value END, ',') AS ou_division,
                    STRING_AGG(CASE WHEN d.ou_attribute_id = 'ou_location' THEN d.ou_attribute_value END, ',') AS ou_location
                FROM (
                    SELECT DISTINCT oa.networkId, oa.ou_attribute_id, oa.ou_attribute_value
                    FROM dbo.Datahub_SCM_Network_Load_Queue q
                    INNER JOIN dbo.dimscm_response_summary_ou_attribute oa
                      ON oa.networkId = q.NetworkID
                    WHERE q.status = 'PENDING'
                      AND oa.ou_attribute_id IN ('ou_division', 'ou_location')
                ) d
                GROUP BY d.networkId
            ) ou
              ON ou.networkId = rs.networkId
            WHERE inc.status = 'PENDING'
              AND (rs.source IS NULL OR rs.source <> 'PO_TRACING')
              AND (rs.submission_status IS NULL OR rs.submission_status NOT IN ('AUTO_CLOSED'))
        ) AS _t) [numPartitions=1]  [codegen id : 1]
Output [28]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, NetworkID#15205026, config_name#15205027, config_version#15205028, entity_id#15205029, submission_status#15205030, is_latest#15205031, is_editable#15205032, launch_date#15205033, expiry_date#15205034, submitted_date#15205035, create_ts#15205036, update_ts#15205037, create_date#15205038, update_date#15205039, source#15205040, recurrence_count#15205041, priority#15205042, remarks#15205043, latest_source#15205044, launch_request_name#15205045, direct_supplier_name#15205046, supplier_uid#15205047, ou_division#15205048, ou_location#15205049]
ReadSchema: struct<brand_context_id:string,company_id:string,launch_config_ref:string,revision_version:smallint,NetworkID:string,config_name:string,config_version:boolean,entity_id:string,submission_status:string,is_latest:boolean,is_editable:boolean,launch_date:string,expiry_date:string,submitted_date:string,create_ts:timestamp,update_ts:timestamp,create_date:date,update_date:date,source:string,recurrence_count:int,priority:string,remarks:string,latest_source:string,launch_request_name:string,direct_supplier_name:string,supplier_uid:string,ou_division:string,ou_location:string>

(4) Project [codegen id : 1]
Output [28]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, NetworkID#15205026, config_name#15205027, config_version#15205028, entity_id#15205029, submission_status#15205030, is_latest#15205031, is_editable#15205032, launch_date#15205033, expiry_date#15205034, submitted_date#15205035, create_ts#15205036, update_ts#15205037, create_date#15205038, update_date#15205039, source#15205040, recurrence_count#15205041, priority#15205042, remarks#15205043, latest_source#15205044, launch_request_name#15205045, CASE WHEN (trim(regexp_replace(regexp_replace(direct_supplier_name#15205046,  ,  , 1),  {2,},  , 1), None) = ) THEN null ELSE trim(regexp_replace(regexp_replace(direct_supplier_name#15205046,  ,  , 1),  {2,},  , 1), None) END AS direct_supplier_name#15205078, supplier_uid#15205047, ou_division#15205048, ou_location#15205049]
Input [28]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, NetworkID#15205026, config_name#15205027, config_version#15205028, entity_id#15205029, submission_status#15205030, is_latest#15205031, is_editable#15205032, launch_date#15205033, expiry_date#15205034, submitted_date#15205035, create_ts#15205036, update_ts#15205037, create_date#15205038, update_date#15205039, source#15205040, recurrence_count#15205041, priority#15205042, remarks#15205043, latest_source#15205044, launch_request_name#15205045, direct_supplier_name#15205046, supplier_uid#15205047, ou_division#15205048, ou_location#15205049]

(5) TableCacheQueryStage
Output [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Arguments: 0

(6) HashAggregate [codegen id : 1]
Input [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Keys [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Functions: []
Aggregate Attributes: []
Results [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]

(7) Exchange
Input [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Arguments: hashpartitioning(brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, 10), ENSURE_REQUIREMENTS, [plan_id=4308898]

(8) ShuffleQueryStage
Output [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Arguments: 1

(9) AQEShuffleRead
Input [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Arguments: coalesced

(10) HashAggregate [codegen id : 2]
Input [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Keys [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Functions: []
Aggregate Attributes: []
Results: []

(11) HashAggregate [codegen id : 2]
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#15205678L]
Results [1]: [count#15205679L]

(12) Exchange
Input [1]: [count#15205679L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4308930]

(13) ShuffleQueryStage
Output [1]: [count#15205679L]
Arguments: 2

(14) HashAggregate [codegen id : 3]
Input [1]: [count#15205679L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#15205255L]
Results [1]: [count(1)#15205255L AS count#15205256L]

(15) HashAggregate
Input [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Keys [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Functions: []
Aggregate Attributes: []
Results [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]

(16) Exchange
Input [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Arguments: hashpartitioning(brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025, 10), ENSURE_REQUIREMENTS, [plan_id=4308843]

(17) HashAggregate
Input [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Keys [4]: [brand_context_id#15205022, company_id#15205023, launch_config_ref#15205024, revision_version#15205025]
Functions: []
Aggregate Attributes: []
Results: []

(18) HashAggregate
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#15205678L]
Results [1]: [count#15205679L]

(19) Exchange
Input [1]: [count#15205679L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4308847]

(20) HashAggregate
Input [1]: [count#15205679L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#15205255L]
Results [1]: [count(1)#15205255L AS count#15205256L]

(21) AdaptiveSparkPlan
Output [1]: [count#15205256L]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.shuffle.partitions	10