3.5.1
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
DataHubScheduler
application UI
Details for Stage 43665 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
1.0 s
Locality Level Summary:
Process local: 1
Input Size / Records:
5.0 MiB / 2
Output Size / Records:
0.0 B / 10000
Associated Job Ids:
32416
DAG Visualization
digraph G { subgraph clusterstage_43665 { label="Stage 43665"; 189882 [labelType="html" label="JDBCRDD [189882]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; 189925 [labelType="html" label="SQLExecutionRDD [189925]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; subgraph cluster447680 { label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 20001 AND 30000) AS page) [numPartitions=1] "; 189883 [labelType="html" label="MapPartitionsRDD [189883]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447681 { label="mapPartitionsInternal"; 189884 [labelType="html" label="MapPartitionsRDD [189884]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447682 { label="mapPartitionsInternal"; 189885 [labelType="html" label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 20001 AND 30000) AS page) [numPartitions=1] [po_item_product_code#17207267,po_item_product_name#17207268,transaction_id#17207269,buyer_gln#17207270,seller_gln#17207271,pod_division#17207272,pod_delivery_date#17207273,pod_product_item_id#17207274,pod_product_entity#17207275,po_number#17207276,product_item_id#17207277,product_entity#17207278,brand_context_id#17207279,po_division#17207280,po_delivery_date#17207281,row_id#17207282L,parent_row_id#17207283,chain_seq#17207284,tier#17207285,production_input_type#17207286,trans_create_ts#17207287,trans_update_ts#17207288,outbound_create_ts#17207289,outbound_supplier_code#17207290,... 111 more fields] PushedFilters: [], ReadSchema: struct<po_item_product_code:string,po_item_product_name:string,transaction_id:string,buyer_gln:st...\n [189885] [Cached]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447787 { label="InMemoryTableScan"; 189921 [labelType="html" label="MapPartitionsRDD [189921]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; 189922 [labelType="html" label="MapPartitionsRDD [189922]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; 189923 [labelType="html" label="MapPartitionsRDD [189923]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447785 { label="DeserializeToObject"; 189924 [labelType="html" label="MapPartitionsRDD [189924]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447788 { label="mapPartitions"; 189926 [labelType="html" label="MapPartitionsRDD [189926]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } } 189882->189883; 189883->189884; 189884->189885; 189885->189921; 189921->189922; 189922->189923; 189923->189924; 189924->189925; 189925->189926; }
189885
Event Timeline
Enable zooming
Tasks: 1. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time