3.5.1
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
DataHubScheduler
application UI
Details for Stage 38321 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
1.0 s
Locality Level Summary:
Process local: 1
Input Size / Records:
4.8 MiB / 2
Output Size / Records:
0.0 B / 10000
Associated Job Ids:
27681
DAG Visualization
digraph G { subgraph clusterstage_38321 { label="Stage 38321"; 164678 [labelType="html" label="JDBCRDD [164678]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164694 [labelType="html" label="SQLExecutionRDD [164694]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; subgraph cluster396335 { label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 50001 AND 60000) AS page) [numPartitions=1] "; 164679 [labelType="html" label="MapPartitionsRDD [164679]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396336 { label="mapPartitionsInternal"; 164680 [labelType="html" label="MapPartitionsRDD [164680]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396337 { label="mapPartitionsInternal"; 164681 [labelType="html" label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 50001 AND 60000) AS page) [numPartitions=1] [po_item_product_code#15500948,po_item_product_name#15500949,transaction_id#15500950,buyer_gln#15500951,seller_gln#15500952,pod_division#15500953,pod_delivery_date#15500954,pod_product_item_id#15500955,pod_product_entity#15500956,po_number#15500957,product_item_id#15500958,product_entity#15500959,brand_context_id#15500960,po_division#15500961,po_delivery_date#15500962,row_id#15500963L,parent_row_id#15500964,chain_seq#15500965,tier#15500966,production_input_type#15500967,trans_create_ts#15500968,trans_update_ts#15500969,outbound_create_ts#15500970,outbound_supplier_code#15500971,... 111 more fields] PushedFilters: [], ReadSchema: struct<po_item_product_code:string,po_item_product_name:string,transaction_id:string,buyer_gln:st...\n [164681] [Cached]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396366 { label="InMemoryTableScan"; 164690 [labelType="html" label="MapPartitionsRDD [164690]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164691 [labelType="html" label="MapPartitionsRDD [164691]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164692 [labelType="html" label="MapPartitionsRDD [164692]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396364 { label="DeserializeToObject"; 164693 [labelType="html" label="MapPartitionsRDD [164693]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396367 { label="mapPartitions"; 164695 [labelType="html" label="MapPartitionsRDD [164695]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } } 164678->164679; 164679->164680; 164680->164681; 164681->164690; 164690->164691; 164691->164692; 164692->164693; 164693->164694; 164694->164695; }
164681
Event Timeline
Enable zooming
Tasks: 1. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time