3.5.1
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
DataHubScheduler
application UI
Details for Stage 38318 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
1 ms
Locality Level Summary:
Process local: 1
Input Size / Records:
4.8 MiB / 2
Shuffle Write Size / Records:
59.0 B / 1
Associated Job Ids:
27679
DAG Visualization
digraph G { subgraph clusterstage_38318 { label="Stage 38318"; 164678 [labelType="html" label="JDBCRDD [164678]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; subgraph cluster396335 { label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 50001 AND 60000) AS page) [numPartitions=1] "; 164679 [labelType="html" label="MapPartitionsRDD [164679]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396336 { label="mapPartitionsInternal"; 164680 [labelType="html" label="MapPartitionsRDD [164680]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396337 { label="mapPartitionsInternal"; 164681 [labelType="html" label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 50001 AND 60000) AS page) [numPartitions=1] [po_item_product_code#15500948,po_item_product_name#15500949,transaction_id#15500950,buyer_gln#15500951,seller_gln#15500952,pod_division#15500953,pod_delivery_date#15500954,pod_product_item_id#15500955,pod_product_entity#15500956,po_number#15500957,product_item_id#15500958,product_entity#15500959,brand_context_id#15500960,po_division#15500961,po_delivery_date#15500962,row_id#15500963L,parent_row_id#15500964,chain_seq#15500965,tier#15500966,production_input_type#15500967,trans_create_ts#15500968,trans_update_ts#15500969,outbound_create_ts#15500970,outbound_supplier_code#15500971,... 111 more fields] PushedFilters: [], ReadSchema: struct<po_item_product_code:string,po_item_product_name:string,transaction_id:string,buyer_gln:st...\n [164681] [Cached]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396343 { label="InMemoryTableScan"; 164682 [labelType="html" label="MapPartitionsRDD [164682]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164683 [labelType="html" label="MapPartitionsRDD [164683]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164684 [labelType="html" label="MapPartitionsRDD [164684]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396339 { label="WholeStageCodegen (1)"; 164685 [labelType="html" label="MapPartitionsRDD [164685]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396338 { label="Exchange"; 164686 [labelType="html" label="MapPartitionsRDD [164686]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } } 164678->164679; 164679->164680; 164680->164681; 164681->164682; 164682->164683; 164683->164684; 164684->164685; 164685->164686; }
164681
Event Timeline
Enable zooming
Tasks: 1. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time