3.5.1
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
DataHubScheduler
application UI
Details for Stage 38286 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
1 s
Locality Level Summary:
Process local: 1
Input Size / Records:
5.0 MiB / 2
Output Size / Records:
0.0 B / 10000
Associated Job Ids:
27653
DAG Visualization
digraph G { subgraph clusterstage_38286 { label="Stage 38286"; 164498 [labelType="html" label="JDBCRDD [164498]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164569 [labelType="html" label="SQLExecutionRDD [164569]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; subgraph cluster395919 { label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] "; 164500 [labelType="html" label="MapPartitionsRDD [164500]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster395920 { label="mapPartitionsInternal"; 164501 [labelType="html" label="MapPartitionsRDD [164501]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster395921 { label="mapPartitionsInternal"; 164502 [labelType="html" label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] [po_item_product_code#15464337,po_item_product_name#15464338,transaction_id#15464339,buyer_gln#15464340,seller_gln#15464341,pod_division#15464342,pod_delivery_date#15464343,pod_product_item_id#15464344,pod_product_entity#15464345,po_number#15464346,product_item_id#15464347,product_entity#15464348,brand_context_id#15464349,po_division#15464350,po_delivery_date#15464351,row_id#15464352L,parent_row_id#15464353,chain_seq#15464354,tier#15464355,production_input_type#15464356,trans_create_ts#15464357,trans_update_ts#15464358,outbound_create_ts#15464359,outbound_supplier_code#15464360,... 111 more fields] PushedFilters: [], ReadSchema: struct<po_item_product_code:string,po_item_product_name:string,transaction_id:string,buyer_gln:st...\n [164502] [Cached]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396046 { label="InMemoryTableScan"; 164565 [labelType="html" label="MapPartitionsRDD [164565]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164566 [labelType="html" label="MapPartitionsRDD [164566]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; 164567 [labelType="html" label="MapPartitionsRDD [164567]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396044 { label="DeserializeToObject"; 164568 [labelType="html" label="MapPartitionsRDD [164568]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } subgraph cluster396047 { label="mapPartitions"; 164570 [labelType="html" label="MapPartitionsRDD [164570]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:182"]; } } 164498->164500; 164500->164501; 164501->164502; 164502->164565; 164565->164566; 164566->164567; 164567->164568; 164568->164569; 164569->164570; }
164502
Event Timeline
Enable zooming
Tasks: 1. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time