3.5.1
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
DataHubScheduler
application UI
Details for Stage 43645 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
1 s
Locality Level Summary:
Process local: 1
Input Size / Records:
5.0 MiB / 2
Output Size / Records:
0.0 B / 10000
Associated Job Ids:
32400
DAG Visualization
digraph G { subgraph clusterstage_43645 { label="Stage 43645"; 189784 [labelType="html" label="JDBCRDD [189784]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; 189854 [labelType="html" label="SQLExecutionRDD [189854]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; subgraph cluster447477 { label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] "; 189788 [labelType="html" label="MapPartitionsRDD [189788]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447478 { label="mapPartitionsInternal"; 189789 [labelType="html" label="MapPartitionsRDD [189789]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447479 { label="mapPartitionsInternal"; 189790 [labelType="html" label="Scan JDBCRelation((SELECT * FROM dbo.fact_trans_chain_po_detail WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] [po_item_product_code#17195855,po_item_product_name#17195856,transaction_id#17195857,buyer_gln#17195858,seller_gln#17195859,pod_division#17195860,pod_delivery_date#17195861,pod_product_item_id#17195862,pod_product_entity#17195863,po_number#17195864,product_item_id#17195865,product_entity#17195866,brand_context_id#17195867,po_division#17195868,po_delivery_date#17195869,row_id#17195870L,parent_row_id#17195871,chain_seq#17195872,tier#17195873,production_input_type#17195874,trans_create_ts#17195875,trans_update_ts#17195876,outbound_create_ts#17195877,outbound_supplier_code#17195878,... 111 more fields] PushedFilters: [], ReadSchema: struct<po_item_product_code:string,po_item_product_name:string,transaction_id:string,buyer_gln:st...\n [189790] [Cached]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447597 { label="InMemoryTableScan"; 189850 [labelType="html" label="MapPartitionsRDD [189850]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; 189851 [labelType="html" label="MapPartitionsRDD [189851]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; 189852 [labelType="html" label="MapPartitionsRDD [189852]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447595 { label="DeserializeToObject"; 189853 [labelType="html" label="MapPartitionsRDD [189853]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } subgraph cluster447598 { label="mapPartitions"; 189855 [labelType="html" label="MapPartitionsRDD [189855]<br>first at /opt/app/scheduler/jobs/batch_table_copy_job.py:169"]; } } 189784->189788; 189788->189789; 189789->189790; 189790->189850; 189850->189851; 189851->189852; 189852->189853; 189853->189854; 189854->189855; }
189790
Event Timeline
Enable zooming
Tasks: 1. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time