== Physical Plan ==
AdaptiveSparkPlan (7)
+- == Final Plan ==
DeserializeToObject (5)
+- TableCacheQueryStage (4), Statistics(sizeInBytes=3.2 MiB, rowCount=7.93E+3)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- * Scan JDBCRelation((SELECT * FROM dbo.dimcompany WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] (3)
+- == Initial Plan ==
DeserializeToObject (6)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- * Scan JDBCRelation((SELECT * FROM dbo.dimcompany WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] (3)
(1) InMemoryTableScan
Output [37]: [_id#8850575, address_city#8850576, address_country#8850577, address_country_code#8850578, address_country_code_alpha3#8850579, address_zip#8850580, address_lat#8850581, address_lng#8850582, company_id#8850583, company_type#8850584, contact_info_email#8850585, contact_info_phone_number#8850586, create_ts#8850587, created_by#8850588, description#8850589, name#8850590, primary_contact#8850591, profile_completeness_score_favgaclity#8850592, profile_completeness_company_score#8850593, iprofile_completeness_incomplete_fields#8850594, update_ts#8850595, updated_by#8850596, verification_status#8850597, status#8850598, address_state#8850599, address_line1#8850600, address_line2#8850601, file_name#8850602, lwg_supported#8850603, unique_code#8850604, profile_info_score#8850605, max_scored_score#8850606, contact_name#8850607, create_date#8850608, update_date#8850609, ETL_runtime#8850610, logo_url#8850612]
Arguments: [_id#8850575, address_city#8850576, address_country#8850577, address_country_code#8850578, address_country_code_alpha3#8850579, address_zip#8850580, address_lat#8850581, address_lng#8850582, company_id#8850583, company_type#8850584, contact_info_email#8850585, contact_info_phone_number#8850586, create_ts#8850587, created_by#8850588, description#8850589, name#8850590, primary_contact#8850591, profile_completeness_score_favgaclity#8850592, profile_completeness_company_score#8850593, iprofile_completeness_incomplete_fields#8850594, update_ts#8850595, updated_by#8850596, verification_status#8850597, status#8850598, ... 13 more fields]
(2) InMemoryRelation
Arguments: [_id#8850575, address_city#8850576, address_country#8850577, address_country_code#8850578, address_country_code_alpha3#8850579, address_zip#8850580, address_lat#8850581, address_lng#8850582, company_id#8850583, company_type#8850584, contact_info_email#8850585, contact_info_phone_number#8850586, create_ts#8850587, created_by#8850588, description#8850589, name#8850590, primary_contact#8850591, profile_completeness_score_favgaclity#8850592, profile_completeness_company_score#8850593, iprofile_completeness_incomplete_fields#8850594, update_ts#8850595, updated_by#8850596, verification_status#8850597, status#8850598, ... 14 more fields], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@7795a5ad,StorageLevel(disk, memory, deserialized, 1 replicas),*(1) Scan JDBCRelation((SELECT * FROM dbo.dimcompany WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] [_id#8850575,address_city#8850576,address_country#8850577,address_country_code#8850578,address_country_code_alpha3#8850579,address_zip#8850580,address_lat#8850581,address_lng#8850582,company_id#8850583,company_type#8850584,contact_info_email#8850585,contact_info_phone_number#8850586,create_ts#8850587,created_by#8850588,description#8850589,name#8850590,primary_contact#8850591,profile_completeness_score_favgaclity#8850592,profile_completeness_company_score#8850593,iprofile_completeness_incomplete_fields#8850594,update_ts#8850595,updated_by#8850596,verification_status#8850597,status#8850598,... 14 more fields] PushedFilters: [], ReadSchema: struct<_id:string,address_city:string,address_country:string,address_country_code:string,address_...
,None)
(3) Scan JDBCRelation((SELECT * FROM dbo.dimcompany WHERE bigid BETWEEN 10001 AND 20000) AS page) [numPartitions=1] [codegen id : 1]
Output [38]: [_id#8850575, address_city#8850576, address_country#8850577, address_country_code#8850578, address_country_code_alpha3#8850579, address_zip#8850580, address_lat#8850581, address_lng#8850582, company_id#8850583, company_type#8850584, contact_info_email#8850585, contact_info_phone_number#8850586, create_ts#8850587, created_by#8850588, description#8850589, name#8850590, primary_contact#8850591, profile_completeness_score_favgaclity#8850592, profile_completeness_company_score#8850593, iprofile_completeness_incomplete_fields#8850594, update_ts#8850595, updated_by#8850596, verification_status#8850597, status#8850598, address_state#8850599, address_line1#8850600, address_line2#8850601, file_name#8850602, lwg_supported#8850603, unique_code#8850604, profile_info_score#8850605, max_scored_score#8850606, contact_name#8850607, create_date#8850608, update_date#8850609, ETL_runtime#8850610, bigid#8850611L, logo_url#8850612]
ReadSchema: struct<_id:string,address_city:string,address_country:string,address_country_code:string,address_country_code_alpha3:string,address_zip:string,address_lat:double,address_lng:double,company_id:string,company_type:string,contact_info_email:string,contact_info_phone_number:string,create_ts:timestamp,created_by:string,description:string,name:string,primary_contact:string,profile_completeness_score_favgaclity:string,profile_completeness_company_score:string,iprofile_completeness_incomplete_fields:string,update_ts:timestamp,updated_by:string,verification_status:smallint,status:string,address_state:string,address_line1:string,address_line2:string,file_name:string,lwg_supported:boolean,unique_code:string,profile_info_score:double,max_scored_score:double,contact_name:string,create_date:date,update_date:date,ETL_runtime:timestamp,bigid:bigint,logo_url:string>
(4) TableCacheQueryStage
Output [37]: [_id#8850575, address_city#8850576, address_country#8850577, address_country_code#8850578, address_country_code_alpha3#8850579, address_zip#8850580, address_lat#8850581, address_lng#8850582, company_id#8850583, company_type#8850584, contact_info_email#8850585, contact_info_phone_number#8850586, create_ts#8850587, created_by#8850588, description#8850589, name#8850590, primary_contact#8850591, profile_completeness_score_favgaclity#8850592, profile_completeness_company_score#8850593, iprofile_completeness_incomplete_fields#8850594, update_ts#8850595, updated_by#8850596, verification_status#8850597, status#8850598, address_state#8850599, address_line1#8850600, address_line2#8850601, file_name#8850602, lwg_supported#8850603, unique_code#8850604, profile_info_score#8850605, max_scored_score#8850606, contact_name#8850607, create_date#8850608, update_date#8850609, ETL_runtime#8850610, logo_url#8850612]
Arguments: 0
(5) DeserializeToObject
Input [37]: [_id#8850575, address_city#8850576, address_country#8850577, address_country_code#8850578, address_country_code_alpha3#8850579, address_zip#8850580, address_lat#8850581, address_lng#8850582, company_id#8850583, company_type#8850584, contact_info_email#8850585, contact_info_phone_number#8850586, create_ts#8850587, created_by#8850588, description#8850589, name#8850590, primary_contact#8850591, profile_completeness_score_favgaclity#8850592, profile_completeness_company_score#8850593, iprofile_completeness_incomplete_fields#8850594, update_ts#8850595, updated_by#8850596, verification_status#8850597, status#8850598, address_state#8850599, address_line1#8850600, address_line2#8850601, file_name#8850602, lwg_supported#8850603, unique_code#8850604, profile_info_score#8850605, max_scored_score#8850606, contact_name#8850607, create_date#8850608, update_date#8850609, ETL_runtime#8850610, logo_url#8850612]
Arguments: createexternalrow(_id#8850575.toString, address_city#8850576.toString, address_country#8850577.toString, address_country_code#8850578.toString, address_country_code_alpha3#8850579.toString, address_zip#8850580.toString, staticinvoke(class java.lang.Double, ObjectType(class java.lang.Double), valueOf, address_lat#8850581, true, false, true), staticinvoke(class java.lang.Double, ObjectType(class java.lang.Double), valueOf, address_lng#8850582, true, false, true), company_id#8850583.toString, company_type#8850584.toString, contact_info_email#8850585.toString, contact_info_phone_number#8850586.toString, staticinvoke(class org.apache.spark.sql.catalyst.util.DateTimeUtils$, ObjectType(class java.sql.Timestamp), toJavaTimestamp, create_ts#8850587, true, false, true), created_by#8850588.toString, description#8850589.toString, name#8850590.toString, primary_contact#8850591.toString, profile_completeness_score_favgaclity#8850592.toString, profile_completeness_company_score#8850593.toString, iprofile_completeness_incomplete_fields#8850594.toString, staticinvoke(class org.apache.spark.sql.catalyst.util.DateTimeUtils$, ObjectType(class java.sql.Timestamp), toJavaTimestamp, update_ts#8850595, true, false, true), updated_by#8850596.toString, staticinvoke(class java.lang.Short, ObjectType(class java.lang.Short), valueOf, verification_status#8850597, true, false, true), status#8850598.toString, ... 50 more fields), obj#9656718: org.apache.spark.sql.Row
(6) DeserializeToObject
Input [37]: [_id#8850575, address_city#8850576, address_country#8850577, address_country_code#8850578, address_country_code_alpha3#8850579, address_zip#8850580, address_lat#8850581, address_lng#8850582, company_id#8850583, company_type#8850584, contact_info_email#8850585, contact_info_phone_number#8850586, create_ts#8850587, created_by#8850588, description#8850589, name#8850590, primary_contact#8850591, profile_completeness_score_favgaclity#8850592, profile_completeness_company_score#8850593, iprofile_completeness_incomplete_fields#8850594, update_ts#8850595, updated_by#8850596, verification_status#8850597, status#8850598, address_state#8850599, address_line1#8850600, address_line2#8850601, file_name#8850602, lwg_supported#8850603, unique_code#8850604, profile_info_score#8850605, max_scored_score#8850606, contact_name#8850607, create_date#8850608, update_date#8850609, ETL_runtime#8850610, logo_url#8850612]
Arguments: createexternalrow(_id#8850575.toString, address_city#8850576.toString, address_country#8850577.toString, address_country_code#8850578.toString, address_country_code_alpha3#8850579.toString, address_zip#8850580.toString, staticinvoke(class java.lang.Double, ObjectType(class java.lang.Double), valueOf, address_lat#8850581, true, false, true), staticinvoke(class java.lang.Double, ObjectType(class java.lang.Double), valueOf, address_lng#8850582, true, false, true), company_id#8850583.toString, company_type#8850584.toString, contact_info_email#8850585.toString, contact_info_phone_number#8850586.toString, staticinvoke(class org.apache.spark.sql.catalyst.util.DateTimeUtils$, ObjectType(class java.sql.Timestamp), toJavaTimestamp, create_ts#8850587, true, false, true), created_by#8850588.toString, description#8850589.toString, name#8850590.toString, primary_contact#8850591.toString, profile_completeness_score_favgaclity#8850592.toString, profile_completeness_company_score#8850593.toString, iprofile_completeness_incomplete_fields#8850594.toString, staticinvoke(class org.apache.spark.sql.catalyst.util.DateTimeUtils$, ObjectType(class java.sql.Timestamp), toJavaTimestamp, update_ts#8850595, true, false, true), updated_by#8850596.toString, staticinvoke(class java.lang.Short, ObjectType(class java.lang.Short), valueOf, verification_status#8850597, true, false, true), status#8850598.toString, ... 50 more fields), obj#9656718: org.apache.spark.sql.Row
(7) AdaptiveSparkPlan
Output [1]: [obj#9656718]
Arguments: isFinalPlan=true