Would like to know if the columns will be inserted into the Hive table in the correct position when I use insert into the target table select * from the temp table created in spark.
I am trying to insert into a partitioned table in hive from spark. I do a select on the df and then create a temp table and then insert into target from the temp table. Would like to know if it will preserve the Order.
df.select("col1","col2","col3","YearMonth").createOrReplaceTempView("Table1")
spark.sql("insert into TargetTable partition(yyyymm) select * from Table1")
>Solution :
Yes, order is maintained.
But in addition,
If your TargetTable has same no of columns and corresponding datatypes from select * query, then there wont be a problem