spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Qiao <richardqiao2...@gmail.com>
Subject Re: optimize hive query to move a subset of data from one partition table to another table
Date Sun, 11 Feb 2018 21:30:59 GMT
Would you mind share your code with us to analyze?

> On Feb 10, 2018, at 10:18 AM, amit kumar singh <amitiemcal@gmail.com> wrote:
> 
> Hi Team,
> 
> We have hive external  table which has 50 tb of data partitioned on year month day
> 
> i want to move last 2 month of data into another table
> 
> when i try to do this through spark ,more than 120k task are getting created
> 
> what is the best way to do this
> 
> thanks
> Rohit


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message