spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: ETL and workflow management on Spark
Date Thu, 22 May 2014 16:01:49 GMT
Hi,
We are in process of migrating Pig on spark. What is your currrent Spark
setup?
Version & cluster management that you use?
Also what is the datasize you are working with right now.
Regards
Mayur

Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>



On Thu, May 22, 2014 at 8:19 PM, William Kang <weliam.cloud@gmail.com>wrote:

> Hi,
> We are moving into adopting the full stack of Spark. So far, we have used
> Shark to do some ETL work, which is not bad but is not prefect either. We
> ended writing UDF and UDGF, UDAF that can be avoided if we could use Pig.
>
> Do you have any suggestions with the ETL solution in Spark stack?
>
> And did any one have a working work flow management solution with Spark?
>
> Many thanks.
>
>
> Cao
>

Mime
View raw message