spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <>
Subject Re: ETL and workflow management on Spark
Date Thu, 22 May 2014 16:01:49 GMT
We are in process of migrating Pig on spark. What is your currrent Spark
Version & cluster management that you use?
Also what is the datasize you are working with right now.

Mayur Rustagi
Ph: +1 (760) 203 3257
@mayur_rustagi <>

On Thu, May 22, 2014 at 8:19 PM, William Kang <>wrote:

> Hi,
> We are moving into adopting the full stack of Spark. So far, we have used
> Shark to do some ETL work, which is not bad but is not prefect either. We
> ended writing UDF and UDGF, UDAF that can be avoided if we could use Pig.
> Do you have any suggestions with the ETL solution in Spark stack?
> And did any one have a working work flow management solution with Spark?
> Many thanks.
> Cao

View raw message