spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Derek Schoettle <>
Subject Re: ETL and workflow management on Spark
Date Thu, 22 May 2014 14:50:42 GMT


From:	William Kang <>
Date:	05/22/2014 10:50 AM
Subject:	ETL and workflow management on Spark

We are moving into adopting the full stack of Spark. So far, we have used
Shark to do some ETL work, which is not bad but is not prefect either. We
ended writing UDF and UDGF, UDAF that can be avoided if we could use Pig.

Do you have any suggestions with the ETL solution in Spark stack?

And did any one have a working work flow management solution with Spark?

Many thanks.

View raw message