spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jimmy McErlain <>
Subject Re: which is the recommended workflow engine for Apache Spark jobs?
Date Mon, 10 Nov 2014 18:43:28 GMT
I have used Oozie for all our workflows with Spark apps but you will have
to use a java event as the workflow element.   I am interested in anyones
experience with Luigi and/or any other tools.

On Mon, Nov 10, 2014 at 10:34 AM, Adamantios Corais <> wrote:

> I have some previous experience with Apache Oozie while I was developing
> in Apache Pig. Now, I am working explicitly with Apache Spark and I am
> looking for a tool with similar functionality. Is Oozie recommended? What
> about Luigi? What do you use \ recommend?


"Nothing under the sun is greater than education. By educating one person
and sending him/her into the society of his/her generation, we make a
contribution extending a hundred generations to come."
-Jigoro Kano, Founder of Judo-

View raw message