spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dennis Suhari <d.suh...@icloud.com.INVALID>
Subject Spark and Oozie
Date Fri, 19 Jul 2019 07:08:41 GMT

Dear experts,

I am using Spark for processing data from HDFS (hadoop). These Spark application are data
pipelines, data wrangling and machine learning applications. Thus Spark submits its job using
YARN. 
This also works well. For scheduling I am now trying to use Apache Oozie, but I am facing
performqnce impacts. A Spark job which tooks 44 seconds when submitting it via CLI now takes
nearly 3 Minutes.

Have you faced similar experiences in using Oozie for scheduling Spark application jobs ?
What alternative workflow tools are you using for scheduling Spark jobs on Hadoop ?


Br,

Dennis

Von meinem iPhone gesendet
Von meinem iPhone gesendet

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message