spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Tanase <atan...@adobe.com>
Subject Re: Differentiate Spark streaming in event logs
Date Mon, 19 Oct 2015 18:58:15 GMT
You could try to start the 2/N jobs with a slightly different log4j template, by prepending
some job type to all the messages...




On 10/19/15, 9:47 PM, "franklyn" <franklynd@gmail.com> wrote:

>Hi I'm running a job to collect some analytics on spark jobs by analyzing
>their event logs. We write the event logs to a single HDFS folder and then
>pick them up in another job. I'd like to differentiate between regular spark
>jobs and spark streaming jobs in the event logs, i was wondering if there is
>an event/property/key that is different between the two.
>
>thanks!,
>
>Franklyn
>
>
>
>--
>View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Differentiate-Spark-streaming-in-event-logs-tp25126.html
>Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>For additional commands, e-mail: user-help@spark.apache.org
>
Mime
View raw message