spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivani Rao <raoshiv...@gmail.com>
Subject using Log4j to log INFO level messages on workers
Date Mon, 02 Jun 2014 23:18:20 GMT
Hello Spark fans,

I am trying to log messages from my spark application. When the main()
function attempts to log, using log.info() it works great, but when I try
the same command from the code that probably runs on the worker, I
initially got an serialization error. To solve that, I created a new logger
in the code that operates on the data, which solved the serialization issue
but now there is no output in the console or on the worker node logs. I
don't see any application level log messages in the spark logs either. When
I use println() instead, I do see console output being  generated.

I tried the following and none of them works

a) pass log4j.properties by using -Dlog4j.properties in my java command
line initiation of the spark application
b) setting the properties within the worker by calling log.addAppender(new
ConsoleAppender)

None of them work.

What am i missing?


Thanks,
Shivani
-- 
Software Engineer
Analytics Engineering Team@ Box
Mountain View, CA

Mime
View raw message