logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ralph Goers <ralph.go...@dslextreme.com>
Subject Re: Flume Appender
Date Thu, 27 Dec 2012 16:01:18 GMT
A single class is an application, albeit simple, and Maven is perfectly suitable for building
it. I use Maven for building everything, including tiny little test projects. That said, Log4j
2 has no requirement that you use Maven.

Yes, using Log4j 2 -> Flume -> HDFS is a good approach.  Most of the dependencies you
need are what Flume requires for an application to communicate with the Flume agent.  The
sample project I pointed you to is slightly more complicated that what you are doing as it
is a sample web application. I suppose I could add another sample that is a standalone app
if that would make life easier. Although it would use Maven you could just look at the result
of its build to know what you need.

Ralph


On Dec 27, 2012, at 2:09 AM, Pradeep Kanchgar wrote:

> Hi Ralph,
> 
> I'm not actually using log4j2 for an application, it's just simple class where I've some
logging statements which need to go on HDFS(Hadoop Distributed File System) And flume does
this job, am I right?
> After you mentioned maven, I searched a bit about maven and found out that it is normally
used for large projects to automate build process. Since I've not used maven I've to invest
some time on it. You mean to say  even for a simple class which logs some statements i need
to use maven. Can I do this without maven ? I'm asking too many questions in order  to make
sure that I'm not on wrong track..!!
> 
> Pradeep
> 
> -----Original Message-----
> From: Ralph Goers [mailto:ralph.goers@dslextreme.com] 
> Sent: Thursday, December 27, 2012 2:00 PM
> To: Log4J Users List
> Subject: Re: Flume Appender
> 
> For information on Maven see http://maven.apache.org.  It is a build tool used by lots
and lots of Java projects.
> 
> Most of those other Appenders only have dependencies on the JDK, which is why they are
part of the core and Flume is in a separate subproject.  The default configuration for using
Flume in Log4j is to use the Avro protocol to send the events to a Flume agent.  Flume also
uses Netty to handle the transport layer. Those are projects separate from Log4j 2 and Flume
and as you can see they have dependencies of their own.  So yes, if you want to use the Avro
sink to publish to a Flume agent you will need all those jars.  If you were using Maven as
your build tool it would handle all this for you. If you use Ant you can use Ivy to accomplish
something similar.
> 
> Ralph 
> 
> On Dec 26, 2012, at 11:23 PM, Pradeep Kanchgar wrote:
> 
>> Hi Ralph,
>> 
>> I'm using this log4j2 for the first time. What is this maven? With log4j2 if I use
other appenders like console or rollingfileappender  it works fine.
>> But in case of this flumeappender, I don't have much clarity.  You 
>> mean to say in order to use flume appender I need all these jar like 
>> Jackson,snappy, jetty..etc
>> 
>> 
>> 
>> -----Original Message-----
>> From: Ralph Goers [mailto:ralph.goers@dslextreme.com]
>> Sent: Thursday, December 27, 2012 11:51 AM
>> To: Log4J Users List
>> Subject: Re: Flume Appender
>> 
>> Running mvn dependency:tree in samples/flume-remote yields
>> 
>> [INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ 
>> flume-remote --- [INFO] 
>> org.apache.logging.log4j.samples:flume-remote:war:2.0-beta4-SNAPSHOT
>> [INFO] +- 
>> org.apache.logging.log4j.samples:flume-common:jar:2.0-beta4-SNAPSHOT:c
>> ompile [INFO] +- 
>> org.apache.logging.log4j:log4j-api:jar:2.0-beta4-SNAPSHOT:compile
>> [INFO] +- 
>> org.apache.logging.log4j:log4j-core:jar:2.0-beta4-SNAPSHOT:compile
>> [INFO] +- 
>> org.apache.logging.log4j.adapters:log4j-web:jar:2.0-beta4-SNAPSHOT:com
>> pile [INFO] |  +- org.slf4j:slf4j-api:jar:1.7.2:compile
>> [INFO] |  \- org.slf4j:slf4j-ext:jar:1.7.2:compile
>> [INFO] |     \- ch.qos.cal10n:cal10n-api:jar:0.7.4:compile
>> [INFO] +- 
>> org.apache.logging.log4j.adapters:log4j-flume-ng:jar:2.0-beta4-SNAPSHO
>> T:compile [INFO] |  \- org.apache.flume:flume-ng-sdk:jar:1.2.0:compile
>> [INFO] |     +- org.apache.avro:avro:jar:1.6.3:compile
>> [INFO] |     |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.2:compile (version
managed from 1.8.8)
>> [INFO] |     |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.2:compile (version
managed from 1.8.8)
>> [INFO] |     |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
>> [INFO] |     |  \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
>> [INFO] |     +- org.apache.avro:avro-ipc:jar:1.6.3:compile
>> [INFO] |     |  +- org.mortbay.jetty:jetty:jar:6.1.26:compile
>> [INFO] |     |  +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
>> [INFO] |     |  +- org.jboss.netty:netty:jar:3.2.7.Final:compile
>> [INFO] |     |  \- org.apache.velocity:velocity:jar:1.7:compile
>> [INFO] |     |     \- commons-collections:commons-collections:jar:3.2.1:compile
>> [INFO] |     +- commons-io:commons-io:jar:2.1:compile
>> [INFO] |     \- org.mortbay.jetty:servlet-api:jar:2.5-20110124:compile
>> [INFO] +- org.springframework:spring-beans:jar:3.1.2.RELEASE:compile
>> [INFO] +- org.springframework:spring-core:jar:3.1.2.RELEASE:compile
>> [INFO] |  +- org.springframework:spring-asm:jar:3.1.2.RELEASE:compile
>> [INFO] |  \- commons-logging:commons-logging:jar:1.1.1:compile
>> [INFO] +- org.springframework:spring-webmvc:jar:3.1.2.RELEASE:compile
>> [INFO] |  +- 
>> org.springframework:spring-context:jar:3.1.2.RELEASE:compile
>> [INFO] |  +- 
>> org.springframework:spring-context-support:jar:3.1.2.RELEASE:compile
>> [INFO] |  +- 
>> org.springframework:spring-expression:jar:3.1.2.RELEASE:compile
>> [INFO] |  \- org.springframework:spring-web:jar:3.1.2.RELEASE:compile
>> [INFO] |     \- aopalliance:aopalliance:jar:1.0:compile
>> [INFO] +- 
>> org.springframework.ws:spring-ws-core:jar:2.1.0.RELEASE:compile
>> [INFO] |  +- 
>> org.springframework.ws:spring-xml:jar:2.1.0.RELEASE:compile
>> [INFO] |  +- org.springframework:spring-aop:jar:3.1.1.RELEASE:compile
>> [INFO] |  +- org.springframework:spring-oxm:jar:3.1.1.RELEASE:compile
>> [INFO] |  |  \- commons-lang:commons-lang:jar:2.5:compile
>> [INFO] |  \- wsdl4j:wsdl4j:jar:1.6.1:compile [INFO] +- 
>> javax.servlet:servlet-api:jar:2.4:provided
>> [INFO] \- junit:junit:jar:4.3.1:test
>> 
>> From this you can see that avro requires jackson and several other jars.  In addition,
you have included avro 1.4.1 while the above shows avro 1.6.3. In looking at the source for
Avro 1.4.1 it does not define that particular class so you will need a more current version.
>> 
>> Ralph
>> 
>> 
>> 
>> 
>> 
>> 
>> On Dec 26, 2012, at 8:56 PM, Pradeep Kanchgar wrote:
>> 
>>> 
>>> Hi ,
>>> 
>>> I'm using flume Appender but getting the following error.
>>> 
>>> 2012-12-26 19:08:41,566 ERROR Unable to invoke method createAppender 
>>> in class org.apache.logging.log4j.flume.appender.FlumeAppender for 
>>> element Flume java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
>>> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at 
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at 
>>> java.lang.reflect.Method.invoke(Unknown Source) at 
>>> org.apache.logging.log4j.core.config.BaseConfiguration.createPluginOb
>>> j
>>> ect(BaseConfiguration.java:711) at
>>> org.apache.logging.log4j.core.config.BaseConfiguration.createConfigur
>>> a
>>> tion(BaseConfiguration.java:477) at
>>> org.apache.logging.log4j.core.config.BaseConfiguration.createConfigur
>>> a
>>> tion(BaseConfiguration.java:469) at
>>> org.apache.logging.log4j.core.config.BaseConfiguration.doConfigure(Ba
>>> s
>>> eConfiguration.java:156) at
>>> org.apache.logging.log4j.core.config.BaseConfiguration.start(BaseConf
>>> i
>>> guration.java:114) at
>>> org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerCo
>>> n
>>> text.java:251) at
>>> org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.
>>> java:267) at
>>> org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:
>>> 1
>>> 34) at
>>> org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log
>>> 4
>>> jContextFactory.java:75) at
>>> org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log
>>> 4
>>> jContextFactory.java:30) at
>>> org.apache.logging.log4j.LogManager.getLogger(LogManager.java:165)
>>> at log4jExample.<clinit>(log4jExample.java:8)
>>> Caused by: java.lang.NoClassDefFoundError: 
>>> org/apache/avro/AvroRemoteException
>>> at
>>> org.apache.logging.log4j.flume.appender.FlumeAppender.createAppender(
>>> F
>>> lumeAppender.java:173)
>>> ... 16 more
>>> Caused by: java.lang.ClassNotFoundException: 
>>> org.apache.avro.AvroRemoteException
>>> at java.net.URLClassLoader$1.run(Unknown Source) at 
>>> java.security.AccessController.doPrivileged(Native Method) at 
>>> java.net.URLClassLoader.findClass(Unknown Source) at 
>>> java.lang.ClassLoader.loadClass(Unknown Source) at 
>>> sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at 
>>> java.lang.ClassLoader.loadClass(Unknown Source) ... 17 more
>>> 
>>> 2012-12-26 19:08:41,567 ERROR Null object returned for Flume in 
>>> appenders
>>> 2012-12-26 19:08:41,577 ERROR Unable to locate appender eventLogger 
>>> for logger
>>> 
>>> my config file : log4j2.xml is
>>> 
>>> <?xml version="1.0" encoding="UTF-8"?> <configuration status="warn"

>>> name="log4jExample" packages=""> <appenders> <Flume name="eventLogger"

>>> suppressExceptions="false" compress="true"> <Agent 
>>> host="172.20.104.226" port="41414"/> <PatternLayout pattern="%m%n"/>

>>> </Flume> </appenders> <loggers> <root level="error">
<appender-ref 
>>> ref="eventLogger"/> </root> </loggers> </configuration>
>>> 
>>> my java file is
>>> 
>>> 
>>> public class log4jExample{
>>> /* Get actual class name to be printed on */  static Logger log = 
>>> LogManager.getLogger(
>>>                    log4jExample.class.getName());
>>> 
>>> public static void main(String[] args)
>>>              throws Exception{
>>> 
>>>       for (int i = 0; i < 100 ; i++) {
>>>               log.trace("Entering the Application");
>>>               log.debug("Current data unavailalbe, using cached values");
>>>               log.info("Hello this is an info message");
>>>               log.error("Dabase unavaliable, connetion lost");
>>>               log.warn("Attention!! Application running in debugmode");
>>>               log.trace("Existing the Application");
>>>         }
>>> 
>>> }
>>> }
>>> 
>>> I've added all the log4j2 jar files in classpath  variable in ~/.bashrc file.
>>> but still getting same error and I've already wasted an entire day in resolving
this, but no solution yet.
>>> And apache avro jar(avro-1.4.1.jar) is also in classpath.
>>> 
>>> Still the problem persists.
>>> 
>>> I've these jar files in my classpath : log4j12-api-2.0-beta3.jar, log4j-core-2.0-beta3.jar,
log4j-flume-ng-2.0-beta3.jar, log4j-jcl-2.0-beta3.jar, log4j-slf4j-impl-2.0-beta3.jar, avro-1.4.1.jar.
>>> 
>>> 
>>> Regards,
>>> Pradeep  Kanchgar
>>> 
>>> 
>>> ________________________________
>>> The contents of this e-mail and any attachment(s) may contain confidential or
privileged information for the intended recipient(s). Unintended recipients are prohibited
from taking action on the basis of information in this e-mail and using or disseminating the
information, and must notify the sender and delete it from their system. L&T Infotech
will not accept responsibility or liability for the accuracy or completeness of, or the presence
of any virus or disabling code in this e-mail"
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org


Mime
View raw message