logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ralph Goers <ralph.go...@dslextreme.com>
Subject Re: log4j2 integration with Hadoop using log4j
Date Fri, 19 Oct 2012 16:02:39 GMT
I've gotten the embedded agent working. It is a bit messy with dependency management. Here
is a sample pom.  I will add this to the documentation.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.myapp</groupId>
    <artifactId>myapp</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>My App</name>
    <description>App to test log4j appender</description>

    <properties>
        <log4j.version>2.0-beta1</log4j.version>
        <slf4j.version>1.6.6</slf4j.version>
        <jackson.version>1.9.3</jackson.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
                <version>${slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-ext</artifactId>
                <version>${slf4j.version></version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-api</artifactId>
                <version>${log4j2_version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j.adapters</groupId>
                <artifactId>log4j12-api</artifactId>
                <version>${log4j2_version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-core</artifactId>
                <version>${log4j2_version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j.adapters</groupId>
                <artifactId>log4j-flume-ng</artifactId>
                <version>${log4j2_version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j.adapters</groupId>
                <artifactId>slf4j-impl</artifactId>
                <version>${log4j2_version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j.adapters</groupId>
                <artifactId>log4j-jcl</artifactId>
                <version>${log4j2_version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j.adapters</groupId>
                <artifactId>log4j-web</artifactId>
                <version>${log4j2_version}</version>
            </dependency>
            <dependency>
                <groupId>org.codehaus.jackson</groupId>
                <artifactId>jackson-core-asl</artifactId>
                <version>${jackson.version}</version>
            </dependency>
            <dependency>
                <groupId>org.codehaus.jackson</groupId>
                <artifactId>jackson-mapper-asl</artifactId>
                <version>${jackson.version}</version>
            </dependency>
            <dependency>
                <groupId>org.codehaus.jackson</groupId>
                <artifactId>jackson-jaxrs</artifactId>
                <version>${jackson.version}</version>
            </dependency>
            <dependency>
                <groupId>org.codehaus.jackson</groupId>
                <artifactId>jackson-xc</artifactId>
                <version>${jackson.version}</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <!-- Log4j 2 API -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
        </dependency>
        <!-- Log4j 1.x API -->
        <dependency>
            <groupId>org.apache.logging.log4j.adapters</groupId>
            <artifactId>log4j12-api</artifactId>
        </dependency>
        <!-- Log4j 2 implementation -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
        </dependency>
        <!-- SLF4J to Log4j 2 binding -->
        <dependency>
            <groupId>org.apache.logging.log4j.adapters</groupId>
            <artifactId>slf4j-impl</artifactId>
        </dependency>
        <!-- SLF4J API -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
        </dependency>
        <!-- SLF4J extensions -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-ext</artifactId>
        </dependency>
        <!-- Commson Logging to Log4j 2 binding -->
        <dependency>
            <groupId>org.apache.logging.log4j.adapters</groupId>
            <artifactId>log4j-jcl</artifactId>
        </dependency>
        <!-- Log4j 2 Servlet Context Listener -->
        <dependency>
            <groupId>org.apache.logging.log4j.adapters</groupId>
            <artifactId>log4j-web</artifactId>
        </dependency>
        <!-- Avro transport protocol - required for Flume -->
        <dependency>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro</artifactId>
            <version>1.6.3</version>
        </dependency>
        <!-- Hadoop core - required by embedded Flume Agent -->
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>1.0.3</version>
            <exclusions>
                <exclusion>
                    <groupId>org.mortbay.jetty</groupId>
                    <artifactId>servlet-api</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.mortbay.jetty</groupId>
                    <artifactId>servlet-api-2.5</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- Flume File Channel - required by embedded Flume agent -->
        <dependency>
            <groupId>org.apache.flume.flume-ng-channels</groupId>
            <artifactId>flume-file-channel</artifactId>
            <version>1.2.0</version>
            <exclusions>
                <exclusion>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
</project>


On Oct 19, 2012, at 8:50 AM, Zhu Wayne wrote:

> Greetings!
> I am trying to use Log4j2 Flume Embedded Agent in PIG UDF code to send UDF
> logs from Data Nodes. However, Hadoop is on log4j 1.2.x version.  How could
> I use both log4j and log4j2 on the same JVM? PIG UDF using log4j2 will be
> called in Hadoop MapReduce using log4j 1.2.x.
> 
> Regards,
> Wayne


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message