flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9011) YarnResourceManager spamming log file at INFO level
Date Wed, 21 Mar 2018 11:23:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407770#comment-16407770
] 

ASF GitHub Bot commented on FLINK-9011:
---------------------------------------

Github user yanghua commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5712#discussion_r176048790
  
    --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java ---
    @@ -351,16 +351,16 @@ static ContainerLaunchContext createTaskExecutorContext(
     		require(yarnClientUsername != null, "Environment variable %s not set", YarnConfigKeys.ENV_HADOOP_USER_NAME);
     
     		final String remoteKeytabPath = env.get(YarnConfigKeys.KEYTAB_PATH);
    -		log.info("TM:remote keytab path obtained {}", remoteKeytabPath);
    -
     		final String remoteKeytabPrincipal = env.get(YarnConfigKeys.KEYTAB_PRINCIPAL);
    -		log.info("TM:remote keytab principal obtained {}", remoteKeytabPrincipal);
    -
     		final String remoteYarnConfPath = env.get(YarnConfigKeys.ENV_YARN_SITE_XML_PATH);
    -		log.info("TM:remote yarn conf path obtained {}", remoteYarnConfPath);
    -
     		final String remoteKrb5Path = env.get(YarnConfigKeys.ENV_KRB5_PATH);
    -		log.info("TM:remote krb5 path obtained {}", remoteKrb5Path);
    +
    +		if (log.isDebugEnabled()) {
    --- End diff --
    
    I know as I said in the previous comment `and each of them would do the same judgement
inside the debug method.`. Here I do a judgement to avoid that if the log level upper then
DEBUG, it could just jump these code.
    



> YarnResourceManager spamming log file at INFO level
> ---------------------------------------------------
>
>                 Key: FLINK-9011
>                 URL: https://issues.apache.org/jira/browse/FLINK-9011
>             Project: Flink
>          Issue Type: Bug
>          Components: ResourceManager, YARN
>    Affects Versions: 1.5.0
>            Reporter: Nico Kruber
>            Assignee: vinoyang
>            Priority: Blocker
>              Labels: flip-6
>             Fix For: 1.5.0
>
>
> For every requested resource, the {{YarnResourceManager}} spams the log with log-level
INFO and the following messages:
> {code}
> 2018-03-16 03:41:20,180 INFO  org.apache.flink.yarn.YarnResourceManager             
       - Received new container: container_1521038088305_0257_01_000102 - Remaining pending
container requests: 301
> 2018-03-16 03:41:20,180 INFO  org.apache.flink.yarn.YarnResourceManager             
       - TaskExecutor container_1521038088305_0257_01_000102 will be started with container
size 8192 MB, JVM heap size 5120 MB, JVM direct memory limit 3072 MB
> 2018-03-16 03:41:20,180 INFO  org.apache.flink.yarn.YarnResourceManager             
       - TM:remote keytab path obtained null
> 2018-03-16 03:41:20,180 INFO  org.apache.flink.yarn.YarnResourceManager             
       - TM:remote keytab principal obtained null
> 2018-03-16 03:41:20,180 INFO  org.apache.flink.yarn.YarnResourceManager             
       - TM:remote yarn conf path obtained null
> 2018-03-16 03:41:20,180 INFO  org.apache.flink.yarn.YarnResourceManager             
       - TM:remote krb5 path obtained null
> 2018-03-16 03:41:20,181 INFO  org.apache.flink.yarn.Utils                           
       - Copying from file:/mnt/yarn/usercache/hadoop/appcache/application_1521038088305_0257/container_1521038088305_0257_01_000001/6766be70-82f7-4999-a371-11c27527fb6e-taskmanager-conf.yaml
to hdfs://ip-172-31-1-91.eu-west-1.compute.internal:8020/user/hadoop/.flink/application_1521038088305_0257/6766be70-82f7-4999-a371-11c27527fb6e-taskmanager-conf.yaml
> 2018-03-16 03:41:20,190 INFO  org.apache.flink.yarn.YarnResourceManager             
       - Prepared local resource for modified yaml: resource { scheme: "hdfs" host: "ip-172-31-1-91.eu-west-1.compute.internal"
port: 8020 file: "/user/hadoop/.flink/application_1521038088305_0257/6766be70-82f7-4999-a371-11c27527fb6e-taskmanager-conf.yaml"
} size: 595 timestamp: 1521171680190 type: FILE visibility: APPLICATION
> 2018-03-16 03:41:20,194 INFO  org.apache.flink.yarn.YarnResourceManager             
       - Creating container launch context for TaskManagers
> 2018-03-16 03:41:20,194 INFO  org.apache.flink.yarn.YarnResourceManager             
       - Starting TaskManagers with command: $JAVA_HOME/bin/java -Xms5120m -Xmx5120m -XX:MaxDirectMemorySize=3072m
 -Dlog.file=<LOG_DIR>/taskmanager.log -Dlogback.configurationFile=file:./logback.xml
-Dlog4j.configuration=file:./log4j.properties org.apache.flink.yarn.YarnTaskExecutorRunner
--configDir . 1> <LOG_DIR>/taskmanager.out 2> <LOG_DIR>/taskmanager.err
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message