gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Piotr Grabuszynski (JIRA)" <>
Subject [jira] [Commented] (GEARPUMP-161) Application logs are not created on Yarn
Date Tue, 02 Aug 2016 12:28:20 GMT


Piotr Grabuszynski commented on GEARPUMP-161:

I've observed such a behavior:
I've been trying to change yarn config, the way I could see Gearpump's logs in logstash. I've
modified /conf/ in Gearpump's zip. Then, Gearpump has been deployed on yarn.
In dashboard - there's a link to home dir (workers -> "home dir" button). I've got into
the worker's directory, then "pack/gearpump-2.11-0.8.0/conf/"
cd /data1/hadoop/yarn/nm/usercache/cf/appcache/application_1468841606343_0158/container_e05_1468841606343_0158_01_000003/pack/gearpump-2.11-0.8.0/conf/
and there were file modified by me.

I was surprised when I've got into conf directory - there were another, very short
#gearpump on yarn
#Fri Jul 22 09:57:57 UTC 2016
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH\:mm\:ss} %p %c{2}\: %m%n

After investigation, the conf folder and it's file is not a part of gearpump
and gearpump-broker applications. It's probably created by gearpump on the fly:
(am I right?)
looks like 
al log4j_properties = s"$confDir/"
val log4j = LogUtil.loadConfiguration
doesn't work

*Result: after manual changes in conf/ file, Apache Gearpump logs were sent
to logstash, so it seems that it's the only file in use.*

> Application logs are not created on Yarn 
> -----------------------------------------
>                 Key: GEARPUMP-161
>                 URL:
>             Project: Apache Gearpump
>          Issue Type: Bug
>          Components: yarn
>    Affects Versions: 0.8.0
>         Environment: yarn deployment: 1 master, 1 worker;
>            Reporter: Karol Brejna
>            Assignee: Huafeng Wang
> I have a simple cluster (1 master, 1 worker) running on YARN.
> I deploy an app. The app lands on a worker (cdh-worker-2 machine in this case).
> I inspect the worker and the app settings in the dashborad.
> Worker dir is shown as: /var/log/hadoop-yarn/container/application_1466296175601_0020/container_e05_1466296175601_0020_01_000003
> App log dir: /data1/hadoop/yarn/nm/usercache/cf/appcache/application_1466296175601_0020/container_e05_1466296175601_0020_01_000003/logs
> The logs are written to worker log dir under stderr.
> But there is no such a folder for app logs. 'logs' doesn exists under indicated location.
> I tried to do a wrokarounds (as I thought it may be a simple permissions issue):
> 1.
> chmod 777 on parent dir (/data1/hadoop/yarn/nm/usercache/cf/appcache/application_1466296175601_0020/container_e05_1466296175601_0020_01_000003)
> restarted app 
> No results here. logs dir not created.
> 2. 
> created logs dir by hand and changed permissions:
> mkdir logs
> chmod 777 logs
> No results here. logs dir not populated.

This message was sent by Atlassian JIRA

View raw message