samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navina Ramesh <nram...@linkedin.com>
Subject Re: Review Request 44293: SAMZA-883 Improve logging for container handling and kafka abdication
Date Wed, 02 Mar 2016 22:27:04 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44293/#review121738
-----------------------------------------------------------




samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerRequestState.java (line 114)
<https://reviews.apache.org/r/44293/#comment183543>

    I don't think this log line should be at info level. The reason being that Yarn continues
to return resources until we actually assign a container to a resource or release the resource.
Hence, if you request resource A with a 30 second timeout, it will continue allocating resources
B, C, C, C, ... etc , if resource A is not available for allocation. This will happen until
the request timesout at 30 seconds. After which we pick one resource and release the rest.
This is the current logic and hence, I suggest we leave this at debug.
    
    Why do you need these log statements to be in info? These were added for the purpose of
debugging.


- Navina Ramesh


On March 2, 2016, 9:39 p.m., Jake Maes wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44293/
> -----------------------------------------------------------
> 
> (Updated March 2, 2016, 9:39 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> SAMZA-883 Improve logging for container handling and kafka refresh
> 
> 
> Diffs
> -----
> 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala 9aa98184f8ab186eff226482f894e8842b9525e5

>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala
b37375360bcad8e3333089d95e8e3da63a1b9aab 
>   samza-kafka/src/main/scala/org/apache/samza/util/ClientUtilTopicMetadataStore.scala
0f91622c0c4c270a376f8b91b3e785cca57bc9dd 
>   samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestKafkaSystemConsumer.scala
ece0359d1a31e65113f4832dd3baddb788cecef4 
>   samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerRequestState.java 54db5e5b2b1b109d202e814809adbfd2bc84fb4b

>   samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerUtil.java 91fae98c074e1648e7168fb8e76d6e1e656816fc

> 
> Diff: https://reviews.apache.org/r/44293/diff/
> 
> 
> Testing
> -------
> 
> Manual testing and log verification (made sure they weren't verbose)
> 
> 
> Thanks,
> 
> Jake Maes
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message