kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viktor Somogyi <viktorsomo...@gmail.com>
Subject Re: Reduce Kafka Client logging
Date Thu, 07 Sep 2017 09:34:25 GMT
Hi Raghav,

I think it is enough to raise the logging level
of org.apache.kafka.clients.producer.ProducerConfig to WARN in log4j.
Also I'd like to mention that if possible, don't recreate the Kafka
producer each time. The protocol is designed for long-living connections
and recreating the connection each time puts pressure on the TCP layer (the
connection is expensive) and also on Kafka as well which may result in
broker failures (typically exceeding the maximum allowed number of file
descriptors).

HTH,
Viktor

On Thu, Sep 7, 2017 at 7:35 AM, Raghav <raghavastic@gmail.com> wrote:

> Due to the nature of code, I have to open a connection to a different Kafka
> broker each time, and send one message. We have several Kafka brokers. So
> my client log is full with the following logs. What log settings should I
> use in log4j just for Kafka producer logs ?
>
>
> 17/09/07 04:44:04 INFO producer.ProducerConfig:180 ProducerConfig values:
>         acks = all
>         batch.size = 16384
>         block.on.buffer.full = false
>         bootstrap.servers = [10.10.10.5:]
>         buffer.memory = 33554432
>         client.id =
>         compression.type = none
>         connections.max.idle.ms = 540000
>         interceptor.classes = null
>         key.serializer = class
> org.apache.kafka.common.serialization.StringSerializer
>         linger.ms = 1
>         max.block.ms = 5000
>         max.in.flight.requests.per.connection = 5
>         max.request.size = 1048576
>         metadata.fetch.timeout.ms = 60000
>         metadata.max.age.ms = 300000
>         metric.reporters = []
>         metrics.num.samples = 2
>         metrics.sample.window.ms = 30000
>         partitioner.class = class
> org.apache.kafka.clients.producer.internals.DefaultPartitioner
>         receive.buffer.bytes = 32768
>         reconnect.backoff.ms = 50
>         request.timeout.ms = 5000
>         retries = 0
>         retry.backoff.ms = 100
>         sasl.kerberos.kinit.cmd = /usr/bin/kinit
>         sasl.kerberos.min.time.before.relogin = 60000
>         sasl.kerberos.service.name = null
>         sasl.kerberos.ticket.renew.jitter = 0.05
>         sasl.kerberos.ticket.renew.window.factor = 0.8
>         sasl.mechanism = GSSAPI
>         security.protocol = PLAINTEXT
>         send.buffer.bytes = 131072
>         ssl.cipher.suites = null
>         ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
>         ssl.endpoint.identification.algorithm = null
>         ssl.key.password = null
>         ssl.keymanager.algorithm = SunX509
>         ssl.keystore.location = null
>         ssl.keystore.password = null
>         ssl.keystore.type = JKS
>         ssl.protocol = TLS
>         ssl.provider = null
>         ssl.secure.random.implementation = null
>         ssl.trustmanager.algorithm = PKIX
>         ssl.truststore.location = null
>         ssl.truststore.password = null
>         ssl.truststore.type = JKS
>         timeout.ms = 30000
>         value.serializer = class
> org.apache.kafka.common.serialization.StringSerializer
>
> On Wed, Sep 6, 2017 at 9:37 PM, Jaikiran Pai <jai.forums2013@gmail.com>
> wrote:
>
> > Can you post the exact log messages that you are seeing?
> >
> > -Jaikiran
> >
> >
> >
> > On 07/09/17 7:55 AM, Raghav wrote:
> >
> >> Hi
> >>
> >> My Java code produces Kafka config overtime it does a send which makes
> log
> >> very very verbose.
> >>
> >> How can I reduce the Kafka client (producer) logging in my java code ?
> >>
> >> Thanks for your help.
> >>
> >>
> >
>
>
> --
> Raghav
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message