kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rosenberg <...@squareup.com>
Subject Re: Compression in kafka 0.8
Date Wed, 05 Jun 2013 07:13:49 GMT
In this case, does the consumer code need to change, account for the
compression (or is this handled automatically in the consumer apis?).

If we decide to start compressing messages to a topic, can a consumer
seamlessly move between the transition from uncompressed to compressed?

Jason


On Tue, Jun 4, 2013 at 11:16 PM, Vadim Keylis <vkeylis2009@gmail.com> wrote:

> Thanks so much Neha
>
>
> On Tue, Jun 4, 2013 at 9:57 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > The only way of storing compressed data on the server is to have the
> > producer send it compressed. The server sends the data in compressed
> format
> > to the consumers.
> >
> > Thanks,
> > Neha
> >
> >
> > On Tue, Jun 4, 2013 at 9:55 PM, Vadim Keylis <vkeylis2009@gmail.com>
> > wrote:
> >
> > > Good evening. I would like to preserve logs in kafka for 2 days. Due to
> > the
> > > size of data it must be compressed otherwise it will take enormous
> amount
> > > of disk space. Its my understanding that the only way to store messages
> > in
> > > kafka compressed is to enable compression on the producer side, is that
> > > correct? Will they be sent to consumer also in compressed format? How
> the
> > > consumer can check if stream been sent compressed or not? Are there
> other
> > > options that will allow storing data in kafka for n number of days
> > > compressed?
> > >
> > > Thanks so much,
> > > Vadim
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message