kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix GV <fe...@mate1inc.com>
Subject Re: Kafka in AWS?
Date Tue, 20 Mar 2012 22:32:22 GMT
The primary use case for Kafka is to use it on AWS...???

Sorry if I put words you didn't intend in your mouth :P ... I just thought
that sounded funny ;)

Sorry for being off-topic. Carry on :/ !

--
Felix



On Tue, Mar 20, 2012 at 6:23 PM, Russell Jurney <russell.jurney@gmail.com>wrote:

> Yeah, that is the part I am hoping someone will contribute :)  I know I can
> write that myself.  I also know it will be buggy and that I will have lots
> of trouble.
>
> If you contribute this code, it would be a huge boon to Kafka.  It is imo
> the primary use case for Kafka atm... if only the code gets into git.
>
> On Tue, Mar 20, 2012 at 3:04 PM, Niek Sanders <niek.sanders@gmail.com
> >wrote:
>
> > Russell,
> >
> > I'm actually in the process of writing a Java code to go from Kafka
> > messages to S3.  I might be able to rip-out my application-specific
> > parts and share something later tonight.
> >
> > The biggest hassle is that you can't append to existing S3 files.  So
> > unless you're planning on uploading each message as a separate S3
> > object, this means you need message aggregation smarts on the Kafka
> > consumer / S3 uploader side of things.
> >
> > Best,
> > Niek
> >
> >
> >
> >
> >
> >
> > On Tue, Mar 20, 2012 at 12:56 PM, Russell Jurney
> > <russell.jurney@gmail.com> wrote:
> > > I wish someone would publish some source that writes events to S3.
> > >
> > > Russell Jurney
> > > twitter.com/rjurney
> > > russell.jurney@gmail.com
> > > datasyndrome.com
> > >
> > > On Mar 20, 2012, at 11:20 AM, Dave Fayram <dfayram@gmail.com> wrote:
> > >
> > >> We've been successfully using Kafka on AWS as well, and JMX wise we
> > >> just use an SSH tunnel.
> > >>
> > >> In general, we've been very happy with the performance on AWS, which
> > >> some people have reservations about due to the I/O situation on most
> > >> Amazon boxes.
> > >>
> > >> On Tue, Mar 20, 2012 at 9:07 AM, Gautam Singaraju
> > >> <gautam.singaraju@gmail.com> wrote:
> > >>> We are have been considering Kafka for a new Data Platform. Has
> someone
> > >>> used Kafka in AWS? If so, could you please share your experiences
> with
> > us?
> > >>>
> > >>> Thank you!
> > >>> ---
> > >>> Gautam
> > >>
> > >>
> > >>
> > >> --
> > >> --
> > >> Dave Fayram
> > >> dfayram@gmail.com
> >
>
>
>
> --
> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
> datasyndrome.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message