kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dillian Murphey <crackshotm...@gmail.com>
Subject Re: Anouncing PyKafka!
Date Fri, 12 Jun 2015 22:31:34 GMT
EXCELLENT WORK. Thank you!

On Wed, Jun 10, 2015 at 11:19 AM, Keith Bourgoin <keith.bourgoin@gmail.com>
wrote:

> I would guess within a week or two, but it's hard to say for sure.  Emmett
> Butler, the other main contributor, has been hard at work on it when he has
> the time.
>
> On Wed, Jun 10, 2015 at 12:18 PM Vijay Patil <vijay2110.tech@gmail.com>
> wrote:
>
> > Thanks a lot Keith for detailed explanation, that will surely help me.
> > Another question, I can see AsyncProducer is not yet implemented, any
> idea
> > by when it can be available?
> >
> > Thanks,
> > Vijay
> >
> > On 10 June 2015 at 18:42, Keith Bourgoin <keith.bourgoin@gmail.com>
> wrote:
> >
> > > Hi Vijay,
> > >
> > > PyKafka support custom partitioners so that you can control how the
> > > producer distributes messages.  Looking at it now, I realize the
> > > documentation around this is a bit lacking, so I'll add that.  In the
> > > meantime, here's an example of how to use the hashing partitioner which
> > is
> > > included with PyKafka.
> > >
> > > client = pykafka.KafkaClient()
> > >
> > > # Producer with random_partitioner
> > > prod = client.topics['foo'].get_producer()
> > > prod.produce(['foo', 'bar', 'baz'])
> > >
> > > # Producer with hashing_partitioner
> > > prod =
> > >
> >
> client.topics['foo'].get_producer(partitioner=pykafka.partitioners.hashing_partitioner)
> > > prod.produce([('p_key1', 'foo'), ('p_key2', 'foo'), ('p_key1', 'baz')])
> > >
> > > ​
> > >
> > > Cheers,
> > >
> > > Keith
> > > ​
> > >
> > > On Wed, Jun 10, 2015 at 6:35 AM Vijay Patil <vijay2110.tech@gmail.com>
> > > wrote:
> > >
> > > > Thanks,
> > > > a question, does it support keyed Producer?
> > > > I mean how it works when I have multiple partitions? Will it be able
> to
> > > > identify partition based of the key which I pass?
> > > >
> > > > On 9 June 2015 at 00:54, Keith Bourgoin <keith.bourgoin@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Kafka folks,
> > > > >
> > > > > I'm happy to announce the 1.0 release of PyKafka
> > > > > <https://github.com/parsely/pykafka>, a new Python driver for
> Kafka!
> > > > >
> > > > > This is an upgrade of an older library, named samsa, which was only
> > > > > compatible with Kafka 0.7.x. We've spent the last few months
> > upgrading
> > > it
> > > > > to work with Kafka 0.8.2.1 and revamping the codebase to be cleaner
> > and
> > > > > more efficient.
> > > > >
> > > > > The biggest difference between PyKafka and other Python drivers is
> > the
> > > > > inclusion of a balanced consumer implementation. We implemented the
> > > same
> > > > > balancing algorithm as the Java driver and PyKafka handles consumer
> > > > > registration via Zookeeper like the Java driver.
> > > > >
> > > > > We at Parse.ly have been using this driver in production for a few
> > > weeks
> > > > > and are confident in its stability. Even so, we're looking to add
> > more
> > > > > features over the next couple months and are always trying to find
> > ways
> > > > to
> > > > > improve performance.
> > > > >
> > > > > You can find the project on PyPI <
> > > https://pypi.python.org/pypi/pykafka/
> > > > >,
> > > > > Github <https://github.com/parsely/pykafka> and our mailing
list
> > > > > <https://groups.google.com/forum/#%21forum/pykafka-user>. We
also
> > > wrote
> > > > a
> > > > > blog
> > > > > post
> > > > > <
> > > > >
> > > >
> > >
> >
> http://blog.parsely.com/post/2006/announcing-pykafka-python-support-for-kafka-0-8-x/
> > > > > >
> > > > > with a bit more info.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Keith.
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message