kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dana Powers <dana.pow...@gmail.com>
Subject Re: The JIRA Awakens [KAFKA-1841]
Date Thu, 24 Dec 2015 17:55:10 GMT
Awesome ! Looks like a great distribution. I'd love to add HDP releases to
the kafka-python integration test suite.

-Dana
On Dec 24, 2015 9:33 AM, "Harsha" <kafka@harsha.io> wrote:

> HI Dana,
>             I worked on that release. Yes HDP-2.3.0 has lot of
>             additional patches on top of 0.8.2.1 mainly the kerberos
>             patches.
> We did missed KAFKA-1841 which was fixed in later maint release. We
> already notified everyone to upgrade HDP-2.3.4 this is the apache kafka
> 0.9.0 + additional patches . We are making sure on our side not to miss
> any compatibility patches like these with 3rd party developers and have
> tests to ensure that.
>
> Thanks,
> Harsha
> On Wed, Dec 23, 2015, at 04:11 PM, Dana Powers wrote:
> > Hi all,
> >
> > I've been helping debug an issue filed against kafka-python related to
> > compatibility w/ Hortonworks 2.3.0.0 kafka release. As I understand it,
> > HDP
> > is currently based on snapshots of apache/kafka trunk, merged with some
> > custom patches from HDP itself.
> >
> > In this case, HDP's 2.3.0.0 kafka release missed a compatibility patch
> > that
> > I believe is critical for third-party library support. Unfortunately the
> > patch -- KAFKA-1841 -- was initially only applied to the 0.8.2 branch (it
> > was merged to trunk several months later in KAFKA-2068). Because it
> > wasn't
> > on trunk, it didn't get included in the HDP kafka releases.
> >
> > If you recall, KAFKA-1841 was needed to maintain backwards and forwards
> > compatibility wrt the change from zookeeper to kafka-backed offset
> > storage.
> > Not having this patch is fine if you only ever use the clients /
> > libraries
> > distributed in the that release -- and I imagine that is probably most
> > folks that are using it. But if you remember the thread on this issue
> > back
> > in the 0.8.2-beta review, the API incompatibility made third-party
> > clients
> > hard to develop and maintain if the goal is to support multiple broker
> > versions w/ the same client code [this is the goal of kafka-python].
> > Anyways, I'm really glad that the fix made it into the apache release,
> > but
> > now I'm sad that it didn't make it into HDP's release.
> >
> > Anyways, I think there's a couple takeaways here:
> >
> > (1) I'd recommend anyone using HDP who intends to use third-party kafka
> > consumers should upgrade to 2.3.4.0 or later. That version appears to
> > include the compatibility patch (KAFKA-2068). Of course if anyone is on
> > list from HDP, they may be able to provide better help on this.
> >
> > (2) I think more care should probably be taken to help vendors or anyone
> > tracking changes on trunk wrt released versions. Is there a list of all
> > KAFKA-XXXX patches that are released but not merged into trunk ?
> > KAFKA-1841
> > is obviously near and dear to my heart, but I wonder if there are other
> > patches like it?
> >
> > Happy holidays to all, and may the force be with you
> >
> > -Dana
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message