kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rosenberg <...@squareup.com>
Subject Re: kafka_2.8.0/0.8.0 pom seems invalid
Date Thu, 05 Dec 2013 21:37:50 GMT
Joe,

I expect the vast majority of kafka users are using java for client apps,
and in most cases, they will just use the default jars they get in the
binary download.

I think the confusing thing here, is that there is a single jar file, that
includes all the broker, consumer and producer client code.  The 'broker'
is really just some scripts wrapped around launching the jvm with this jar
file.  If you download the binary, you also have scripts that launch
consumers and clients, using that same jar, just passing different command
line arguments.

So, really, there are not separate artifacts to speak of, from a broker,
consumer, producer stand point.   Good news, I understand there are plans
in the works to have separate jars for the broker, consumer clients,
producer clients, in 0.9.x.

Thanks for the info around 2.8.2 and beyond.  However, I think we'll try to
stay using 2.8.0 for the broker (since that seems to be the stable version
used at LinkedIn).  And in our env, we use a single artifact (via maven)
for all our broker and client apps.....

I don't think the issue in KAFKA-1163 is likely an upload issue (since when
I run 'sbt make-pom' locally, it builds it the same way as the broken one
uploaded to the public repo).

At the very least, I should think you'd want to remove the currently broken
one from the public repo, since from a maven standpoint, it's not
usable.....Perhaps a simple patch that only fixes the pom issue (call it
0.8.0.1?).

Jason






On Thu, Dec 5, 2013 at 9:43 AM, Joe Stein <joe.stein@stealth.ly> wrote:

> << Heh, that sounds amazing, considering that's the binary release
> version you've put up for download :)
>
> Not really.  The Kafka broker compiled in 2.8.0 Scala is a result of "that
> is how it runs at LinkedIn" originally (and still I think) and the how it
> got run everywhere else as a result.  So the majority of installs from
> inception has been 2.8.0 Kafka brokers... Now... for applications producing
> and consuming (at least for Scala shops) 2.8.0 is just not what you want...
> I switched from 2.8.0 to 2.9.0-1 almost immediately because 2.9.0-1 had
> lots of features that I wanted in my business domain application (which had
> nothing to-do with Kafka) and eventually to 2.10 (again, for language
> features in the business domain application having nothing to-do with
> Kafka). So, would Kafka benefit from compiling the broker in something
> other than 2.8.0? Maybe if we wanted to use language features in the broker
> or such otherwise it is proven to be stable so why add risk to something by
> changing it (especially without testing and proving it in production for
> the community) if it is working just fine... so, long story short it is
> about risk mitigation from a production perspective.
>
> if you are in a Java environment then it would be best to use 2.8.2 (or
> even move forward) for the producer/consumer to avoid bugs between things
> you might be doing in your code and the producers and consumers.  The
> broker is 100% self contained so if it was going to hit any Scala bugs from
> 2.8.0 it would have done so already in the last few years.
>
> << Anyway, it seems this pom issue ought to be solvable, no?
>
> it is very possible it was an upload issue during the release and running
> it again would fix. I will update the release notes and write some code to
> use to test every single version next time on staging before the vote...
> and that others can use during the vote which David Arthur has now also
> created too for ant/ivy
>
> Hope things makes sense and helps to clear things up.  There has been talk
> in 0.8.1 to start to move things forward from a Scala perspective.
>
> /*******************************************
>  Joe Stein
>  Founder, Principal Consultant
>  Big Data Open Source Security LLC
>  http://www.stealth.ly
>  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
> On Wed, Dec 4, 2013 at 10:20 PM, Jason Rosenberg <jbr@squareup.com> wrote:
>
> > Here's the jira:  https://issues.apache.org/jira/browse/KAFKA-1163
> >
> > "I was honestly not aware folks still used 2.8.0 and there have been
> talks
> > about discontinuing that."
> >
> > Heh, that sounds amazing, considering that's the binary release version
> > you've put up for download :)
> >
> > I am using that too for both client/server (for no particular reason,
> > everything I use with kafka (brokers, consumers, producers) are embedded
> in
> > java apps, so I have less scala dependencies anyway, maybe I should
> > upgrade?).
> >
> > Anyway, it seems this pom issue ought to be solvable, no?
> >
> > Jason
> >
> >
> > On Wed, Dec 4, 2013 at 5:17 PM, hsy541@gmail.com <hsy541@gmail.com>
> wrote:
> >
> > > Thanks Joe. If any 0.8.0 client is compatible with this release, I'll
> > just
> > > upgrade my client.
> > >
> > >
> > > On Wed, Dec 4, 2013 at 1:59 PM, Joe Stein <joe.stein@stealth.ly>
> wrote:
> > >
> > > > There is a ticket for the 2.8.0 POM issue (I would send the link to
> it
> > > but
> > > > JIRA seems to be down so don't know the ticket off the top of my
> head).
> > > >
> > > > The binary releases are only done using 2.8.0 but that really
> shouldn't
> > > > matter since that is just the broker and your producers and consumers
> > are
> > > > communicating over the wire protocol using TCP/IP to it (so you can
> run
> > > > whatever client you want with whatever language available and version
> > of
> > > > the language).  There have been talks about changing (using another
> > Scala
> > > > version to build the broker) but this will be more of a consensus for
> > > > operating the broker in production with that Scala version and having
> > > > confidence around that from use.
> > > >
> > > > I haven't looked into the 2.8.0 POM issue yet myself it could be code
> > > > related or even part of the publishing process that bonked at some
> > point
> > > > (it happens).
> > > >
> > > > Have you tried doing a publish-local to see if that works?
> > > >
> > > > I was honestly not aware folks still used 2.8.0 and there have been
> > talks
> > > > about discontinuing that.
> > > >
> > > > Why do you need the 2.8.0 client jar but a 2.10 broker?
> > > >
> > > > Lastly if it becomes such a big issue for the community and a fix is
> > > > required before we can get trunk released we could always role 0.8.1
> > from
> > > > branch and make the trunk 0.8.2 however we should go through some due
> > > > diligence since 0.8.1 is not really that far off and the trade off
> may
> > > not
> > > > make sense.
> > > >
> > > > /*******************************************
> > > >  Joe Stein
> > > >  Founder, Principal Consultant
> > > >  Big Data Open Source Security LLC
> > > >  http://www.stealth.ly
> > > >  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > > > ********************************************/
> > > >
> > > >
> > > > On Wed, Dec 4, 2013 at 4:48 PM, hsy541@gmail.com <hsy541@gmail.com>
> > > wrote:
> > > >
> > > > > Hi All, I was trying to upgrade the kafka to 0.8 but I get an empty
> > jar
> > > > > file for
> > > > > <dependency>
> > > > >     <groupId>org.apache.kafka</groupId>
> > > > >     <artifactId>kafka_2.8.0</artifactId>
> > > > >     <version>0.8.0</version>
> > > > > </dependency>
> > > > >
> > > > >
> > > > > However
> > > > > <dependency>
> > > > >     <groupId>org.apache.kafka</groupId>
> > > > >     <artifactId>kafka_2.8.2</artifactId>
> > > > >     <version>0.8.0</version>
> > > > > </dependency>
> > > > >
> > > > > is good for me.
> > > > >
> > > > > BTW from the download page I can only see kafka_2.8.0_0.8.0. Where
> > can
> > > I
> > > > > download the scala 2.10 version. Is there any difference between
> > > > different
> > > > > version?
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message