samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yi Pan <nickpa...@gmail.com>
Subject Re: Running apache samza with Kafka Client 1.0 - JIRA - SAMZA - 1418
Date Wed, 29 Nov 2017 21:32:46 GMT
Hey, Debraj,

The exception you saw looks strange. The exception class
TopicExistsException is from org.apache.kafka:kafka-clients:0.10.1.1 (which
is the version included in samza 0.13.1), which should exist if you have
the whole samza jars installed in your classpath. Could you check whether
you have multiple versions of kafka-clients jars in your classpath?

Thanks!

-Yi

On Wed, Nov 29, 2017 at 8:45 AM, Debraj Manna <subharaj.manna@gmail.com>
wrote:

> Hi
>
> Hi
>
> I am trying to upgrade a single node kafka broker to latest 1.0 from 0.10
> with samza 0.13.1.
>
> The steps followed
>
>
>    1. Stopped Kafka Broker
>    2. Replaced bin/, libs & site-docs/ with the latest
>    3. Started Kafka
>
> But I am seeing the below exception in logs - Is this related to the that
> is being discussed here?
>
> 017-11-29 16:21:36.591 [main-EventThread] ClientCnxn [INFO] EventThread
> shut down
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/kafka/common/errors/TopicExistsException
>         at org.apache.samza.system.kafka.KafkaSystemAdmin$$anonfun$
> createStream$2.apply(KafkaSystemAdmin.scala:442)
>         at org.apache.samza.system.kafka.KafkaSystemAdmin$$anonfun$
> createStream$2.apply(KafkaSystemAdmin.scala:440)
>         at org.apache.samza.util.ExponentialSleepStrategy.run(
> ExponentialSleepStrategy.scala:81)
>         at org.apache.samza.system.kafka.KafkaSystemAdmin.createStream(
> KafkaSystemAdmin.scala:422)
>         at org.apache.samza.system.kafka.KafkaSystemAdmin.
> createCoordinatorStream(KafkaSystemAdmin.scala:337)
>         at org.apache.samza.job.JobRunner.run(JobRunner.scala:88)
>         at org.apache.samza.job.JobRunner$.doOperation(JobRunner.scala:52)
>         at org.apache.samza.job.JobRunner$.main(JobRunner.scala:47)
>         at org.apache.samza.job.JobRunner.main(JobRunner.scala)
> Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.
> errors.TopicExistsException
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         ... 9 more
>
> On Mon, Nov 27, 2017 at 1:29 PM, Yi Pan <nickpan47@gmail.com> wrote:
>
> > Hi, Debraj,
> >
> > Makes sense. I have opened a ticket to upgrade Samza's Kafka client
> > libraries to 0.11 (SAMZA-1509). If you have concerns/suggestions, please
> > add comments to that ticket.
> >
> > Thanks!
> >
> > -Yi
> >
> > On Wed, Nov 22, 2017 at 10:04 AM, Debraj Manna <subharaj.manna@gmail.com
> >
> > wrote:
> >
> > > Since we were moving our brokers to 1.0 so was trying to have the same
> > > client version.
> > >
> > > Sent from GMail on Android
> > >
> > > On Nov 22, 2017 11:32 PM, "Yi Pan" <nickpan47@gmail.com> wrote:
> > >
> > > > And just one more question on your requirement for Kafka Client 1.0:
> is
> > > > this requirement because of your broker version is Kafka 1.0? Or you
> > want
> > > > some client side features from Kafka client 1.0?
> > > >
> > > > Just talked to our in-house Kafka expert and confirmed that Kafka
> > client
> > > > 0.10.1 should be able to talk to higher version of Kafka broker,
> > > including
> > > > 1.0, if the concern is on client/broker compatibility.
> > > >
> > > > -Yi
> > > >
> > > > On Wed, Nov 22, 2017 at 9:52 AM, Yi Pan <nickpan47@gmail.com> wrote:
> > > >
> > > > > Hi, Debraj,
> > > > >
> > > > > First, yes, this is the correct forum for user questions as well.
> > > > >
> > > > > Second, we are actively working on a patch to bring in Kafka 0.11
> > > client
> > > > > in Samza. It would require a lot of testing on Samza side to make
> > sure:
> > > > 1)
> > > > > back-compat w/ older version of brokers; 2) performance test. We
> > > already
> > > > > have a PR to include Kafka 0.11 under review.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > -Yi
> > > > >
> > > > > On Wed, Nov 22, 2017 at 6:04 AM, Debraj Manna <
> > > subharaj.manna@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Hi
> > > > >>
> > > > >>
> > > > >> I am facing the same below issue when trying to use kafka client
> 1.0
> > > > with
> > > > >> samza
> > > > >>
> > > > >> https://issues.apache.org/jira/browse/SAMZA-1418
> > > > >>
> > > > >> I could not find any related ticket with a fix.
> > > > >>
> > > > >> How are we supposed to run apache samza with the latest Kafka
> Client
> > > > 1.0?
> > > > >> It looks like I have to use kafka-clients-0.11.0.0.jar .
> > > > >>
> > > > >> *Versions*
> > > > >> Kafka Broker / Cluster - 1.0
> > > > >> samza-api-0.13.0.jar
> > > > >> samza-core_2.11-0.13.0.jar
> > > > >> samza-kafka_2.11-0.13.0.jar
> > > > >> samza-kv_2.11-0.13.0.jar
> > > > >> samza-yarn_2.11-0.13.0.jar
> > > > >>
> > > > >>
> > > > >> *PS - I could not find any users mailing list for samza in mailing
> > > list
> > > > >> <http://samza.apache.org/community/mailing-lists.html>
page. If
> > this
> > > is
> > > > >> not
> > > > >> the right forum to ask this query. Can you please point me to
the
> > > > correct
> > > > >> forum?*
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message