kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pena Quijada Alexander <a.penaquij...@reply.it>
Subject R: Kafka offset problem when using Spark Streaming...
Date Sun, 13 May 2018 17:54:58 GMT
Hi all,

sorry for delay answer, I was a bit busy in the last few days at work. We have found the root
cause of our problem, we have had a network problem with one of our kafka brokers, in fact,
in that broker there weren't any active partitions anymore, all the partitions were in the
other brokers. We solved the network problem and we rebalanced the Kafka partitions, after
that, the problem evidenced in my last email has been fixed!

We're using Spark 2.0 and Kafka 0.9.

Thanks a lot for the kind cooperation!

Best regards,

Alex

-----Messaggio originale-----
Da: Ted Yu <yuzhihong@gmail.com>
Inviato: mercoledì 9 maggio 2018 17:58
A: users@kafka.apache.org
Oggetto: Re: Kafka offset problem when using Spark Streaming...

Can you give us more information:

release of Spark and Kafka you're using

anything interesting from broker / client logs - feel free to use pastebin to pass snippet
if needed

Thanks

On Wed, May 9, 2018 at 3:42 AM, Pena Quijada Alexander < a.penaquijada@reply.it> wrote:

> Hi all,
>
> We're facing some problems with ours Spark Streaming jobs, from
> yesterday we have got the following error into our logs when the jobs fail:
>
> java.lang.AssertionError: assertion failed: Beginning offset 562747 is
> after the ending offset 562743 for topic elk-topic partition 0.
>
> Any help about this issue will be appreciated!
>
> Many thanks in advance for the kind cooperation.
>
> Best regards,
>
> Alex
>
>
>
> ________________________________
>
> --
> The information transmitted is intended for the person or entity to
> which it is addressed and may contain confidential and/or privileged material.
> Any review, retransmission, dissemination or other use of, or taking
> of any action in reliance upon, this information by persons or
> entities other than the intended recipient is prohibited. If you
> received this in error, please contact the sender and delete the material from any computer.
>



________________________________

--
The information transmitted is intended for the person or entity to which it is addressed
and may contain confidential and/or privileged material. Any review, retransmission, dissemination
or other use of, or taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received this in error, please
contact the sender and delete the material from any computer.
Mime
View raw message