spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serkan TAS <Serkan....@enerjisa.com>
Subject RE: Spark application fail wit numRecords error
Date Thu, 02 Nov 2017 05:27:55 GMT
Hi,

I checked the following threads but i am still not sure if it is misuse, common o a bug.

https://stackoverflow.com/questions/34989539/spark-streaming-from-kafka-has-error-numrecords-must-not-be-negative

https://stackoverflow.com/questions/41319530/why-does-spark-streaming-application-with-kafka-fail-with-requirement-failed-n

https://forums.databricks.com/questions/11055/how-to-resolve-illegalargumentexception-requiremen.html



From: Prem Sure [mailto:sparksure542@gmail.com]
Sent: Wednesday, November 1, 2017 8:11 PM
To: Serkan TAS <Serkan.Tas@enerjisa.com>
Cc: user@spark.apache.org
Subject: Re: Spark application fail wit numRecords error

Hi, any offset left over for new topic consumption?, case can be the offset is beyond current
latest offset and cuasing negative.
hoping kafka brokers health is good and are up, this can also be a reason sometimes.

On Wed, Nov 1, 2017 at 11:40 AM, Serkan TAS <Serkan.Tas@enerjisa.com<mailto:Serkan.Tas@enerjisa.com>>
wrote:
Hi,

I searched the error in kafka but i think at last, it is related with spark not kafka.

Has anyone faced to an exception that is terminating program with error "numRecords must not
be negative" while streaming  ?

Thanx in advance.

Regards.

________________________________

Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler içerebilir. Şayet
mesajın gönderildiği kişi değilseniz, bu iletiyi çoğaltmak ve dağıtmak yasaktır.
Bu mesajı yanlışlıkla alan kişi, bu durumu derhal gönderene telefonla ya da e-posta
ile bildirmeli ve bilgisayarından silmelidir. Bu iletinin içeriğinden yalnızca iletiyi
gönderen kişi sorumludur.

This communication may contain information that is legally privileged, confidential or exempt
from disclosure. If you are not the intended recipient, please note that any dissemination,
distribution, or copying of this communication is strictly prohibited. Anyone who receives
this message in error should notify the sender immediately by telephone or by return communication
and delete it from his or her computer. Only the person who has sent this message is responsible
for its content.


________________________________

Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler içerebilir. Şayet
mesajın gönderildiği kişi değilseniz, bu iletiyi çoğaltmak ve dağıtmak yasaktır.
Bu mesajı yanlışlıkla alan kişi, bu durumu derhal gönderene telefonla ya da e-posta
ile bildirmeli ve bilgisayarından silmelidir. Bu iletinin içeriğinden yalnızca iletiyi
gönderen kişi sorumludur.

This communication may contain information that is legally privileged, confidential or exempt
from disclosure. If you are not the intended recipient, please note that any dissemination,
distribution, or copying of this communication is strictly prohibited. Anyone who receives
this message in error should notify the sender immediately by telephone or by return communication
and delete it from his or her computer. Only the person who has sent this message is responsible
for its content.
Mime
View raw message