spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Checkpointing is super slow
Date Fri, 02 Oct 2015 13:52:00 GMT
Why are you sure it's checkpointing speed?

Have you compared it against checkpointing to hdfs, s3, or local disk?

On Fri, Oct 2, 2015 at 1:17 AM, Sourabh Chandak <sourabh3934@gmail.com>
wrote:

> Hi,
>
> I have a receiverless kafka streaming job which was started yesterday
> evening and was running fine till 4 PM today. Suddenly post that writing of
> checkpoint has slowed down and it is now not able to catch up with the
> incoming data. We are using the DSE stack with Spark 1.2 and Cassandra for
> checkpointing. Spark streaming is done using a backported code.
>
> Running nodetool shows that the Read latency of the cfs keyspace is ~8.5
> ms.
>
> Can someone please help me resolve this?
>
> Thanks,
> Sourabh
>
>

Mime
View raw message