spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anshu shukla <anshushuk...@gmail.com>
Subject Calculating tuple count /input rate with time
Date Tue, 23 Jun 2015 09:49:19 GMT
I am calculating input rate using the following logic.

And i think this foreachRDD is always running on driver (println are
seen on driver)

1- Is there any other way to do that in less cost .

2- Will this give me the correct count for rate  .


//code -

inputStream.foreachRDD(new Function<JavaRDD<String>, Void>() {
    @Override
    public Void call(JavaRDD<String> stringJavaRDD) throws Exception {
        System.out.println(System.currentTimeMillis()+",spoutstringJavaRDD,"
+ stringJavaRDD.count() );
        return null;
    }
});



-- 
Thanks & Regards,
Anshu Shukla

Mime
View raw message