kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Holmberg <jan.holmb...@perigeum.fi>
Subject Kafka - Kudu streaming > unpredictable write times
Date Tue, 28 Mar 2017 15:24:52 GMT
I'm wondering the reason, why simple Spark prog. reading streaming data from Kafka and writing
result to Kudu, has unpredictable write times. In most cases, when running the prog, write
times are systematically 4 sec regardless of the number of messages (anything from 50 to 2000
messages per batch). But occasionally when starting the prog, it runs substantially faster
where write times are below 0,5 sec with exactly same code base, settings etc.

Our environment is plain AWS cluster with 3 slaves where each slave has Kafka and Kudu tablet
server instance with CDH 5.10 & Kudu 1.2  & Spark 1.6. 

Any hints what to look at?

cheers,
-jan
Mime
View raw message