spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sateesh Kavuri <>
Subject Spark streaming multi-tasking during I/O
Date Fri, 21 Aug 2015 10:36:40 GMT

My scenario goes like this:
I have an algorithm running in Spark streaming mode on a 4 core virtual
machine. Majority of the time, the algorithm does disk I/O and database
I/O. Question is, during the I/O, where the CPU is not considerably loaded,
is it possible to run any other task/thread so as to efficiently utilize
the CPU?

Note that one DStream of the algorithm runs completely on a single CPU

Thank you,

View raw message