spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang, Ningjun (LNG-NPV)" <ningjun.w...@lexisnexis.com>
Subject How to force parallel processing of RDD using multiple thread
Date Thu, 15 Jan 2015 20:54:38 GMT
I have a standalone spark cluster with only one node with 4 CPU cores. How can I force spark
to do parallel processing of my RDD using multiple threads? For example I can do the following

Spark-submit  --master local[4]

However I really want to use the cluster as follow

Spark-submit  --master spark://10.125.21.15:7070

In that case, how can I make sure the RDD is processed with multiple threads/cores?

Thanks
Ningjun


Mime
View raw message