spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang, Ningjun (LNG-NPV)" <>
Subject How to force parallel processing of RDD using multiple thread
Date Thu, 15 Jan 2015 20:54:38 GMT
I have a standalone spark cluster with only one node with 4 CPU cores. How can I force spark
to do parallel processing of my RDD using multiple threads? For example I can do the following

Spark-submit  --master local[4]

However I really want to use the cluster as follow

Spark-submit  --master spark://

In that case, how can I make sure the RDD is processed with multiple threads/cores?


View raw message