spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brad Cox <>
Subject Re: spark.executor.cores
Date Fri, 15 Jul 2016 23:16:30 GMT
Mitch: could you elaborate on: You can practically run most of your unit testing with Local
mode and deploy variety of options including running SQL queries, reading data from CSV files,
writing to HDFS, creating Hive tables including ORC tables and doing Spark Streaming.

In particular, are you saying there's a way to run in local mode and still read/write HDFS
on a non-local cluster? Or do you mean run on the cluster as yarn-whichever, which is the
only way I've found that works to date?

Please elaborate. I'm shakey on how to aim spark at local files versus hdfs files without
a bunch of random experimenting around. 

Dr. Brad J. Cox    Cell: 703-594-1883 Skype: dr.brad.cox

To unsubscribe e-mail:

View raw message