spark-user mailing list archives: September 2015

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · 21 · 22 · Next »Thread · Author · Date
shenyan zhen Re: read compressed hdfs files using SparkContext.textFile? Tue, 08 Sep, 21:37
Richard Marscher Re: New to Spark - Paritioning Question Tue, 08 Sep, 21:38
Sandy Ryza Re: Spark on Yarn vs Standalone Tue, 08 Sep, 22:02
Понькин Алексей Re: [streaming] DStream with window performance issue Tue, 08 Sep, 22:33
Tathagata Das Re: Batchdurationmillis seems "sticky" with direct Spark streaming Tue, 08 Sep, 23:23
Mike Wright Re: New to Spark - Paritioning Question Tue, 08 Sep, 23:37
Dmitry Goldenberg Re: Batchdurationmillis seems "sticky" with direct Spark streaming Tue, 08 Sep, 23:38
Ulanov, Alexander RE: Spark ANN Wed, 09 Sep, 00:46
mark Re: Partitions with zero records & variable task times Wed, 09 Sep, 01:14
Nagaraj Chandrashekar Re: Support of other languages? Wed, 09 Sep, 01:40
Tathagata Das Re: Batchdurationmillis seems "sticky" with direct Spark streaming Wed, 09 Sep, 03:14
David Rosenstrauch Event logging not working when worker machine terminated Wed, 09 Sep, 03:15
Jeff Zhang Re: Event logging not working when worker machine terminated Wed, 09 Sep, 03:18
Tathagata Das Re: Exception when restoring spark streaming with batch RDD from checkpoint. Wed, 09 Sep, 03:24
Dmitry Goldenberg Re: Batchdurationmillis seems "sticky" with direct Spark streaming Wed, 09 Sep, 03:28
Tathagata Das Re: Getting Started with Spark Wed, 09 Sep, 03:40
Tathagata Das Re: Batchdurationmillis seems "sticky" with direct Spark streaming Wed, 09 Sep, 03:42
Nirmal Fernando Re: Applying transformations on a JavaRDD using reflection Wed, 09 Sep, 03:45
Dmitry Goldenberg Re: Batchdurationmillis seems "sticky" with direct Spark streaming Wed, 09 Sep, 04:02
Chintan Bhatt Contribution in Apche Spark Wed, 09 Sep, 04:20
Jörn Franke Re: Best way to import data from Oracle to Spark? Wed, 09 Sep, 04:31
Ruslan Dautkhanov Re: Best way to import data from Oracle to Spark? Wed, 09 Sep, 05:04
Alexander Pivovarov Re: Spark on Yarn vs Standalone Wed, 09 Sep, 05:48
Jeff Zhang Task serialization error for mllib.MovieLensALS Wed, 09 Sep, 06:14
Terry Hole Re: Meets "java.lang.IllegalArgumentException" when test spark ml pipe with DecisionTreeClassifier Wed, 09 Sep, 06:18
Akhil Das Re: Partitions with zero records & variable task times Wed, 09 Sep, 06:29
Akhil Das Re: No auto decompress in Spark Java textFile function? Wed, 09 Sep, 06:40
Akhil Das Re: foreachRDD causing executor lost failure Wed, 09 Sep, 07:00
Akhil Das Re: Contribution in Apche Spark Wed, 09 Sep, 07:04
李铖 How to read compressed parquet file Wed, 09 Sep, 07:29
Feynman Liang Re: Spark ANN Wed, 09 Sep, 07:55
Reynold Xin Re: Best way to import data from Oracle to Spark? Wed, 09 Sep, 08:40
Tom Seddon java.lang.NoSuchMethodError and yarn-client mode Wed, 09 Sep, 08:41
Aniket Bhatnagar Re: java.lang.NoSuchMethodError and yarn-client mode Wed, 09 Sep, 08:49
Ashish Dutt Re: hadoop2.6.0 + spark1.4.1 + python2.7.10 Wed, 09 Sep, 08:59
Cheng Lian Re: How to read compressed parquet file Wed, 09 Sep, 09:21
李铖 Re: How to read compressed parquet file Wed, 09 Sep, 09:45
Reynold Xin [ANNOUNCE] Announcing Spark 1.5.0 Wed, 09 Sep, 09:47
prachicsa I am very new to Spark. I have a very basic question. I have an array of values: listofECtokens: Array[String] = Array(EC-17A5206955089011B, EC-17A5206955089011A) I want to filter an RDD for all of these token values. I tried the following way: val ECtokens = for (token <- listofECtokens) rddAll.filter(line => line.contains(token)) Output: ECtokens: Unit = () I got an empty Unit even when there are records with these tokens. What am I doing wrong? Wed, 09 Sep, 09:55
prachicsa Filtering records for all values of an array in Spark Wed, 09 Sep, 09:58
szy I want to know the parition result in each node Wed, 09 Sep, 10:02
Akhil Das Re: I am very new to Spark. I have a very basic question. I have an array of values: listofECtokens: Array[String] = Array(EC-17A5206955089011B, EC-17A5206955089011A) I want to filter an RDD for all of these token values. I tried the following way: val ECtokens = for (token <- listofECtokens) rddAll.filter(line => line.contains(token)) Output: ECtokens: Unit = () I got an empty Unit even when there are records with these tokens. What am I doing wrong? Wed, 09 Sep, 10:13
Steve Loughran Re: How to read files from S3 from Spark local when there is a http proxy Wed, 09 Sep, 10:16
mark Re: Spark summit Asia Wed, 09 Sep, 10:24
Tom Seddon Re: java.lang.NoSuchMethodError and yarn-client mode Wed, 09 Sep, 10:28
Ted Yu Re: java.lang.NoSuchMethodError and yarn-client mode Wed, 09 Sep, 10:35
Ted Yu Re: I am very new to Spark. I have a very basic question. I have an array of values: listofECtokens: Array[String] = Array(EC-17A5206955089011B, EC-17A5206955089011A) I want to filter an RDD for all of these token values. I tried the following way: val ECtokens = for (token <- listofECtokens) rddAll.filter(line => line.contains(token)) Output: ECtokens: Unit = () I got an empty Unit even when there are records with these tokens. What am I doing wrong? Wed, 09 Sep, 10:43
Umesh Kacha Re: Why is huge data shuffling in Spark when using union()/coalesce(1,false) on DataFrame? Wed, 09 Sep, 11:10
Robin East Re: Applying transformations on a JavaRDD using reflection Wed, 09 Sep, 11:19
Sun, Rui RE: Support of other languages? Wed, 09 Sep, 11:52
Jeetendra Gangele bad substitution for [hdp.version] Error in spark on YARN job Wed, 09 Sep, 12:14
Tom Barber Help getting Spark JDBC metadata Wed, 09 Sep, 12:17
jarod7736 Re: long running Spark Streaming job and eventlog files Wed, 09 Sep, 13:27
Maximo Gurmendez Re: Partitioning a RDD for training multiple classifiers Wed, 09 Sep, 13:30
Tóth Zoltán Re: OutOfMemory error with Spark ML 1.5 logreg example Wed, 09 Sep, 13:40
Chris Teoh Re: No auto decompress in Spark Java textFile function? Wed, 09 Sep, 13:44
Cody Koeninger Re: [streaming] DStream with window performance issue Wed, 09 Sep, 14:04
shahab [Spark on Amazon EMR] : File does not exist: hdfs://ip-x-x-x-x:/.../spark-assembly-1.4.1-hadoop2.6.0-amzn-0.jar Wed, 09 Sep, 14:28
David Rosenstrauch Re: Event logging not working when worker machine terminated Wed, 09 Sep, 14:30
Richard Marscher Re: New to Spark - Paritioning Question Wed, 09 Sep, 14:49
Adrian Bridgett JNI issues with mesos Wed, 09 Sep, 14:59
Понькин Алексей Re: [streaming] DStream with window performance issue Wed, 09 Sep, 15:00
Cody Koeninger Re: Java vs. Scala for Spark Wed, 09 Sep, 15:00
Adrian Bridgett Re: JNI issues with mesos Wed, 09 Sep, 15:18
prachicsa Loading json data into Pair RDD in Spark using java Wed, 09 Sep, 15:50
Charles Chao Re: Event logging not working when worker machine terminated Wed, 09 Sep, 15:50
David Rosenstrauch Re: Event logging not working when worker machine terminated Wed, 09 Sep, 15:54
Charles Chao Re: Event logging not working when worker machine terminated Wed, 09 Sep, 15:56
michael.engl...@nomura.com spark history server + yarn log aggregation issue Wed, 09 Sep, 15:57
Marius Soutier spark.kryo.registrationRequired: Tuple2 is not registered Wed, 09 Sep, 16:00
Ted Yu Re: Loading json data into Pair RDD in Spark using java Wed, 09 Sep, 16:00
Tim Chen Re: JNI issues with mesos Wed, 09 Sep, 16:43
Sandy Ryza Driver OOM after upgrading to 1.5 Wed, 09 Sep, 17:10
Nicolas Monchy Spark Streaming checkpoints and code upgrade Wed, 09 Sep, 17:19
Thomas Gerber Cores per executors Wed, 09 Sep, 17:56
Bryan Jeffrey Problems with Local Checkpoints Wed, 09 Sep, 18:00
Jeetendra Gangele Re: bad substitution for [hdp.version] Error in spark on YARN job Wed, 09 Sep, 18:04
Rajeev Prasad Spark UI keep redirecting to /null and returns 500 Wed, 09 Sep, 18:26
Adrian Bridgett Re: JNI issues with mesos Wed, 09 Sep, 18:33
Reynold Xin Re: Driver OOM after upgrading to 1.5 Wed, 09 Sep, 18:37
Tathagata Das Re: Spark Streaming checkpoints and code upgrade Wed, 09 Sep, 18:40
Tathagata Das Re: Batchdurationmillis seems "sticky" with direct Spark streaming Wed, 09 Sep, 18:44
Dmitry Goldenberg Re: Batchdurationmillis seems "sticky" with direct Spark streaming Wed, 09 Sep, 19:03
Samya Spark streaming -> cassandra : Fault Tolerance Wed, 09 Sep, 19:09
unk1102 Spark rdd.mapPartitionsWithIndex() hits physical memory limit after huge data shuffle Wed, 09 Sep, 19:37
Cody Koeninger Re: Spark streaming -> cassandra : Fault Tolerance Wed, 09 Sep, 19:43
Maximo Gurmendez Re: Partitioning a RDD for training multiple classifiers Wed, 09 Sep, 19:51
Nicolas Monchy Re: Spark Streaming checkpoints and code upgrade Wed, 09 Sep, 20:42
Ted Yu Re: performance when checking if data frame is empty or not Wed, 09 Sep, 20:56
Sandy Ryza Re: Driver OOM after upgrading to 1.5 Wed, 09 Sep, 21:12
Dean Wampler Re: [Spark on Amazon EMR] : File does not exist: hdfs://ip-x-x-x-x:/.../spark-assembly-1.4.1-hadoop2.6.0-amzn-0.jar Wed, 09 Sep, 21:28
Reynold Xin Re: Driver OOM after upgrading to 1.5 Wed, 09 Sep, 21:31
Richard Marscher Re: What should be the optimal value for spark.sql.shuffle.partition? Wed, 09 Sep, 21:41
Richard Marscher Re: spark.shuffle.spill=false ignored? Wed, 09 Sep, 21:48
Burak Yavuz Re: Adding/subtracting org.apache.spark.mllib.linalg.Vector in Scala? Wed, 09 Sep, 21:53
Richard Marscher Re: What should be the optimal value for spark.sql.shuffle.partition? Wed, 09 Sep, 22:01
sethah Re: Spark MLlib Decision Tree Node Accuracy Wed, 09 Sep, 22:05
Thomas Dudziak Accumulator with non-java-serializable value ? Wed, 09 Sep, 22:18
sethah Re: Does Spark.ml LogisticRegression assumes only Double valued features? Wed, 09 Sep, 22:59
Ashish Shenoy ArrayIndexOutOfBoundsException when using repartitionAndSortWithinPartitions() Wed, 09 Sep, 23:45
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · 21 · 22 · Next »Thread · Author · Date
Box list
Jun 202174
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137