spark-user mailing list archives: November 2016

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · Next »Thread · Author · Date
Daniel Haviv Using mapWithState without a checkpoint Thu, 17 Nov, 17:45
geoHeil Fill nan with last (good) value Thu, 17 Nov, 17:57
Mich Talebzadeh analysing ibm mq messages using spark streaming Thu, 17 Nov, 18:34
Samy Dindane How to load only the data of the last partition Thu, 17 Nov, 19:05
Daniel Haviv   Re: How to load only the data of the last partition Thu, 17 Nov, 19:47
Samy Dindane     Re: How to load only the data of the last partition Fri, 18 Nov, 09:11
Rabin Banerjee       Re: How to load only the data of the last partition Fri, 18 Nov, 15:48
Koert Kuipers replace some partitions when writing dataframe Thu, 17 Nov, 19:09
Jain, Nishit Spark AVRO S3 read not working for partitioned data Thu, 17 Nov, 19:11
Jon Gregg   Re: Spark AVRO S3 read not working for partitioned data Thu, 17 Nov, 20:14
shyla deshpande Spark 2.0.2, Structured Streaming with kafka source... Unable to parse the value to Object.. Thu, 17 Nov, 19:30
shyla deshpande   Re: Spark 2.0.2, Structured Streaming with kafka source... Unable to parse the value to Object.. Thu, 17 Nov, 21:05
Michael Armbrust     Re: Spark 2.0.2, Structured Streaming with kafka source... Unable to parse the value to Object.. Tue, 22 Nov, 01:00
Cesar does column order matter in dataframe.repartition? Thu, 17 Nov, 19:41
Sean Owen   Re: does column order matter in dataframe.repartition? Thu, 17 Nov, 20:09
KhajaAsmath Mohammed Spark Submit --> Unable to reach cluster manager to request executors Thu, 17 Nov, 20:15
Hster Geguri kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Thu, 17 Nov, 23:58
Cody Koeninger   Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 16:53
Irina Truong Long-running job OOMs driver process Fri, 18 Nov, 01:51
Alexis Seigneurin   Re: Long-running job OOMs driver process Fri, 18 Nov, 02:50
Keith Bourgoin     Re: Long-running job OOMs driver process Fri, 18 Nov, 14:31
Steve Loughran       Re: Long-running job OOMs driver process Fri, 18 Nov, 15:07
Nathan Lande       Re: Long-running job OOMs driver process Fri, 18 Nov, 15:08
Alexis Seigneurin         Re: Long-running job OOMs driver process Fri, 18 Nov, 15:17
Yong Zhang           Re: Long-running job OOMs driver process Fri, 18 Nov, 15:30
Keith Bourgoin             Re: Long-running job OOMs driver process Fri, 18 Nov, 15:36
fuz_woo GraphX Pregel not update vertex state properly, cause messages loss Fri, 18 Nov, 03:47
rohit13k   Re: GraphX Pregel not update vertex state properly, cause messages loss Wed, 23 Nov, 17:15
rohit13k   Re: GraphX Pregel not update vertex state properly, cause messages loss Wed, 23 Nov, 18:09
rohit13k     Re: GraphX Pregel not update vertex state properly, cause messages loss Mon, 28 Nov, 10:55
Dale Wang   Re: GraphX Pregel not update vertex state properly, cause messages loss Thu, 24 Nov, 03:10
吴 郎     Re: GraphX Pregel not update vertex state properly, cause messages loss Thu, 24 Nov, 15:04
Rohit Verma Is selecting different datasets from same parquet file blocking. Fri, 18 Nov, 04:21
Sreekanth Jella sort descending with multiple columns Fri, 18 Nov, 07:15
Stuart White   Re: sort descending with multiple columns Fri, 18 Nov, 14:33
Rabin Banerjee     Re: sort descending with multiple columns Fri, 18 Nov, 15:40
Sreekanth Jella     Re: sort descending with multiple columns Mon, 21 Nov, 22:34
kant kodali How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Fri, 18 Nov, 07:42
kant kodali   Re: How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Fri, 18 Nov, 10:29
Michael Armbrust     Re: How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Tue, 22 Nov, 01:12
kant kodali       Re: How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Tue, 22 Nov, 05:34
Michael Armbrust         Re: How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Tue, 22 Nov, 22:42
kant kodali           Re: How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Wed, 23 Nov, 10:48
Michael Armbrust             Re: How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Tue, 29 Nov, 02:12
Re: org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 0
Phillip Henry   Re: org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 0 Fri, 18 Nov, 08:20
Julian Keppel Kafka direct approach,App UI shows wrong input rate Fri, 18 Nov, 10:38
Cody Koeninger   Re: Kafka direct approach,App UI shows wrong input rate Sat, 19 Nov, 17:06
Julian Keppel     Re: Kafka direct approach,App UI shows wrong input rate Tue, 22 Nov, 10:15
chrism Sporadic ClassNotFoundException with Kryo Fri, 18 Nov, 14:09
Kristoffer Sjögren DataFrame select non-existing column Fri, 18 Nov, 14:32
Mendelson, Assaf   RE: DataFrame select non-existing column Fri, 18 Nov, 19:03
Kristoffer Sjögren     Re: DataFrame select non-existing column Fri, 18 Nov, 19:18
Muthu Jayakumar       Re: DataFrame select non-existing column Sat, 19 Nov, 06:16
Mendelson, Assaf       RE: DataFrame select non-existing column Sat, 19 Nov, 06:45
Kristoffer Sjögren         Re: DataFrame select non-existing column Sat, 19 Nov, 14:56
Mendelson, Assaf           RE: DataFrame select non-existing column Sun, 20 Nov, 09:55
Kristoffer Sjögren             Re: DataFrame select non-existing column Sun, 20 Nov, 14:13
Mendelson, Assaf               RE: DataFrame select non-existing column Mon, 21 Nov, 07:36
Anjali Gautam Issue in application deployment on spark cluster Fri, 18 Nov, 15:16
Asmeet   Re: Issue in application deployment on spark cluster Fri, 18 Nov, 15:55
Rabin Banerjee Will spark cache table once even if I call read/cache on the same table multiple times Fri, 18 Nov, 15:36
Yong Zhang   Re: Will spark cache table once even if I call read/cache on the same table multiple times Fri, 18 Nov, 15:44
Rabin Banerjee     Re: Will spark cache table once even if I call read/cache on the same table multiple times Fri, 18 Nov, 19:16
Taotao.Li       Re: Will spark cache table once even if I call read/cache on the same table multiple times Sun, 20 Nov, 11:18
Yong Zhang       Re: Will spark cache table once even if I call read/cache on the same table multiple times Sun, 20 Nov, 15:32
Mich Talebzadeh Successful streaming with ibm/ mq to flume then to kafka and finally spark streaming Fri, 18 Nov, 16:53
kant kodali How to expose Spark-Shell in the production? Fri, 18 Nov, 19:26
Shreya Agarwal   RE: How to expose Spark-Shell in the production? Wed, 23 Nov, 23:28
lminer Run spark with hadoop snapshot Fri, 18 Nov, 19:31
Steve Loughran   Re: Run spark with hadoop snapshot Sat, 19 Nov, 12:48
Luke Miner     Re: Run spark with hadoop snapshot Sat, 19 Nov, 22:39
Mukesh Jha Spark driver not reusing HConnection Fri, 18 Nov, 22:37
Mukesh Jha   Re: Spark driver not reusing HConnection Mon, 21 Nov, 04:29
Mukesh Jha     Re: Spark driver not reusing HConnection Wed, 23 Nov, 20:55
Mukesh Jha       Re: Spark driver not reusing HConnection Wed, 23 Nov, 20:58
Kürşat Kurt java.lang.OutOfMemoryError: Java heap space Sat, 19 Nov, 00:47
Yanbo Liang   Re: java.lang.OutOfMemoryError: Java heap space Sat, 19 Nov, 16:42
Kürşat Kurt     RE: java.lang.OutOfMemoryError: Java heap space Sun, 20 Nov, 23:30
learning_spark Reading LZO files with Spark Sat, 19 Nov, 04:20
Sean Owen   Re: Reading LZO files with Spark Sat, 19 Nov, 10:53
Yuval.Itzchakov Stateful aggregations with Structured Streaming Sat, 19 Nov, 13:46
Michael Armbrust   Re: Stateful aggregations with Structured Streaming Tue, 22 Nov, 01:10
debasishg using StreamingKMeans Sat, 19 Nov, 16:46
Cody Koeninger   Re: using StreamingKMeans Sat, 19 Nov, 17:22
Debasish Ghosh     Re: using StreamingKMeans Sat, 19 Nov, 19:24
ayan guha       Re: using StreamingKMeans Sat, 19 Nov, 22:21
Debasish Ghosh         Re: using StreamingKMeans Sat, 19 Nov, 23:29
ayan guha           Re: using StreamingKMeans Sun, 20 Nov, 00:37
Debasish Ghosh             Re: using StreamingKMeans Sun, 20 Nov, 01:11
Julian Keppel               Re: using StreamingKMeans Mon, 21 Nov, 12:52
Usage of mllib api in ml
janardhan shetty   Usage of mllib api in ml Sat, 19 Nov, 17:03
Yanbo Liang     Re: Usage of mllib api in ml Sun, 20 Nov, 09:09
Marco Mistroni       Re: Usage of mllib api in ml Sun, 20 Nov, 12:52
janardhan shetty         Re: Usage of mllib api in ml Sun, 20 Nov, 16:28
Marco Mistroni           Re: Usage of mllib api in ml Sun, 20 Nov, 16:38
Hster Geguri Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:12
Cody Koeninger   Re: Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:27
Hster Geguri     Re: Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:56
vr spark covert local tsv file to orc file on distributed cloud storage(openstack). Sat, 19 Nov, 17:21
Steve Loughran   Re: covert local tsv file to orc file on distributed cloud storage(openstack). Sun, 20 Nov, 16:52
vr spark     Re: covert local tsv file to orc file on distributed cloud storage(openstack). Thu, 24 Nov, 18:44
Steve Loughran       Re: covert local tsv file to orc file on distributed cloud storage(openstack). Fri, 25 Nov, 19:07
Meeraj Kunnumpurath Logistic Regression Match Error Sat, 19 Nov, 18:10
Meeraj Kunnumpurath   Re: Logistic Regression Match Error Sat, 19 Nov, 18:59
Meeraj Kunnumpurath     Re: Logistic Regression Match Error Sat, 19 Nov, 19:10
Meeraj Kunnumpurath       Re: Logistic Regression Match Error Sat, 19 Nov, 19:26
Adam Smith HPC with Spark? Simultaneous, parallel one to one mapping of partition to vcore Sun, 20 Nov, 00:44
Stephen Boesch   Re: HPC with Spark? Simultaneous, parallel one to one mapping of partition to vcore Sun, 20 Nov, 00:52
Stuart White Create a Column expression from a String Sun, 20 Nov, 02:12
Luciano Resende   Re: Create a Column expression from a String Sun, 20 Nov, 03:11
Michael Armbrust   Re: Create a Column expression from a String Tue, 22 Nov, 00:56
Stuart White     Re: Create a Column expression from a String Tue, 22 Nov, 01:00
Mich Talebzadeh Using Flume as Input Stream to Spark Sun, 20 Nov, 10:12
Kappaganthu, Sivaram (ES) Error in running twitter streaming job Sun, 20 Nov, 11:09
Marco Mistroni   Re: Error in running twitter streaming job Sun, 20 Nov, 13:04
Kappaganthu, Sivaram (ES)     RE: Error in running twitter streaming job Sun, 20 Nov, 17:04
Marco Mistroni       RE: Error in running twitter streaming job Sun, 20 Nov, 18:03
Re: Flume integration
Mich Talebzadeh   Re: Flume integration Sun, 20 Nov, 11:30
Ian Brooks     Re: Flume integration Sun, 20 Nov, 16:40
Mich Talebzadeh       Re: Flume integration Sun, 20 Nov, 17:29
ayan guha     Re: Flume integration Sun, 20 Nov, 20:59
Mich Talebzadeh       Re: Flume integration Sun, 20 Nov, 21:29
ayan guha         Re: Flume integration Sun, 20 Nov, 22:06
Mich Talebzadeh           Re: Flume integration Sun, 20 Nov, 22:18
Ian Brooks       Re: Flume integration Mon, 21 Nov, 10:27
Mich Talebzadeh         Re: Flume integration Mon, 21 Nov, 10:37
Ian Brooks           Re: Flume integration Mon, 21 Nov, 10:52
shyla deshpande How do I access the nested field in a dataframe, spark Streaming app... Please help. Sun, 20 Nov, 17:59
pandees waran   Re: How do I access the nested field in a dataframe, spark Streaming app... Please help. Sun, 20 Nov, 18:24
Jon Gregg     Re: How do I access the nested field in a dataframe, spark Streaming app... Please help. Sun, 20 Nov, 21:10
shyla deshpande       Re: How do I access the nested field in a dataframe, spark Streaming app... Please help. Sun, 20 Nov, 21:44
Fwd: Yarn resource utilization with Spark pipe()
Sameer Choudhary   Fwd: Yarn resource utilization with Spark pipe() Sun, 20 Nov, 22:02
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · Next »Thread · Author · Date
Box list
Sep 2019100
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137