spark-user mailing list archives: November 2016

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next »Thread · Author · Date
Stuart White Re: Best practice for preprocessing feature with DataFrame Thu, 17 Nov, 14:57
David Robison newAPIHadoopFile throws a JsonMappingException: Infinite recursion (StackOverflowError) error Thu, 17 Nov, 15:11
Stuart White Re: Best practice for preprocessing feature with DataFrame Thu, 17 Nov, 15:15
titli batali Fwd: Spark Partitioning Strategy with Parquet Thu, 17 Nov, 15:25
neil90 Re: Spark SQL join and subquery Thu, 17 Nov, 16:26
Sood, Anjali RE: Spark SQL join and subquery Thu, 17 Nov, 16:27
Georg Heiler Fill na with last value Thu, 17 Nov, 16:36
Arijit Re: Spark Streaming Data loss on failure to write BlockAdditionEvent failure to WAL Thu, 17 Nov, 16:48
Holden Karau Re: SparkILoop doesn't run Thu, 17 Nov, 16:53
Xiaomeng Wan Re: Spark Partitioning Strategy with Parquet Thu, 17 Nov, 16:58
Andrés Ivaldi Re: Grouping Set Thu, 17 Nov, 17:05
Dirceu Semighini Filho Re: Spark Streaming Data loss on failure to write BlockAdditionEvent failure to WAL Thu, 17 Nov, 17:18
Daniel Haviv Using mapWithState without a checkpoint Thu, 17 Nov, 17:45
geoHeil Fill nan with last (good) value Thu, 17 Nov, 17:57
Cody Koeninger Re: Kafka segmentation Thu, 17 Nov, 18:17
Mich Talebzadeh analysing ibm mq messages using spark streaming Thu, 17 Nov, 18:34
Hoang Bao Thien Re: Kafka segmentation Thu, 17 Nov, 18:48
Cody Koeninger Re: Kafka segmentation Thu, 17 Nov, 18:50
Hoang Bao Thien Re: Kafka segmentation Thu, 17 Nov, 18:53
Samy Dindane How to load only the data of the last partition Thu, 17 Nov, 19:05
Koert Kuipers replace some partitions when writing dataframe Thu, 17 Nov, 19:09
Jain, Nishit Spark AVRO S3 read not working for partitioned data Thu, 17 Nov, 19:11
Mohit Jaggi Re: SparkILoop doesn't run Thu, 17 Nov, 19:16
shyla deshpande Spark 2.0.2, Structured Streaming with kafka source... Unable to parse the value to Object.. Thu, 17 Nov, 19:30
Cesar does column order matter in dataframe.repartition? Thu, 17 Nov, 19:41
titli batali Re: Spark Partitioning Strategy with Parquet Thu, 17 Nov, 19:45
Daniel Haviv Re: How to load only the data of the last partition Thu, 17 Nov, 19:47
Sean Owen Re: does column order matter in dataframe.repartition? Thu, 17 Nov, 20:09
Jon Gregg Re: Spark AVRO S3 read not working for partitioned data Thu, 17 Nov, 20:14
KhajaAsmath Mohammed Spark Submit --> Unable to reach cluster manager to request executors Thu, 17 Nov, 20:15
kant kodali Re: How do I convert json_encoded_blob_column into a data frame? (This may be a feature request) Thu, 17 Nov, 20:24
Reynold Xin Re: How do I convert json_encoded_blob_column into a data frame? (This may be a feature request) Thu, 17 Nov, 20:44
Zsolt Tóth Map and MapParitions with partition-local variable Thu, 17 Nov, 20:57
shyla deshpande Re: Spark 2.0.2, Structured Streaming with kafka source... Unable to parse the value to Object.. Thu, 17 Nov, 21:05
Shixiong(Ryan) Zhu Re: Spark 2.0.2 with Kafka source, Error please help! Thu, 17 Nov, 21:07
Shixiong(Ryan) Zhu Re: HiveContext.getOrCreate not accessible Thu, 17 Nov, 21:34
ayan guha Re: Spark Partitioning Strategy with Parquet Thu, 17 Nov, 21:38
Koert Kuipers Re: Configure spark.kryoserializer.buffer.max at runtime does not take effect Thu, 17 Nov, 21:59
shyla deshpande Re: Spark 2.0.2 with Kafka source, Error please help! Thu, 17 Nov, 23:55
Hster Geguri kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Thu, 17 Nov, 23:58
Irina Truong Long-running job OOMs driver process Fri, 18 Nov, 01:51
Felix Cheung Re: How to propagate R_LIBS to sparkr executors Fri, 18 Nov, 02:15
Alexis Seigneurin Re: Long-running job OOMs driver process Fri, 18 Nov, 02:50
Rohit Verma Re: Map and MapParitions with partition-local variable Fri, 18 Nov, 03:30
fuz_woo GraphX Pregel not update vertex state properly, cause messages loss Fri, 18 Nov, 03:47
Rohit Verma Is selecting different datasets from same parquet file blocking. Fri, 18 Nov, 04:21
kant kodali Re: Configure spark.kryoserializer.buffer.max at runtime does not take effect Fri, 18 Nov, 05:56
Sreekanth Jella sort descending with multiple columns Fri, 18 Nov, 07:15
kant kodali How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Fri, 18 Nov, 07:42
Phillip Henry Re: org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 0 Fri, 18 Nov, 08:20
Samy Dindane Re: How to load only the data of the last partition Fri, 18 Nov, 09:11
kant kodali Re: How do I flatten JSON blobs into a Data Frame using Spark/Spark SQL Fri, 18 Nov, 10:29
Julian Keppel Kafka direct approach,App UI shows wrong input rate Fri, 18 Nov, 10:38
benoitdr RE: CSV to parquet preserving partitioning Fri, 18 Nov, 10:55
chrism Sporadic ClassNotFoundException with Kryo Fri, 18 Nov, 14:09
Keith Bourgoin Re: Long-running job OOMs driver process Fri, 18 Nov, 14:31
Kristoffer Sjögren DataFrame select non-existing column Fri, 18 Nov, 14:32
Stuart White Re: sort descending with multiple columns Fri, 18 Nov, 14:33
Steve Loughran Re: Any with S3 experience with Spark? Having ListBucket issues Fri, 18 Nov, 14:56
Steve Loughran Re: Long-running job OOMs driver process Fri, 18 Nov, 15:07
Nathan Lande Re: Long-running job OOMs driver process Fri, 18 Nov, 15:08
Anjali Gautam Issue in application deployment on spark cluster Fri, 18 Nov, 15:16
Alexis Seigneurin Re: Long-running job OOMs driver process Fri, 18 Nov, 15:17
Yong Zhang Re: Long-running job OOMs driver process Fri, 18 Nov, 15:30
Rabin Banerjee Will spark cache table once even if I call read/cache on the same table multiple times Fri, 18 Nov, 15:36
Keith Bourgoin Re: Long-running job OOMs driver process Fri, 18 Nov, 15:36
Rabin Banerjee Re: sort descending with multiple columns Fri, 18 Nov, 15:40
Yong Zhang Re: Will spark cache table once even if I call read/cache on the same table multiple times Fri, 18 Nov, 15:44
Rabin Banerjee Re: How to load only the data of the last partition Fri, 18 Nov, 15:48
Asmeet Re: Issue in application deployment on spark cluster Fri, 18 Nov, 15:55
Mich Talebzadeh Successful streaming with ibm/ mq to flume then to kafka and finally spark streaming Fri, 18 Nov, 16:53
Mendelson, Assaf RE: DataFrame select non-existing column Fri, 18 Nov, 19:03
Rabin Banerjee Re: Will spark cache table once even if I call read/cache on the same table multiple times Fri, 18 Nov, 19:16
Kristoffer Sjögren Re: DataFrame select non-existing column Fri, 18 Nov, 19:18
kant kodali How to expose Spark-Shell in the production? Fri, 18 Nov, 19:26
lminer Run spark with hadoop snapshot Fri, 18 Nov, 19:31
Mukesh Jha Spark driver not reusing HConnection Fri, 18 Nov, 22:37
Kürşat Kurt java.lang.OutOfMemoryError: Java heap space Sat, 19 Nov, 00:47
learning_spark Reading LZO files with Spark Sat, 19 Nov, 04:20
Muthu Jayakumar Re: DataFrame select non-existing column Sat, 19 Nov, 06:16
Mendelson, Assaf RE: DataFrame select non-existing column Sat, 19 Nov, 06:45
Sean Owen Re: Reading LZO files with Spark Sat, 19 Nov, 10:53
Steve Loughran Re: Run spark with hadoop snapshot Sat, 19 Nov, 12:48
Yuval.Itzchakov Stateful aggregations with Structured Streaming Sat, 19 Nov, 13:46
Kristoffer Sjögren Re: DataFrame select non-existing column Sat, 19 Nov, 14:56
Yanbo Liang Re: why is method predict protected in PredictionModel Sat, 19 Nov, 15:51
Cody Koeninger Re: Kafka segmentation Sat, 19 Nov, 16:17
Yanbo Liang Re: VectorUDT and ml.Vector Sat, 19 Nov, 16:18
Yanbo Liang Re: java.lang.OutOfMemoryError: Java heap space Sat, 19 Nov, 16:42
debasishg using StreamingKMeans Sat, 19 Nov, 16:46
Cody Koeninger Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 16:53
Yanbo Liang Re: Spark ML DataFrame API - need cosine similarity, how to convert to RDD Vectors? Sat, 19 Nov, 17:01
janardhan shetty Usage of mllib api in ml Sat, 19 Nov, 17:03
Cody Koeninger Re: Kafka direct approach,App UI shows wrong input rate Sat, 19 Nov, 17:06
Hster Geguri Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:12
vr spark covert local tsv file to orc file on distributed cloud storage(openstack). Sat, 19 Nov, 17:21
Cody Koeninger Re: using StreamingKMeans Sat, 19 Nov, 17:22
Cody Koeninger Re: Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:27
Hster Geguri Re: Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:56
Meeraj Kunnumpurath Logistic Regression Match Error Sat, 19 Nov, 18:10
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next »Thread · Author · Date
Box list
Jul 2021132
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137