spark-user mailing list archives: January 2018

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Michael Mansour Re: [EXT] How do I extract a value in foreachRDD operation Mon, 22 Jan, 17:25
KhajaAsmath Mohammed Production Critical : Data loss in spark streaming Mon, 22 Jan, 17:36
Toy Re: [EXT] How do I extract a value in foreachRDD operation Mon, 22 Jan, 17:50
Sathish Kumaran Vairavelu Re: Spark querying C* in Scala Mon, 22 Jan, 17:51
Weichen Xu Re: Has there been any explanation on the performance degradation between spark.ml and Mllib? Mon, 22 Jan, 19:13
Mich Talebzadeh Spark vs Snowflake Mon, 22 Jan, 21:51
Patrick McCarthy Re: Spark vs Snowflake Mon, 22 Jan, 22:09
Tathagata Das Re: [Spark structured streaming] Use of (flat)mapgroupswithstate takes long time Mon, 22 Jan, 23:03
KhajaAsmath Mohammed Spark Streaming data loss checkpoint directory Tue, 23 Jan, 02:27
David Rosenstrauch How to hold some data in memory while processing rows in a DataFrame? Tue, 23 Jan, 03:24
naresh Goud Re: How to hold some data in memory while processing rows in a DataFrame? Tue, 23 Jan, 03:41
vermanurag Re: How to hold some data in memory while processing rows in a DataFrame? Tue, 23 Jan, 04:26
Rohit Karlupia Spark Tuning Tool Tue, 23 Jan, 05:01
Roger Marin Re: Spark Tuning Tool Tue, 23 Jan, 05:02
Holden Karau Re: Spark Tuning Tool Tue, 23 Jan, 05:16
lucas.g...@gmail.com Re: Spark Tuning Tool Tue, 23 Jan, 05:21
Joe Wang Spark SQL bucket pruning support Tue, 23 Jan, 06:11
sd wang Re: run spark job in yarn cluster mode as specified user Tue, 23 Jan, 07:49
Christiaan Ras Re: [Spark structured streaming] Use of (flat)mapgroupswithstate takes long time Tue, 23 Jan, 10:45
igor.berman Re: external shuffle service in mesos Tue, 23 Jan, 11:13
Christiaan Ras [Structured streaming] Merging streaming with semi-static datasets Tue, 23 Jan, 11:32
Conconscious Re: Spark querying C* in Scala Tue, 23 Jan, 12:22
David Rosenstrauch Re: How to hold some data in memory while processing rows in a DataFrame? Tue, 23 Jan, 18:52
David Rosenstrauch Re: How to hold some data in memory while processing rows in a DataFrame? Tue, 23 Jan, 18:52
Toy I can't save DataFrame from running Spark locally Tue, 23 Jan, 19:33
Patrick Alwell Re: I can't save DataFrame from running Spark locally Tue, 23 Jan, 20:05
Toy Re: I can't save DataFrame from running Spark locally Tue, 23 Jan, 20:46
Vasyl Harasymiv S3 token times out during data frame "write.csv" Tue, 23 Jan, 22:58
Jörn Franke Re: S3 token times out during data frame "write.csv" Tue, 23 Jan, 23:03
Vasyl Harasymiv Re: S3 token times out during data frame "write.csv" Tue, 23 Jan, 23:09
manish ranjan Re: Spark Tuning Tool Tue, 23 Jan, 23:23
☼ R Nair (रविशंकर नायर) Re: Spark Tuning Tool Tue, 23 Jan, 23:34
Mich Talebzadeh Re: Spark Tuning Tool Wed, 24 Jan, 00:00
Stephen Joung write parquet with statistics min max with binary field Wed, 24 Jan, 01:30
Stephen Joung Re: write parquet with statistics min max with binary field Wed, 24 Jan, 01:31
Raj Adyanthaya Re: Spark Tuning Tool Wed, 24 Jan, 01:54
namesuperwood uncontinuous offset in kafka will cause the spark streaming failure Wed, 24 Jan, 05:48
Justin Miller Re: uncontinuous offset in kafka will cause the spark streaming failure Wed, 24 Jan, 06:23
namesuperwood Re: uncontinuous offset in kafka will cause the spark streamingfailure Wed, 24 Jan, 06:45
hsy...@gmail.com Question about accumulator Wed, 24 Jan, 06:46
hsy...@gmail.com Questions about using pyspark 2.1.1 pushing data to kafka Wed, 24 Jan, 07:02
namesuperwood Re: uncontinuous offset in kafka will cause the spark streamingfailure Wed, 24 Jan, 07:19
Michael Shtelma spark.sql call takes far too long Wed, 24 Jan, 12:16
lucas.g...@gmail.com Re: spark.sql call takes far too long Wed, 24 Jan, 15:09
Timothy Chen Re: Spark Tuning Tool Wed, 24 Jan, 17:02
Cody Koeninger Re: uncontinuous offset in kafka will cause the spark streamingfailure Wed, 24 Jan, 17:59
Shmuel Blitz Re: Spark Tuning Tool Wed, 24 Jan, 19:16
Tecno Brain Providing Kafka configuration as Map of Strings Wed, 24 Jan, 20:32
Tecno Brain Re: Providing Kafka configuration as Map of Strings Wed, 24 Jan, 20:48
Mutahir Ali Apache Hadoop and Spark Wed, 24 Jan, 20:50
Cody Koeninger Re: Providing Kafka configuration as Map of Strings Wed, 24 Jan, 22:31
Deepak Sharma CI/CD for spark and scala Thu, 25 Jan, 03:52
Sunita Arvind Re: a way to allow spark job to continue despite task failures? Thu, 25 Jan, 04:34
Fawze Abujaber Scala version changed in spark job Thu, 25 Jan, 06:40
Iain Cundy Kafka deserialization to Structured Streaming SQL - Encoders.bean result doesn't match itself? Thu, 25 Jan, 12:11
Jean Georges Perrin Re: S3 token times out during data frame "write.csv" Thu, 25 Jan, 13:10
jamison.bennett Re: Apache Hadoop and Spark Thu, 25 Jan, 14:58
Conconscious Custom build - missing images on MasterWebUI Thu, 25 Jan, 18:09
Margusja Get broadcast (set in one method) in another method Thu, 25 Jan, 20:04
Gourav Sengupta Re: Get broadcast (set in one method) in another method Thu, 25 Jan, 23:00
kant kodali how to create a DataType Object using the String representation in Java using Spark 2.2.0? Fri, 26 Jan, 00:22
Chandu Spark Standalone Mode, application runs, but executor is killed Fri, 26 Jan, 03:09
M Singh Apache Spark - Custom structured streaming data source Fri, 26 Jan, 04:36
kant kodali Re: how to create a DataType Object using the String representation in Java using Spark 2.2.0? Fri, 26 Jan, 06:39
Kurt Fehlhauer Re: how to create a DataType Object using the String representation in Java using Spark 2.2.0? Fri, 26 Jan, 07:11
Tathagata Das Re: Apache Spark - Custom structured streaming data source Fri, 26 Jan, 07:32
Esa Heikkinen Best active groups, forums or contacts for Spark ? Fri, 26 Jan, 11:15
Jacek Laskowski Re: Best active groups, forums or contacts for Spark ? Fri, 26 Jan, 11:56
Chandu Re: Best active groups, forums or contacts for Spark ? Fri, 26 Jan, 13:18
Rick Moritz Re: how to create a DataType Object using the String representation in Java using Spark 2.2.0? Fri, 26 Jan, 13:52
Marco Mistroni Re: Best active groups, forums or contacts for Spark ? Fri, 26 Jan, 14:08
Chandu Re: Spark Standalone Mode, application runs, but executor is killed Fri, 26 Jan, 14:34
Chandu Re: Spark Standalone Mode, application runs, but executor is killed Fri, 26 Jan, 14:35
Chandu Re: Spark Standalone Mode, application runs, but executor is killed Fri, 26 Jan, 14:36
Chandu Re: Spark Standalone Mode, application runs, but executor is killed Fri, 26 Jan, 14:41
Chandu Re: Best active groups, forums or contacts for Spark ? Fri, 26 Jan, 14:42
M Singh Re: Apache Spark - Custom structured streaming data source Fri, 26 Jan, 15:10
M Singh Apache Spark - Spark Structured Streaming - Watermark usage Fri, 26 Jan, 18:14
Jacek Laskowski Re: Apache Spark - Spark Structured Streaming - Watermark usage Fri, 26 Jan, 18:59
Antoine Bonnin Optimize sort merge join Sat, 27 Jan, 14:17
puneetloya Spark Streaming Cluster queries Sat, 27 Jan, 17:07
Franco Victorio Semi-supervised learning in MLlib Sat, 27 Jan, 17:29
CCInCharge Custom Catalyst Optimizer Strategy for DataFrame Writes? Sat, 27 Jan, 23:17
Gourav Sengupta Re: S3 token times out during data frame "write.csv" Sun, 28 Jan, 05:49
Jörn Franke Re: S3 token times out during data frame "write.csv" Sun, 28 Jan, 08:20
Nicolas Paris Re: Vectorized ORC Reader in Apache Spark 2.3 with Apache ORC 1.4.1. Sun, 28 Jan, 12:15
Dongjoon Hyun Re: Vectorized ORC Reader in Apache Spark 2.3 with Apache ORC 1.4.1. Sun, 28 Jan, 18:40
Stephen Joung Re: write parquet with statistics min max with binary field Mon, 29 Jan, 03:04
Richard Primera Spark Dataframe Writer _temporary directory Mon, 29 Jan, 05:24
韩盼 unsubscribe Mon, 29 Jan, 06:28
kant kodali How and when the types of the result set are figured out in Spark? Mon, 29 Jan, 06:52
ayan guha mapGroupsWithState in Python Mon, 29 Jan, 07:25
Nick Pentreath Re: Reverse MinMaxScaler in SparkML Mon, 29 Jan, 14:33
Michael Segel Re: Reading Hive RCFiles? Mon, 29 Jan, 15:32
KhajaAsmath Mohammed Spark Streaming checkpoint Mon, 29 Jan, 18:47
Arnav kumar Type Casting Error in Spark Data Frame Mon, 29 Jan, 21:26
Patrick McCarthy Re: Type Casting Error in Spark Data Frame Mon, 29 Jan, 21:42
Jean Georges Perrin Schema - DataTypes.NullType Mon, 29 Jan, 22:05
Jean Georges Perrin Schema - DataTypes.NullType Mon, 29 Jan, 22:07
Jean Georges Perrin Re: Type Casting Error in Spark Data Frame Mon, 29 Jan, 22:08
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Box list
Aug 201994
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137